Abstract
Accurately predicting the Remaining Useful Life (RUL) of lithium-ion batteries is crucial for battery management systems. Deep learning-based methods have been shown to be effective in predicting RUL by leveraging battery capacity time series data. However, the representation learning of features such as long-distance sequence dependencies and mutations in capacity time series still needs to be improved. To address this challenge, this paper proposes a novel deep learning model, the MLP-Mixer and Mixture of Expert (MMMe) model, for RUL prediction. The MMMe model leverages the Gated Recurrent Unit and Multi-Head Attention mechanism to encode the sequential data of battery capacity to capture the temporal features and a re-zero MLP-Mixer model to capture the high-level features. Additionally, we devise an ensemble predictor based on a Mixture-of-Experts (MoE) architecture to generate reliable RUL predictions. The experimental results on public datasets demonstrate that our proposed model significantly outperforms other existing methods, providing more reliable and precise RUL predictions while also accurately tracking the capacity degradation process. Our code and dataset are available at the website of github.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Tang X, Liu K, Li K, Widanage W D, Kendrick E, Gao F. Recovering large-scale battery aging dataset with machine learning. Patterns, 2021, 2(8): 100302
Wang Z, Liu N, Guo Y. Adaptive sliding window LSTM NN based RUL prediction for lithium-ion batteries integrating LTSA feature reconstruction. Neurocomputing, 2021, 466: 178–189
Ge M F, Liu Y, Jiang X, Liu J. A review on state of health estimations and remaining useful life prognostics of lithium-ion batteries. Measurement, 2021, 174: 109057
Rauf H, Khalid M, Arshad N. Machine learning in state of health and remaining useful life estimation: theoretical and technological development in battery degradation modelling. Renewable and Sustainable Energy Reviews, 2022, 156: 111903
Zhai Q, Ye Z S. RUL prediction of deteriorating products using an adaptive wiener process model. IEEE Transactions on Industrial Informatics, 2017, 13(6): 2911–2921
Wang Y, Tian J, Sun Z, Wang L, Xu R, Li M, Chen Z. A comprehensive review of battery modeling and state estimation approaches for advanced battery management systems. Renewable and Sustainable Energy Reviews, 2020, 131: 110015
Deng Z, Yang L, Deng H, Cai Y, Li D. Polynomial approximation pseudo-two-dimensional battery model for online application in embedded battery management system. Energy, 2018, 142: 838–850
Yang L, Cai Y, Yang Y, Deng Z. Supervisory long-term prediction of state of available power for lithium-ion batteries in electric vehicles. Applied Energy, 2020, 257: 114006
Son J, Zhou S, Sankavaram C, Du X, Zhang Y. Remaining useful life prediction based on noisy condition monitoring signals using constrained Kalman filter. Reliability Engineering & System Safety, 2016, 152: 38–50
Su X, Wang S, Pecht M, Zhao L, Ye Z. Interacting multiple model particle filter for prognostics of lithium-ion batteries. Microelectronics Reliability, 2017, 70: 59–69
Tian J, Xu R, Wang Y, Chen Z. Capacity attenuation mechanism modeling and health assessment of lithium-ion batteries. Energy, 2021, 221: 119682
Li Y, Liu K, Foley A M, Zülke A, Berecibar M, Nanini-Maury E, Van Mierlo J, Hoster H E. Data-driven health estimation and lifetime prediction of lithium-ion batteries: a review. Renewable and Sustainable Energy Reviews, 2019, 113: 109254
Li S, Fang H, Shi B. Remaining useful life estimation of lithium-ion battery based on interacting multiple model particle filter and support vector regression. Reliability Engineering & System Safety, 2021, 210: 107542
Shu X, Li G, Shen J, Lei Z, Chen Z, Liu Y. A uniform estimation framework for state of health of lithium-ion batteries considering feature extraction and parameters optimization. Energy, 2020, 204: 117957
Liu Z, Cheng Y, Wang P, Yu Y, Long Y. A method for remaining useful life prediction of crystal oscillators using the Bayesian approach and extreme learning machine under uncertainty. Neurocomputing, 2018, 305: 27–38
Shen D, Wu L, Kang G, Guan Y, Peng Z. A novel online method for predicting the remaining useful life of lithium-ion batteries considering random variable discharge current. Energy, 2021, 218: 119490
Yang B, Liu R, Zio E. Remaining useful life prediction based on a double-convolutional neural network architecture. IEEE Transactions on Industrial Electronics, 2019, 66(12): 9521–9530
Ding P, Liu X, Li H, Huang Z, Zhang K, Shao L, Abedinia O. Useful life prediction based on wavelet packet decomposition and two-dimensional convolutional neural network for lithium-ion batteries. Renewable and Sustainable Energy Reviews, 2021, 148: 111287
Zhang Y, Xiong R, He H, Pecht M G. Long short-term memory recurrent neural network for remaining useful life prediction of lithiumion batteries. IEEE Transactions on Vehicular Technology, 2018, 67(7): 5695–5705
Zhao S, Zhang C, Wang Y. Lithium-ion battery capacity and remaining useful life prediction using board learning system and long short-term memory neural network. Journal of Energy Storage, 2022, 52: 104901
Chen D, Hong W, Zhou X. Transformer network for remaining useful life prediction of lithium-ion batteries. IEEE Access, 2022, 10: 19621–19628
Zheng L, He Y, Chen X, Pu X. Optimization of dilated convolution networks with application in remaining useful life prediction of induction motors. Measurement, 2022, 200: 111588
Ragab M, Chen Z, Wu M, Kwoh C K, Yan R, Li X. Attention-based sequence to sequence model for machine remaining useful life prediction. Neurocomputing, 2021, 466: 58–68
Wu J Y, Wu M, Chen Z, Li X L, Yan R. Degradation-aware remaining useful life prediction with LSTM autoencoder. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1–10
Jin R, Chen Z, Wu K, Wu M, Li X, Yan R. Bi-LSTM-based two-stream network for machine remaining useful life prediction. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1–10
Xiao L, Zhang L, Niu F, Su X, Song W. RETRACTED: remaining useful life prediction of wind turbine generator based on 1D-CNN and Bi-LSTM. International Journal of Fatigue, 2022, 163: 107051
Rouhi Ardeshiri R, Ma C. Multivariate gated recurrent unit for battery remaining useful life prediction: a deep learning approach. International Journal of Energy Research, 2021, 45(11): 16633–16648
Chen Z, Wu M, Zhao R, Guretno F, Yan R, Li X. Machine remaining useful life prediction via an attention-based deep learning approach. IEEE Transactions on Industrial Electronics, 2021, 68(3): 2521–2531
Tolstikhin I O, Houlsby N, Kolesnikov A, Beyer L, Zhai X, Unterthiner T, Yung J, Steiner A, Keysers D, Uszkoreit J, Lucic M, Dosovitskiy A. MLP-Mixer: an all-MLP architecture for vision. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021
Touvron H, Bojanowski P, Caron M, Cord M, El-Nouby A, Grave E, Izacard G, Joulin A, Synnaeve G, Verbeek J, Jegou H. ResMLP: feedforward networks for image classification with data-efficient training. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(4): 5314–5321
Chen S, Xie E, Ge C, Chen R, Liang D, Luo P. CycleMLP: a MLP-like architecture for dense prediction. In: Proceedings of the 10th International Conference on Learning Representations. 2022
Yu T, Li X, Cai Y, Sun M, Li P. S2-MLP: spatial-shift MLP architecture for vision. In: Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2022, 3615–3624
Jacobs R A, Jordan M I, Nowlan S J, Hinton G E. Adaptive mixtures of local experts. Neural Computation, 1991, 3(1): 79–87
Liu H, Liu Z, Jia W, Lin X. Remaining useful life prediction using a novel feature-attention-based end-to-end approach. IEEE Transactions on Industrial Informatics, 2021, 17(2): 1197–1207
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000–6010
Wu Y, Li W, Wang Y, Zhang K. Remaining useful life prediction of lithium-ion batteries using neural network and bat-based particle filter. IEEE Access, 2019, 7: 54843–54854
Liu J, Saxena A, Goebel K, Saha B, Wang W. An adaptive recurrent neural network for remaining useful life prediction of lithium-ion batteries. In: Proceedings of Annual Conference of the Prognostics and Health Management Society. 2010
Williard N, He W, Osterman M, Pecht M. Comparative analysis of features for determining state of health in lithium-ion batteries. International Journal of Prognostics and Health Management, 2013, 4(1): 1–7
Shi Z, Chehade A. A dual-LSTM framework combining change point detection and remaining useful life prediction. Reliability Engineering & System Safety, 2021, 205: 107257
Nagulapati V M, Lee H, Jung D, Brigljevic B, Choi Y, Lim H. Capacity estimation of batteries: influence of training dataset size and diversity on data driven prognostic models. Reliability Engineering & System Safety, 2021, 216: 108048
Acknowledgements
We are very grateful to the anonymous reviewers for their effort in evaluating our paper. This work was supported by the National Natural Science Foundation of China (Grant Nos. 62102191, 61872114, and 61871020).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Competing interests The authors declare that they have no competing interests or financial conflicts to disclose.
Additional information
Lingling Zhao is an associate professor at Faculty of Computing, Harbin Institute of Technology, China. She received the PhD, MS, and BS degrees from Harbin Institute of Technology, China. Her current research interests include machine learning.
Shitao Song is currently a postgraduate student in the School of Electrical Engineering, Liaoning University of Technology, China. His research interest includes machine learning and micro-grid scheduling optimization.
Pengyan Wang received his BE degree in Computer Science and Technology from Northeast Electric Power University, China in 2021. Currently, he is pursuing a ME degree in Computer Science and Technology at Northeast Electric Power University, China. His research interest includes lithium-ion battery safety.
Chunyu Wang is a professor at the Faculty of Computing, Harbin Institute of Technology, China. He received his BS, MS, and PhD degrees in computer science and technology from Harbin Institute of Technology, China. His current research interests include bioinformatics and machine learning.
Junjie Wang received the BS degree in Information management and information system from Institute of Disaster Prevention in 2013, the MS degree in software engineering in 2015, and the PhD degree in Computer science and technology in 2020 from the Harbin Institute of Technology, China. He is a Lecturer with the School of Biomedical Engineering and Informatics, Nanjing Medical University, China. His current research interests include bioinformatics and deep learning.
Maozu Guo received the PhD degree from the School of Computer Science and Technology, Harbin Institute of Technology, China. He is currently a Professor with the School of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, China. His current research interests include machine learning and artificial intelligence.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zhao, L., Song, S., Wang, P. et al. A MLP-Mixer and mixture of expert model for remaining useful life prediction of lithium-ion batteries. Front. Comput. Sci. 18, 185329 (2024). https://doi.org/10.1007/s11704-023-3277-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11704-023-3277-4