Federated Transfer Learning Strategy: A Novel Cross-Device Fault Diagnosis Method Based on Repaired Data
Abstract
:1. Introduction
- A novel federated learning strategy is proposed to solve the problem that the source client lacks complete samples for network training, which rarely occurs in current federated transfer learning research.
- The joint function proposed for optimizing source-client networks in federated transfer learning strategies employs Wasserstein distance and multi-kernel MMD to measure domain distances and effectively alleviates the model-negative transfer phenomenon caused by distribution discrepancies through periodic training.
- To address the challenge of diagnosing targets across different devices and under varying working conditions, an adaptive global model update method is employed by the central server. This approach ensures excellent fault diagnosis performance while safeguarding source client data privacy.
2. Materials and Methods
2.1. Federated Learning
2.2. Transfer Learning
2.3. Random Forest
3. Proposed Federated Transfer Learning Scheme
3.1. Network Architecture and Training Initialization
3.2. Source-Client Periodic Training
3.3. Federated Learning Dynamic Interaction
4. Experimental Verification
4.1. Dataset Description
4.1.1. CWRU
4.1.2. MDS
4.1.3. GPTFS
4.2. Different Comparison Schemes
4.3. Cross-Machine Federated Transfer Learning Tasks and Parameters Setting
4.4. Diagnosis Result and Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Xiao, Y.; Shao, H.; Feng, M.; Han, T.; Wan, J.; Liu, B. Towards trustworthy rotating machinery fault diagnosis via attention uncertainty in Transformer. J. Manuf. Syst. 2023, 70, 186–201. [Google Scholar] [CrossRef]
- Wang, D.; Chen, Y.; Shen, C.; Zhong, J.; Peng, Z.; Li, C. Fully interpretable neural network for locating resonance frequency bands for machine condition monitoring. Mech. Syst. Signal Process. 2022, 168, 108673. [Google Scholar] [CrossRef]
- Wang, Z.; She, Q.; Ward, T. Generative adversarial networks in computer vision: A survey and taxonomy. ACM Comput. Surv. (CSUR) 2021, 54, 1–38. [Google Scholar] [CrossRef]
- Otter, D.; Medina, J.; Kalita, J. A survey of the usages of deep learning for natural language processing. IEEE Trans. Neural Netw. Learn. Syst. 2020, 32, 604–624. [Google Scholar] [CrossRef] [PubMed]
- Lei, Y.; Yang, B.; Jiang, X.; Jia, F.; Li, N.; Nandi, A.K. Applications of machine learning to machine fault diagnosis: A review and roadmap. Mech. Syst. Signal Process. 2020, 138, 106587. [Google Scholar] [CrossRef]
- Yan, S.; Shao, H.; Min, Z.; Peng, J.; Cai, B.; Liu, B. FGDAE: A new machinery anomaly detection method towards complex operating conditions. Reliab. Eng. Syst. Saf. 2023, 236, 109319. [Google Scholar] [CrossRef]
- Jia, S.; Li, Y.; Mao, G.; Noman, K. Multi-representation symbolic convolutional neural network: A novel multisource cross-domain fault diagnosis method for rotating system. Struct. Health Monit. 2023, 2023, 14759217231157487. [Google Scholar] [CrossRef]
- Zhang, W.; Li, X. Data privacy preserving federated transfer learning in machinery fault diagnostics using prior distributions. Struct. Health Monit. 2022, 21, 1329–1344. [Google Scholar] [CrossRef]
- Kaissis, G.A.; Makowski, M.R.; Rückert, D.; Braren, R.F. Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2020, 2, 305–311. [Google Scholar] [CrossRef]
- Khan, L.U.; Saad, W.; Han, Z.; Hossain, E.; Hong, C.S. Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges. IEEE Commun. Surv. Tutor. 2021, 23, 1759–1799. [Google Scholar] [CrossRef]
- Rasha, A.H.; Li, T.; Huang, W.; Gu, J.; Li, C. Federated learning in smart cities: Privacy and security survey. Inf. Sci. 2023, 632, 833–857. [Google Scholar]
- McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; y Arcas, B.A. Communication-Efficient Learning of Deep Networks from Decentralized Data. Artif. Intell. Stat. 2017, 54, 1273–1282. [Google Scholar]
- Li, Q.; He, B.; Song, D. Model-Contrastive Federated Learning. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Kuala Lumpur, 18 December 2021; pp. 10713–10722. [Google Scholar]
- Marfoq, O.; Neglia, G.; Bellet, A.; Kameni, L.; Vidal, R. Federated Multi-Task Learning under a Mixture of Distributions. Adv. Neural Inf. Process. Syst. 2021, 34, 15434–15447. [Google Scholar]
- Zhu, H.; Jin, Y. Multi-Objective Evolutionary Federated Learning. IEEE Trans. Neural Netw. Learn. Syst. 2020, 31, 1310–1322. [Google Scholar] [CrossRef]
- Li, C.; Zhang, S.; Qin, Y.; Estupinan, E. A systematic review of deep transfer learning for machinery fault diagnosis. Neurocomputing 2020, 407, 121–135. [Google Scholar] [CrossRef]
- Zhang, S.; Lei, S.U.; Jiefei, G.U.; Ke, L.I.; Lang, Z.H.O.U.; Pecht, M. Rotating machinery fault detection and diagnosis based on deep domain adaptation: A survey. Chin. J. Aeronaut. 2023, 36, 45–74. [Google Scholar] [CrossRef]
- Chen, X.; Shao, H.; Xiao, Y.; Yan, S.; Cai, B.; Liu, B. Collaborative fault diagnosis of rotating machinery via dual adversarial guided unsupervised multi-domain adaptation network. Mech. Syst. Signal Process. 2023, 198, 110427. [Google Scholar] [CrossRef]
- Li, B.; Tang, B.; Deng, L.; Wei, J. Joint attention feature transfer network for gearbox fault diagnosis with imbalanced data. Mech. Syst. Signal Process. 2022, 176, 109146. [Google Scholar] [CrossRef]
- Abreha, H.; Hayajneh, M.; Serhani, M. Federated Learning in Edge Computing: A Systematic Survey. Sensors 2022, 22, 450. [Google Scholar] [CrossRef]
- Cui, Y.; Li, Y.; Ye, C. Sample-Based and Feature-Based Federated Learning for Unconstrained and Constrained Nonconvex Optimization via Mini-batch SSCA. IEEE Trans. Signal Process. 2022, 70, 3832–3847. [Google Scholar] [CrossRef]
- Li, T.; Sahu, A.K.; Talwalkar, A.; Smith, V. Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Process. Mag. 2020, 37, 50–60. [Google Scholar] [CrossRef]
- Liu, Y.; Kang, Y.; Xing, C.; Chen, T.; Yang, Q. A Secure Federated Transfer Learning Framework. Intell. Syst. 2020, 35, 70–82. [Google Scholar] [CrossRef]
- Lee, W. Reward-based participant selection for improving federated reinforcement learning. ICT Express 2022. [Google Scholar] [CrossRef]
- Li, Y.; Cui, Y.; Lau, V. An optimization framework for federated edge learning. IEEE Trans. Wirel. Commun. 2022, 22, 934–949. [Google Scholar] [CrossRef]
- Sharma, S.; Xing, C.; Liu, Y.; Kang, Y. Secure and efficient federated transfer learning. In Proceedings of the 2019 IEEE International Conference on Big Data, Los Angeles, CA, USA, 9–12 December 2019; pp. 2569–2576. [Google Scholar]
- Zhang, T.; Chen, J.; Li, F.; Zhang, K.; Lv, H.; He, S.; Xu, E. Intelligent fault diagnosis of machines with small & imbalanced data: A state-of-the-art review and possible extensions. ISA Trans. 2022, 119, 152–171. [Google Scholar]
- Iman, M.; Arabnia, H.; Rasheed, K. A review of deep transfer learning and recent advancements. Technologies 2023, 11, 40. [Google Scholar] [CrossRef]
- Liu, S.; Wang, H.; Tang, J.; Zhang, X. Research on fault diagnosis of gas turbine rotor based on adversarial discriminative domain adaption transfer learning. Measurement 2022, 196, 111174. [Google Scholar] [CrossRef]
- He, Y.; Tang, H.; Ren, Y.; Kumar, A. A deep multi-signal fusion adversarial model based transfer learning and residual network for axial piston pump fault diagnosis. Measurement 2022, 192, 110889. [Google Scholar] [CrossRef]
- Cerrada, M.; Zurita, G.; Cabrera, D.; Sánchez, R.V.; Artés, M.; Li, C. Fault diagnosis in spur gears based on genetic algorithm and random forest. Mech. Syst. Signal Process. 2016, 70, 87–103. [Google Scholar] [CrossRef]
- Tang, G.; Pang, B.; Tian, T.; Zhou, C. Fault diagnosis of rolling bearings based on improved fast spectral correlation and optimized random forest. Appl. Sci. 2018, 8, 1859. [Google Scholar] [CrossRef]
- Segal, M.R. Machine Learning Benchmarks and Random Forest Regression; Center for Bioinformatics & Molecular Biostatistics: Sichuan, China, 2004. [Google Scholar]
- Borup, D.; Christensen, B.J.; Mühlbach, N.S.; Nielsen, M.S. Targeting predictors in random forest regression. Soc. Sci. Electron. Publ. 2020, 39, 841–868. [Google Scholar] [CrossRef]
- Bao, H.; Yan, Z.; Ji, S.; Wang, J.; Jia, S.; Zhang, G.; Han, B. An enhanced sparse filtering method for transfer fault diagnosis using maximum classifier discrepancy. Meas. Sci. Technol. 2021, 32, 085105. [Google Scholar] [CrossRef]
- Zhang, W.; Li, X.; Ma, H.; Luo, Z.; Li, X. Federated learning for machinery fault diagnosis with dynamic validation and self-supervision. Knowl. Based Syst. 2021, 213, 106679. [Google Scholar] [CrossRef]
- Zhang, W.; Li, X. Federated Transfer Learning for Intelligent Fault Diagnostics Using Deep Adversarial Networks with Data Privacy. IEEE/ASME Trans. Mechatron. 2021, 27, 430–439. [Google Scholar] [CrossRef]
- Maaten, L.; Hinton, G. Visualizing Data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2605. [Google Scholar]
Dataset | Object Types | Working Conditions | Health Types | No. of Samples | |
---|---|---|---|---|---|
Code | Load/Speed | ||||
CWRU | SKF6205 | A | 0 hp/1797 rpm | 1 Normal 3 Inner Race 3 Outer Race | 100 3 × 100 3 × 100 |
B | 1 hp/1772 rpm | ||||
C | 2 hp/1750 rpm | ||||
D | 3 hp/1730 rpm | ||||
MDS | NU205EM | E | 1000 rpm | 1 Normal | 100 |
F | 1300 rpm | 1 Inner Race | 100 | ||
G | 1500 rpm | 1 Outer Race | 100 | ||
GPTFS | NU205EM | H | 1000 rpm | 1 Normal 3 Inner Race 3 Outer Race | 100 3 × 100 3 × 100 |
I | 1500 rpm | ||||
J | 20 N/1500 rpm | ||||
K | 2000 rpm | ||||
L | 20 N/2000 rpm |
Task | Client #1 | Client #2 | Client #3 | Target Client | |
---|---|---|---|---|---|
Case 1 (3 types) | Source | C (CWRU-1750) | F | K | C, I (CWRU-1750 mixed with GPTFS-1500) |
Ideal Data | Ideal Data | Ideal Data | |||
Client task | B (CWRU-1772) | G | I | ||
Case 2 (3 types) | Source | C | F (MDS-1300) | K | C, I |
Repair Data (25%) | Repair Data (25%) | Repair Data (25%) | |||
Client task | B | G (MDS-1500) | I | ||
Case 3 (3 types) | Source | C | F | K (GPTFS-2000) | C, I |
Repair Data (37.5%) | Repair Data (37.5%) | Repair Data (37.5%) | |||
Client task | B | G | I (GPTFS-1500) | ||
Case 4 (7 types) | Source | C | K | J (GPTFS-20/1500) | D, J (CWRU-1730 mixed with GPTFS-20/1500) |
Repair Data (25%) | Repair Data (25%) | Repair Data (25%) | |||
Client task | D (CWRU-1730) | I | L (GPTFS-20/2000) |
Parameter | Value | Parameter | Value |
---|---|---|---|
Source_input | 1200 | The number of decision trees | 100 |
Target_input | 1200 | The number of leaves | 5 |
Classification_input | 150 | 15 | |
Sample_size | 100 | 1 | |
Label_1 | 7 | The number of source clients K | 3 |
Label_2 | 3 | Experience coefficient | 0.5 |
Learning-rate | 0.0005 | Experience coefficient | 0.5 |
Sample_size | 100 | Experience coefficient | 4 |
Federation dynamic interaction cycle | 20 | Local training cycle | 100 |
Client Task | MAE | MAPE | MBE | RMSE | R2 | ||
---|---|---|---|---|---|---|---|
Case 2 (25%) | Client 1 | CWRU-NC | 0.01684 | 0.03367 | 0.00084 | 0.02243 | 0.87892 |
CWRU-IRF3 | 0.08165 | 0.00211 | 0.00112 | 0.11341 | 0.98029 | ||
CWRU-ORF3 | 0.0298 | 0.01867 | 0.00195 | 0.0432 | 0.95764 | ||
Client 2 | MDS-NC | 0.21635 | 0.05039 | 0.02098 | 0.27182 | 0.73556 | |
MDS-IRF3 | 0.49061 | 0.10457 | −0.04878 | 0.37851 | 0.65363 | ||
MDS-ORF3 | 1.0169 | 0.0381 | −0.04772 | 1.7058 | 0.84441 | ||
Client 3 | GPTFS-NC | 2.2796 | 0.068 | −0.11896 | 2.867 | 0.66156 | |
GPTFS-IRF3 | 2.7295 | 0.08345 | −0.01088 | 3.8418 | 0.68568 | ||
GPTFS-ORF3 | 2.0915 | 0.10786 | 0.20306 | 2.7645 | 0.63849 | ||
Case 3 (37.5%) | Client 1 | CWRU-NC | 0.01631 | 0.02180 | 0.00057 | 0.02165 | 0.8835 |
CWRU-IRF3 | 0.08739 | 0.00142 | −0.00014 | 0.11952 | 0.97492 | ||
CWRU-ORF3 | 0.02895 | 0.01 | 0.00262 | 0.03942 | 0.9602 | ||
Client 2 | MDS-NC | 0.30154 | 0.05548 | −0.02627 | 0.42285 | 0.64347 | |
MDS-IRF3 | 0.32726 | 0.06213 | −0.00511 | 0.41679 | 0.60667 | ||
MDS-ORF3 | 1.1004 | 0.02657 | −0.02394 | 1.7745 | 0.80962 | ||
Client 3 | GPTFS-NC | 2.2504 | 0.048 | 0.06712 | 2.8307 | 0.65488 | |
GPTFS-IRF3 | 2.7417 | 0.0568 | −0.031 | 3.7691 | 0.68315 | ||
GPTFS-ORF3 | 1.9348 | 0.0709 | −0.0393 | 2.5791 | 0.6464 | ||
Case 4 (25%) | Client 1 | CWRU-IRF1 | 0.05333 | 0.02967 | −0.00562 | 0.08707 | 0.90167 |
CWRU-IRF2 | 0.01672 | 0.00638 | 0.00046 | 0.02179 | 0.96908 | ||
CWRU-ORF1 | 0.10435 | 0.00504 | −0.00899 | 0.15351 | 0.97724 | ||
CWRU-ORF2 | 0.02529 | 0.02034 | 0.0025 | 0.03482 | 0.95266 | ||
Client 2 | GPTFS-IRF1 | 2.9788 | 0.0828 | 0.13673 | 4.0143 | 0.69904 | |
GPTFS-IRF2 | 2.9474 | 0.05992 | −0.07722 | 3.6758 | 0.71246 | ||
GPTFS-ORF1 | 3.0049 | 0.06826 | −0.1066 | 3.9895 | 0.68667 | ||
GPTFS-ORF2 | 2.0797 | 0.0932 | 0.09133 | 2.7262 | 0.67296 | ||
Client 3 | GPTFS-NC/20N | 1.1284 | 0.0719 | 0.00471 | 1.444 | 0.72461 | |
GPTFS-IRF1/20N | 3.4072 | 0.098 | −0.05176 | 5.6176 | 0.60803 | ||
GPTFS-IRF2/20N | 1.9464 | 0.06854 | 0.06075 | 2.5784 | 0.68535 | ||
GPTFS-IRF3/20N | 1.825 | 0.06872 | −0.00536 | 2.2947 | 0.70688 | ||
GPTFS-ORF1/20N | 2.2651 | 0.07551 | −0.0794 | 2.7943 | 0.68356 | ||
GPTFS-ORF2/20N | 3.2422 | 0.04868 | 0.16 | 4.4067 | 0.77534 | ||
GPTFS-ORF3/20N | 3.1238 | 0.06493 | 0.12395 | 4.1924 | 0.74757 |
Client Task | Baseline | FedAvg | FTLS-DPP | Proposed Method | |
---|---|---|---|---|---|
Case 1 | Client 1 | 88.3 (7.34) | 94.36 (5.4) | 100 (0) | 100 (0) |
Client 2 | 66.7 (11.67) | 74.12 (9.58) | 96.9 (3.46) | 98.67 (0.2) | |
Client 3 | 54.3 (14.03) | 63.66 (3.56) | 80.01 (7.63) | 100 (0) | |
Target | 66.96 (1.66) | 84.78 (14.76) | 92.83 (7.21) | 98.76 (1.04) | |
Case 2 | Client 1 | 40.27 (5.69) | 62 (0.22) | 75 (6.89) | 81.65 (2.34) |
Client 2 | 38.09 (2.32) | 41.78 (2.81) | 39.22 (0.74) | 78.36 (1.2) | |
Client 3 | 48.45 (5.18) | 62.89 (1.26) | 65.89 (5.04) | 78.44 (2.17) | |
Target | 39.11 (6.15) | 51.67 (13.33) | 65.34 (2.14) | 80.17 (3.14) | |
Case 3 | Client 1 | 40.22 (1.59) | 43.56 (2.96) | 44.89 (5.41) | 73.34 (2.48) |
Client 2 | 61.44 (2.37) | 62.67 (3.56) | 60.89 (0.95) | 67.76 (2.71) | |
Client 3 | 36.67 (4.89) | 42.44 (13.92) | 29.45 (11.18) | 68.35 (3.14) | |
Target | 57.27 (3.42) | 56.44 (5.25) | 57.89 (1.03) | 78.06 (3.22) | |
Case 4 | Client 1 | 57.62 (8.44) | 58.42 (6.57) | 76.28 (11.01) | 99.71 (0.17) |
Client 2 | 67.23 (14.79) | 43.57 (21.33) | 81 (6.84) | 99.88 (0.1) | |
Client 3 | 59.09 (2.89) | 71.47 (9.17) | 87.14 (4.81) | 99.86 (0.15) | |
Target | 79.38 (0.38) | 78.8 (1.06) | 81.4 (3.12) | 86.58 (2.06) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yan, Z.; Sun, J.; Zhang, Y.; Liu, L.; Gao, Z.; Chang, Y. Federated Transfer Learning Strategy: A Novel Cross-Device Fault Diagnosis Method Based on Repaired Data. Sensors 2023, 23, 7302. https://doi.org/10.3390/s23167302
Yan Z, Sun J, Zhang Y, Liu L, Gao Z, Chang Y. Federated Transfer Learning Strategy: A Novel Cross-Device Fault Diagnosis Method Based on Repaired Data. Sensors. 2023; 23(16):7302. https://doi.org/10.3390/s23167302
Chicago/Turabian StyleYan, Zhenhao, Jiachen Sun, Yixiang Zhang, Lilan Liu, Zenggui Gao, and Yuxing Chang. 2023. "Federated Transfer Learning Strategy: A Novel Cross-Device Fault Diagnosis Method Based on Repaired Data" Sensors 23, no. 16: 7302. https://doi.org/10.3390/s23167302