A Dual-Attentive Multimodal Fusion Method for Fault Diagnosis Under Varying Working Conditions

Source abstract

Deep learning-based fault diagnosis methods have gained extensive attention in recent years due to their outstanding performance. The model input can take the form of multiple domains, such as the time domain, frequency domain, and time–frequency domain, with commonalities and differences between them. Fusing multimodal features is crucial for enhancing diagnostic effectiveness. In addition, original signals typically exhibit nonstationary characteristics influenced by varying working conditions. In this paper, a dual-attentive multimodal fusion method combining a multiscale dilated CNN (DAMFM-MD) is proposed for rotating machinery fault diagnosis. Firstly, multimodal data are constructed by combining original signals, FFT-based frequency spectra, and STFT-based time–frequency images. Secondly, a three-branch multiscale CNN is developed for discriminative feature learning to consider nonstationary factors. Finally, a two-stage sequential fusion is designed to achieve multimodal complementary fusion considering the features with commonality and differentiation. The performance of the proposed method was experimentally verified through a series of industrial case analyses. The proposed DAMFM-MD method achieves the best F-score of 99.95%, an accuracy of 99.96%, and a recall of 99.95% across four sub-datasets, with an average fault diagnosis response time per sample of 1.095 milliseconds, outperforming state-of-the-art methods.