Temporal Sequence Modeling for Rare Failure Prediction in Industrial Machinery Using a Hybrid CNN-LSTM Model

Abstract

<p>Predictive maintenance is vital for enhancing industrial machinery reliability by detecting rare failures in high-dimensional, imbalanced sensor data. This study analyzes a dataset of 220320 samples, with failure events (BROKEN) constituting only 0.003%. Initial state classification using autoencoders and ensemble classifiers (Random Forest, Extreme Gradient Boosting) failed to detect these rare failures due to extreme class imbalance. To address this, we developed a hybrid Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) model for temporal sequence modeling, utilizing convolutional layers for spatial feature extraction and LSTM layers for capturing temporal dependencies within 6-sample windows, classified as Normal or Transient (failure precursors). Main contributions include a robust feature selection method for imbalanced datasets, a comprehensive comparative analysis of state classification, temporal modeling, and unsupervised anomaly detection (Isolation Forest, One-Class Support Vector Machine), insights into temporal failure precursors, and an evaluation framework using Leave-One-Out Cross-Validation (LOOCV) to ensure robust assessment of rare events, balancing accuracy and early detection. The hybrid CNN-LSTM model achieved 97.88% overall accuracy, detecting 6 out of 7 failure cases with 85.71% recall, though precision was 30% due to false positives. Compared to baseline CNN (4.82% precision, 57.14% recall), LSTM (1.02% precision, 28.57% recall), Isolation Forest (50.00% precision, 14.29% recall), and One-Class SVM (8.00% precision, 28.57% recall), the hybrid model significantly improved rare failure detection by effectively capturing spatial and temporal patterns. These results highlight the efficacy of the CNN-LSTM approach with LOOCV for proactive maintenance, offering substantial improvements in industrial reliability and safety for real-world applications with extreme class imbalance.</p><p>Index Terms—Predictive Maintenance, Convolutional Neural Network, Long Short-Term Memory, Industrial Machinery, Anomaly Detection, Rare Failure Detection</p>