Remaining Useful Life Prediction of Aeroengine Based on Principal Component Analysis and One-Dimensional Convolutional Neural Network

，

1.College of Civil Aviation，Nanjing University of Aeronautics and Astronautics，Nanjing 210016，P.R.China；

2.Center for Drug Inspection of Jiangsu Medical Products Administration，Nanjing 210019，P.R.China

Abstract:In order to directly construct the mapping between multiple state parameters and remaining useful life（RUL），and reduce the interference of random error on prediction accuracy，a RUL prediction model of aeroengine based on principal component analysis（PCA）and one-dimensional convolution neural network（1D-CNN）is proposed in this paper.Firstly，multiple state parameters corresponding to massive cycles of aeroengine are collected and brought into PCA for dimensionality reduction，and principal components are extracted for further time series prediction.Secondly，the 1D-CNN model is constructed to directly study the mapping between principal components and RUL.Multiple convolution and pooling operations are applied for deep feature extraction，and the end-to-end RUL prediction of aeroengine can be realized.Experimental results show that the most effective principal component from the multiple state parameters can be obtained by PCA，and the long time series of multiple state parameters can be directly mapped to RUL by 1D-CNN，so as to improve the efficiency and accuracy of RUL prediction.Compared with other traditional models，the proposed method also has lower prediction error and better robustness.

Key words：aeroengine；remaining useful life（RUL）；principal component analysis（PCA）；one-dimensional convolution neural network（1D-CNN）；time series prediction；state parameters

0 Introduction

The continuous progress of data acquisition technology and artificial intelligence algorithm provides effective support for aeroengine health management.By monitoring the state parameters and predicting the health status of the aeroengine，it is available to provide timely maintenance，and ensure the safe and reliable operation of the aeroengine［1］.As one of the important indicators of aeroengine health management，the remaining useful life（RUL）is used to comprehensively measure the health status of the aeroengine and provide guidance for further maintenance and replacement［2］.

The research methods of RUL prediction mainly include stochastic process model，classical machine learning and deep learning［3］.Hidden Markov model，Gaussian process and Wiener process have been widely used as valuable stochastic models，which can be used for simulating the performance degradation process.Li et al.［4］proposed an improved time varying and condition adaptive hidden Markov model for tool wear state estimation and RUL prediction in micro-milling.Kumar et al.［5］applied Kullback-Leibler divergence and Gaussian processes regression for bearing degradation assessment and RUL estimation.Liao et al.［6］described the multi-phase degradation pattern based on Wiener process.The statistical analysis method is easily affected by engineering experience and subjective factors.At the same time，the generality of statistical analysis model is also poor，and repeated modeling is needed for different research objectives.Machine learning can analyze the relationship between state parameter and performance degradation，and extracts data features for performance degradation prediction［7］.Li et al.［8］developed a multiple model framework with particle filter and support vector regression for RUL estimation of lithium-ion battery.Wang et al.［9］investigated first-order partial derivatives with a back-propagation neural network for engineering uncertain optimization problems.Liet al.［10］designed an extended version of radial basis function-extreme learning machine for mixed data classification.Shallow machine learning has less hidden layer，simple model structure and poor ability of deep feature extraction for complex nonlinear multidimensional samples，so it is often used in data preprocessing to reduce the sample dimension.As an important branch of machine learning，deep learning also has a preliminary application in remaining useful life.Deep learning models can mine the deep hidden features of samples and get more accurate prediction results by training and optimizing of multilayer neural network［11］.Haris et al.［12］presented a novel combination of deep belief network（DBN）with Bayesian optimization and hyperband to predict the RUL of supercapacitors in the early degradation phases.Liu et al.［13］explored the degradation process of bearings by enhanced encoder-decoder framework.Cui et al.［14］discussed an effective fault diagnosis method for aeroengines based on the gravitational search algorithm and the stack autoencoder（GSA-SAE）.Deng et al.［15］introduced a long-short term feature processing method for predicting the RUL of the aircraft engine.Yao et al.［16］combined the improved one-dimensional convolution neural network（1D-CNN）and a simple recurrent unit（SRU）network to predict RUL of roller bearings.

However，there are mainly three limitations in the existing methods：

（1）Most of the research is based on the study of state parameters and performance degradation to derive the RUL.In the process of applying correlation distribution for the research of performance degradation，it will inevitably produce errors due to the interferences of environment and noise，which will affect the subsequent RUL prediction.

（2）Nowadays，most of the research is to analyze the multi-dimensional state parameters directly.However，there are many useless and noisy items in the massive state parameters，so it is necessary to find typical state parameters that can really reflect the change of RUL and improve the efficiency of model training.

（3）Compared with the mainstream problems of deep learning processing such as image classification and speech recognition，the RUL prediction lacks the supports of sufficient sample dimension and size.Therefore，it is challenging to select the appropriate deep learning method to make better use of the powerful feature extraction ability of deep learning model.

Aiming at the above limitations，a novel RUL prediction model based on principal component analysis（PCA）and 1D-CNN is proposed in this paper.The main contributions are summarized as follows：

（1）PCA is used to extract the principal component from long time series and multiple state parameters，so as to obtain comprehensive indexes for further RUL prediction.

（2）1D-CNN is selected as the time series prediction model to process one-dimensional samples，and the deep feature extraction is realized through multiple convolution layers and pool layers.

（3）Taking the comprehensive indexes of multiple state parameters and RUL as the input and output of 1D-CNN，it is available to achieve end-toend RUL prediction.

The rest of this paper is introduced as follows：The theoretical background of PCA and 1D-CNN are discussed in Sections 1 and 2，respectively.Section 3 presents the structure of the proposed RUL prediction method of the aeroengine based on PCA and 1D-CNN.Section 4 introduces related experiments to testify the effectiveness of the proposed method.Finally，the conclusions of this work are given in Section 5.

1 Principal Component Analysis

PCA is the most commonly used linear dimensionality reduction method，which can be used to map high-dimensional data to low dimensional space through linear projection，and expect the maximum amount of information in the projected dimension.The purpose of PCA dimensionality reduction is to reduce the dimension of the original feature without losing too much information，which can be described to project the original feature to the dimension with the largest amount of projection information.The original features are projected onto these dimensions to minimize the loss of information.Suppose that there aremsamples containingn-dimensional features，and the sample matrixX n×mwith a size ofn×mcan be obtained as follows

The procedures are detailed to describe principal component analysis method as follow［17］：

（1）Feature centralization.By solving and subtracting the average value of each feature of the sample，the centralized data set can be obtained as

（3）The eigenvalueλand eigenvectorβof the covariance matrix are calculated by Eq.（3），and the eigenvector is regularized as Eq.（4）.

（4）The firstqlargest eigenvalues and corresponding eigenvectors are retained as

（5）The original features are transformed into the new space constructed by the aboveQfeature vectors，so as to realize the sample dimension reduction as

2 One-Dimensional Convolutional Neural Network

Convolutional neural network（CNN）can map the simple features of data to more complex high-dimensional patterns through convolution and pooling operations.Different dimensions of CNN have the same structure and processing method.The key difference lies in the dimension of the input data and how the convolution kernel slides among samples.1D-CNN can directly deal with long time series，and realize feature extraction of samples by adjusting the convolution kernel size and moving direction，so as to avoid the complex operation of constructing and processing multi-dimensional samples［18］.

A typical 1D-CNN model with two hidden layers is shown in Fig.1.After several convolution and pooling operations，the input data is transformed into multiple feature maps with fewer time series.Then the feature maps are expanded into one-dimensional fully connected layer（FCL）.Subsequently，the model output can be achieved by dimension reduction and activation.Finally，the loss function is obtained by solving the error between the model output and the real output，and the back propagation algorithm and stochastic gradient descent method are used to finetune the model，so as to obtain trained 1D-CNN for time series prediction.

Fig.1 Model structure diagram of 1D-CNN

3 Proposed Remaining Useful Life Prediction Model

In this paper，the RUL prediction model of aeroengine based on PCA and 1D-CNN is proposed.Fig.2 shows the procedures of the proposed RUL prediction model，which includes two parts：multiple state parameter dimensionality reduction and RUL prediction based on multiple comprehensive indexes.

Fig.2 Structure of denoising autoencoder

For multiple state parameter dimensionality reduction，mstate parameters andntime series collected in the whole life cycle of the aeroengine constitute the datasetX n×m，which can be centralized as.In addition，the covariance matrix can be solved asC=m.Then the eigenvalueλand eigenvectorβare calculated according toCβ=λβ.The largestqeigenvalues and the corresponding eigenvectorQare extracted to form new space.The original sampleis transferred to the new space to obtain the reduced dimension sampleY=Q.With the application of principal component analysis，principal components are extracted from multiple state parameters with long time series，which can provide comprehensive indicators for further RUL prediction.

For the RUL prediction based on multiple comprehensive indexes，the reduced comprehensive indexes are divided into one-dimensional time series samples according to a certain step size.Subsequently，those samples are brought into the 1D-CNN model with multiple layers.The hidden features are extracted by multiple convolution layers and pooling layers，and then the features are extended to the fully connected layer.Finally，the RUL prediction results can be achieved by the processing of dense layer and activation function.The error between the predicted RUL and the actual RUL is defined as loss function，and the 1D-CNN model can be finetuned by back propagation algorithm and random gradient descent，so as to obtain the final RUL prediction model.To sum up，the proposed model can extract comprehensive indexes from multiple state parameters through PCA and 1D-CNN，and realize end-to-end prediction from state parameters to RUL，so as to provide more accurate guidance for preventive maintenance of aeroengine.

4 Case Study

In this section，the commercial modular aeropropulsion system simulation（C-MAPSS）dataset is used to testify the effectiveness of proposed RUL prediction model.The C-MAPSS dataset is the simulation of aircraft engine sensors，which provides a snapshot of data taken during a single operational cycle for different variables.The engine is operating normally at the start of each time series，and develops a fault at some point during the series［19］.By calculating the number of operational cycles，we can obtain the RUL and corresponding state parameters.The dataset consists of multiple multivariate time series including three operations and 21 sensor parameters.After removing the invariable parameters，16 indexes shown in Table 1 can be obtained for further research.

Table 1 Indicators of C-MAPSSdataset

Fig.3 shows the selected 16 state parameters corresponding to 192 cycles of an engine life cycle.It can be seen that some state parameters oscillate up and down as the number of cycles increases or decreases.With the increase of the number of cycles，the degree of performance degradation increases and the remaining life decreases.Therefore，it is necessary to extract the comprehensive indexes with clear change trend from the state parameters for the further RUL prediction.

Fig.3 Variation curve of aeroengine state parameters in whole life cycle

The 16 indicators are brought into PCA model，and the common factor variance can be obtained，as shown in Table 2.

Table 2 Common factor variances of state parameter s

The eigenvalues and variance percentages are shown in Table 3.

Table 3 Eigenvalues and variance percentages of state parameters

The component matrix can be obtained by selecting two principal components with eigenvalues greater than 1，as shown in Table 4.

Table 4 Composition matrixes of state parameters

Bring the original data into the new space composed of principal eigenvalues and eigenvectors，it is available to obtain the comprehensive indexes after dimension reduction，as shown in Fig.4.PCA is used to extract the principal component from long time series and multiple state parameters，so as to obtain comprehensive indexes for further RUL prediction.

Fig.4 Schematic diagram of principal components after dimension reduction

Two comprehensive indexes with one-dimensional time series samples are input into 1D-CNN model，and the 3D structure of the model input is defined as（183，10，2），which means 183 samples，10 steps and 2 channels.The structure of the 1D-CNN model is shown in Table 5.The model takes the mean square error（MSE）as the loss function and uses Adam algorithm for model optimization.In addition，coefficient determinationRis also been used to evaluate the effect of the regression model，which can be defined as

Table 5 Model structure of 1D-CNN

wherefiis the predicted value of the propoded model andy ithe real value.The range ofRis［0，1］.

Bring 500 train samples and 100 test samples into 1D-CNN for model training and optimizing.After 100 epochs of training，the test MSE decreases to 0.004，and theRcoefficient increases to 0.995，which means that the proposed 1D-CNN model has less error and better effect in regression analysis.Different levels of Gaussian noisel*N(0，1)are added in test set，wherelis used to control the size of the added noise.Table 6 lists six datasets with different levels of Gaussian noise.The RUL prediction results can be obtained by bringing these six datasets into the proposed PCA-1DCNN model，as shown in Fig.5.It can be seen that the proposed model can still get the prediction results close to the real RUL under different levels of Gaussian noise.

Fig.5 Residual life prediction results with different datasets

Table 6 Residual life prediction dataset

Several comparison models are built to verify the effectiveness of the proposed method.Recurrent neural network（RNN）and autoregressive integrated moving average model（ARIMA）are used to predict the time series of multiple state parameters directly，and then the prediction results are brought into neural network to get RUL.Table 7 lists the prediction results of different datasets.It can be concluded that the RUL prediction error of the proposed model is lower than traditional RNN and ARIMA under different levels of Gaussian noise.

Table 7 Residual life prediction dataset

5 Discussion and Conclusions

To achieve accurate RUL prediction of aeroengine with long time series and multiple state parameters，a novel end-to-end RUL prediction model based on PCA and 1D-CNN is proposed in this paper.The matrix of multiple state parameters and long time series is introduced into PCA model to extract principal components with high variance percentage，and the new sample space is reconstructed by principal eigenvalue and eigenvector to obtain the comprehensive indicators with reduced dimension.Then，several one-dimensional time series samples corresponding to multiple principal components are input into the 1D-CNN model as multiple channels for deep hidden feature extraction.Through convolution and pooling operations，the extracted features are expanded to the fully connected layer，and the RUL prediction results are output through activation function.In general，the proposed PCA-1DCNN method can get the comprehensive indexes by dimension reduction based on PCA，and map the principal indexes extracted from state parameters to RUL，so as to achieve end-to-end RUL prediction.Therefore，it is unnecessary to study the complex stochastic process of performance degradation，which improves the prediction efficiency of the model.The example verification results show that the proposed PCA-1DCNN model can get the prediction results close to the real RUL under different levels of Gaussian noise.Compared with traditional prediction models，it also has lower error.It can be concluded that the proposed model has good robustness and high prediction accuracy for RUL prediction of aeroengine with long time series and multiple state parameters.