Online Prediction of Ship Rolling based on Varying Parameters LSSVM

LIU Sheng,YANG Zhen

(College of Automation,Harbin Engineering University,Harbin 150001,China)

1 Introduction

Recently,support vector machine theory develops rapidly in application of ship motion modeling field[1-5].The reason is that support vector machine can obtain global optimal solution,possess good generalization ability and provide powerful methods and tools for practical problems which are small sample and nonlinear.Online learning algorithm of LSSVM[6]can track dynamic characteristics of time-varying nonlinear,which is consequently applied to prediction of ship motion.However,hyper-parameters of online training algorithm are set subjectively and all the samples use the same hyper-parameters,which mean that hyper-parameters can not adjust automatically as samples change.Change characteristics of ship motion are generally difficult to predict.Maybe a change rule is in a period,but another change rule is in another period.The fixed hyper-parameters inevitably can not obtain ideal modeling and prediction result.The current SVM parameter selection method[7-8]is offline approximation optimization,which needs large computation.Accordingly,it is necessary to design a modeling method that makes hyper-parameters divorce from current poor settings and adjust dynamically along with time.In addition,ship motion has chaos characteristics in many cases[9].Evolution rule of chaotic attractor can be found by phase space reconstruction.Because of the predictability of chaotic systems,the next few moments states of system can be obtained by current moment sates of system,namely the next few moments prediction values are gained in time series.

Using phase space reconstruction to ship rolling motion time series,the obtained samples are applied to online LSSVM modeling prediction.Meanwhile,an online modeling prediction method is designed,where hyper-parameters can adaptively adjust as change of ship rolling motion.The proposed method divides the whole modeling prediction process into initial stage and several forecast stages.Three LSSVMs with different hyper-parameters are used to parallel modeling,where one is prediction LSSVM and the others are comparison LSSVM in every forecast stage.At the end of each forecast period,LSSVM which has the smallest prediction error summation is selected as prediction LSSVM of the next prediction stage,and the others are as comparison LSSVM,all of which need to set new hyper-parameters.The proposed method can find that hyper-parameters of online LSSVM prediction model are compatible with ship motion characteristics,and improve prediction accuracy.

2 Phase space reconstruction

A common problem is lack of all variables information about system for actual dynamic system.Phase space reconstruction provides an effective way to describe dynamic characteristics of system[10].Phase space trajectory of system is reconstructed by the observed one-dimensional time series,which is seen as a topological representation of dynamic characteristics of system.Then,using some fitting algorithm to fit phase space of system,the prediction of the original system is based on the fitting system.Therefore,phase space reconstruction is the foundation to effectively predict chaos time series.

For chaos time series x（t）,t=1,2,…,n,m-dimensional phase space can be reconstructed by choosing appropriate embedded dimensional m and delay time τ.

where,t=1,2,…,M,M=n-（m-1）τ.

3 LSSVM and learning algorithm

3.1 Standard LSSVM regression

where w is the coefficient,b is the bias.According to the minimization principle of structural risk,regression problem is converted into the optimization problem constrained.The object function is defined as follows:

where C is the penalty factor,eiis the approximation error.To solve the optimization problem above,the corresponding Lagrange function is given by

where αiis the Lagrange multiplier.According to Karush-Kuhn-Tuncker conditions,we have

eliminating eiand w,Eq.(5)can be changed into:

3.2 Incremental learning algorithm of LSSVM regression

For incremental learning algorithm of LSSVM,the sample is incremented along with the time,at t time,sample set used for learning can be expressed aswherethen

Formula(6)can be rewritten as:

solution of Eq.(8)can be obtained:

The regression model can be obtained:

from Eqs.(9)and(10),in order to get α（t）and b（t）,solving H（t）-1is the key.When the new sampleis added to the sample set,H（t+1）can be written as:

3.3 Online learning algorithm of LSSVM regression

The samples are window moving for online learning.And the samples remain constant in the window by increasing a new sample and reducing an old sample at the same time.Sample set used for learning can be expressed aswherel is the length of window.ThenLSSVM model for function estimation is obtained:

The same with incremental learning algorithm,in order to get α（t）and b（t）,solving H（t）-1is still the key.At t time,Htis a l×l dimension matrix:

According to the block matrix inversion[6],H（t）-1is obtained:

At t+1 time,the newer samples（xt+l,yt+l）are added and the older samples are discarded.However,H（t+1）is still a l×l dimension square matrix:

4 Online prediction of ship rolling based on varying parameters LSSVM

4.1 Online prediction model based on LSSVM

Saturated correlation dimension algorithm and mutual information function method[10]are used to determine embedded dimension m and delay time τ for prediction seriesrespectively.Then put prediction seriesinto m dimension new data space:

The former m-1 dimension of Xt+1is known history data,which is converted into single output as follows:

where f（x）is the map from Rmto R.The essence of the forecast problem is how to get the best approximation on f（x）.f（x）has strong nonlinear,so nonlinear function is adopted to fit the mapping.Combined with online learning algorithm of LSSVM,training data is got as follows:

The relay forecast method is adopted in the paper.Assume that the current training samples are subset of Eq.(22)and time window length is W,namely W training samples are in the window,where W＜M.Using online learning algorithm of LSSVM,the current regression function is gained:

Further,the result of the first step forecasting is

When a new sample enters the window,the oldest sample X1is removed from the window,then the length W of training samples remains.LSSVM is used to conduct online training and complete P step forecasting according to Eqs.(23)～(26),when the back samples come in turn,prediction method can be done like before.The hyper-parameter σ of LSSVM is fixed in the whole prediction process,which is called fixed parameter online prediction method.

4.2 Online modeling method based on parameter-varying LSSVM

P step prediction model established in previous section just considers samples update,but hyper-parameters of LSSVM are fixed and do not adjust automatically with samples change,where it inevitably affects result of modeling and prediction for dynamic modeling process.In order to solve this problem,online modeling method with σ adapting to samples changes is proposed.Three LSSVMs denoted by LSSVMi（i=1,2,3）are running simultaneously,whose σ is respectively denoted by σi.Online prediction process is seen from Fig.1.Foregoing W steps are called learning stage in the initial stage which starts from n+1 moment.Three LSSVMs start incremental learning from the first sample until n+W moment.Then online learning stage is called comparison stage starts with Q step.The expected outputis respectively obtained when a sample goes through three LSSVMs at kstep,after which prediction error absolute valueis separately calculated and is accumulated to prediction error accumulation variables ESi（i=1,2,3）.Then an online learning is completed.Calculation ofand accumulation of ESiare repeated operation Q time.

Fig.1 Time series prediction process classification

ESi（i=1,2,3）are ordered by ESi1≤ESi2≤ESi3at the end of initial stage.LSSVMi1is seen as forecasting LSSVM in the first prediction stage,whose σ1remains;LSSVMi2and LSSVMi3are seen as comparison LSSVM.Empty data in the LSSVMi2and LSSVMi3,and reset σ2and σ3between region［σimin,σimax］as the following rules:（σmin+σi1）/2 and σi2remains,which may achieve better prediction result;Similarly,if σi1≤σi3≤σi2,then σi3remains and σi2=（σmin+σi1）/2.

(1)If σi1≤σi2≤σi3,parameter σ of LSSVM takes smaller value than σi1,such as σi3=

(2)If σi3≤σi2≤σi1,parameter σ of LSSVM takes larger value than σi1,such as σi2re-mains and σi3=σi1+（σmax-σi1）×1/2,which may achieve better prediction result;Similarly,if σi2≤σi3≤σi1,then σi2=σi1+（σmax-σi1）×1/2 and σi3remains.

(3)If σi2≤σi1≤σi3or σi3≤σi1≤σi2,parameter σ of LSSVM takes value between region［σi2,σi1］and［σi1,σi3］,such as σi2=（σi1+σi2′）/2,σi3=（σi1+σi3′）/2,where σi2′and σi3′represent original value,respectively,which may achieve better prediction result.

In the learning stage of the first prediction stage,LSSVMi1starts online learning with window length M,and P step prediction by using prediction models(Eqs.(23)～(26))designed in the previous section;LSSVMi2and LSSVMi3carry on incremental learning.In the comparison stage of the first prediction stage,LSSVMi1still conducts online learning and then P step prediction,simultaneously accumulates ESi1in the stage;LSSVMi2and LSSVMi3conduct online learning,and simultaneously accumulate ESi2and ESi3.After the last step of the first prediction stage,the LSSVM is seen as forecasting LSSVM in the second prediction stage,which has the smallest prediction error accumulation value.Meanwhile,the data in the other two LSSVM is cleared up and reset σ value.Then the second prediction stage starts the same as the first one.

4.3 Real-time prediction step of online modeling based on parameter-varying LSSVM

Radial basis function is selected as kernel function of LSSVM:

Prediction algorithm of online modeling based on time-varying LSSVM is summarized as follows:

Step 1 Samples set of Eq.(22)is obtained by phase space reconstruction theory,and determine time window length W value.Set other parameters:Q,C,σmin,σmaxand σi（i=1,2,3）,

where σiis the point of quarter between region［σimin,σimax］.

Step 2 Operation of initial stage

(1)LSSVMi（i=1,2,3）conduct incremental learning algorithm in learning stage,and step into comparison stage after completing W samples.

(2)LSSVMi（i=1,2,3）conduct online learning in Q steps of comparison stage,and their each prediction error absolute value are accumulated into ESi（i=1,2,3）.

Step 3 ESi（i=1,2,3）are ordered at the end of the initial stage or every prediction stage.Then choose the LSSVM that has the smallest ESias forecasting LSSVM next prediction stage,empty data in the other two LSSVM that are selected as comparison LSSVM next prediction stage,and reset σ value according to the previous section.

Step 4 Operation of prediction stage

(1)Prediction LSSVM conducts online learning in W steps of learning stage,simultaneously online prediction models(Eqs.(23)～(26))are used to get P steps prediction values after each step learning.In addition,the other two comparison LSSVMs conduct incremental learning.

(2)Three LSSVMs conduct online learning and accumulate prediction error ESi（i=1,2,3）in Q step of comparison stage.However,prediction LSSVM obtains P steps prediction values using online prediction model after each step learning,and the other two LSSVMs just are used for comparison.

Step 5 Go to step 3,repeat the process and complete online modeling prediction.

5 Example of prediction for ship rolling motion

In order to the efficiency of the proposed method,rolling data of some ship is used for online modeling prediction.Time series data of rolling angle for some ship is generated at 5 level sea condition,3 meters wave height,90°encounter angle and 0.5 s sample period.Delay time τ=3 and embedded dimension m=10 of rolling angle data are determined according to section 3.1,and set related parameters:W=50,Q=10,C=246,σmin=0.1,σmax=10,σ1=2.5,σ2=5,σ3=7.5,P=10.Online prediction methods that have fixed parameter and time-varying parameter are used to predict rolling angle.Prediction and actual curves of prediction stage using fixed parameter method in Fig.2.Prediction and actual curves of the same prediction stage using time-varying parameter method in Fig.3.(Mean Suare Error,MSE)and(Relative MSE,RMSE)are used to represent prediction performance index[11].Statistics result is shown in Tab.1.

where,q={20,40,60,80,100,120}is total prediction points.

Fig.2 Forecasting and actual curve of fixed-parameter method

Fig.3 Forecasting and actual curve of varying parameter method

From prediction results of Figs.2 and 3,prediction method with fixed parameter and timevarying parameter can obtain good result in(0-30 s)of prediction stage 1.However,prediction results of both methods are very different in(30-60 s)of prediction stage 2.Prediction method with time-varying parameter still can gain good prediction results,while prediction method with fixed parameter is not obviously as good as prediction stage 1.

Tab.1 Error statistics of two prediction models

From statistics of Tab.1,the average MSE and RMSE of prediction model with time-varying parameter are 0.372 1 and 6.85%in the two prediction stage,respectively,while MSE and RMSE of prediction model with fixed parameter are 0.623 1 and 12.60%in the two prediction stages.Prediction average accuracy of the prediction model with fixed parameter is lower,in which about 16%prediction error in prediction stage 2 is obviously larger than prediction error in prediction stage 1 and the precision decreases soon.

The optimal parameter of LSSVM is set subjectively in online prediction method with fixed parameter.Rolling angle curve is a complex changing process,while kernel parameter can not adaptively adjust in prediction process.Although approximate optimal parameters are captured by offline optimization with initial stage samples,these parameters will not be optimal with change of dynamic characteristics.The proposed online prediction method with time-varying sets an optimal interval.The proposed method can ensure that parameters of prediction model are adjusted to adapt next prediction stage in initial and at the end of prediction stage.Thereby,the proposed method has a self-adaptive ability.However,a prediction stage is needed when the parameters are adjusted each time,which will result to hysteresis.If dynamic characteristics of system change quickly,the proposed method can not be adjusted in time.

6 Conclusions

Real-time prediction method of ship rolling motion is studied based on chaos theory and online LSSVM algorithm in this paper.The obtained online time series is mapped into m dimension characteristics space by phase space reconstruction.Then online LSSVM is used to build up real-time prediction model,namely prediction model with fixed parameter that has the features of online update samples set,online learning and real-time prediction.In addition,because of the fixed kernel parameters of prediction model can not be adjusted as dynamic changes of rolling motion,an online modeling method is designed on the base of parametervarying LSSVM.A self-adaptive ability of kernel parameters is realized,including next prediction LSSVM is selected at the end of prediction stage and kernel parameters of the other comparison LSSVMs are set by heuristic rules in the specified interval.So the better prediction result can be obtained in different prediction stages.

[1]Xu Feng,Zou Zaojian,Song Xin.Parametric identification of AUV′s maneuvering motion based on support vector machines[J].Journal of Ship Mechanics,2011,15(9):981-987.

[2]Liu Sheng,Yang Zhen.Real-time forecasting of ship roll motion based on improved support vector machine[J].Journal of Harbin Engineering University,2011,32(5):607-612.

[3]Liu Sheng,Song Jia,Li Gaoyun.Robust LSSVM control for ship course-keeping system[J].Control and Decision,2010,25(4):551-555.

[4]Liu Sheng,Song Jia,Li Gaoyun.LSSVM-IMC control for ship course-keeping system[J].Electric Machines and Control,2009,13(1):183-188.

[5]Jian Na.SVM and its application to fault prediction of ship heading control system[D].Harbin:College of Automation,Harbin Engineering University,2008.

[6]Zhang Haoran,Wang Xiaodong.Incremental and online learning algorithm for regression least squares support vector machine[J].Chinese Journal of Computers,2006,29(3):400-406.

[7]Guo X C,Yang J H,Wu C G.A novel LS-SVMs hyper-parameters selection based on particle swarm optimization[J].Neurocomputing,2008,71(16):3211-3215.

[8]Zhao Mingyuan,Fu Chong,Ji Luping.Feature selection and parameter optimization for support vector machines:A new approach based on genetic algorithm with feature chromosomes[J].Expert Systems with Applications,2011,38(2):5197-5204.

[9]Hou Jianjun,Dong Fang,Cai Feng.Extreme short term prediction of ship swaying motions based on combination of chaos and neural network[J].Ship Science and Technology,2008,30(1):67-70.

[10]Zhang Yong,Guan Wei.Combined dynamic modeling to forecast traffic volume time series[J].Journal of Jilin University(Engineering and Technology Edition),2010,40(5):1209-1214.

[11]Peng Xiuyan,Men Zhiguo,Liu Changde.Volterra-kernel estimation and its application based on Kalman filtering algorithm[J].Systems Engineering and Electronics,2010,32(11):2431-2435.