A Fast Bayesian Evaluation Algorithm Based on the Second Categoryof Maximum Likelihood

MENG Jing（孟 晶），ZHENG Rong（鄭 榕），LIU Jun（劉 君）

1Key Laboratory of National Defense Science and Technology for Underwater Acoustic Warfare，Unit 91388of the PLA，Zhanjiang 524000，China

2Unit 92961of the PLA，Sanya 572000，China

3Unit 91872of the PLA，Beijing 100000，China

Introduction

With development of simulation technology，simulation information has been increasingly applied on evaluation for modern weaponry test.If we make the best of simulation information，not only the total number of field test can be reduced significantly，but also the confidence coefficient of evaluation can be raised accordingly.

As the core part of small sample technology，Bayesian method［1-2］has been applied successfully and widely in evaluation for weaponry test at home and abroad.The precondition of applying Bayesian method is precise analysis and proper use of prior information.In the course of applying Bayesian method to weaponry test，the common problem is that information produced by field test is small sample which would be flooded with a good deal of prior information.Therefore，we can't simply mix field information with prior knowledge in order to evaluate weaponry test.We should analyze the reliability［3-4］of prior knowledge reasonably to make the best of it.

Directing at evaluation for qualifying rate which is common in weaponry test，firstly a fast Bayesian evaluation algorithm is presented based on the elaborate analysis of prior information reliability and the second category of maximum likelihood，secondly a fast calculation formula for unilateral confidence lower limit is given，and at last a typical example is demonstrated.

1 Classical Evaluation Algorithm for Safe-or-Failure Test

In the course ofweaponry test for qualifying rate，the field data Yn＝（y1，y2，…，yn2）is safe-or-failure which follows a binomial distribution B（n2，θ）.The parameter，qualifying rateθis very important in the weaponry equipment indexes.The classical evaluation algorithm for safe-or-failure test under confidence coefficient 1-αis given as

where，θLis the unilateral confidence lower limit，and（n2，f2）is the total number and the maximum of failure number for field test which can be precisely calculated from the formula.In the classical evaluation algorithm for safe-or-failure test，total number of test n2is always very large，so the solution（n2，f2）is not tolerant.Because Bayesian method uses prior information to evaluate，it can be better and tolerant than the classical evaluation algorithm which has been successfully applied［5］.

2 Fast Bayesian Evaluation Algorithm for Qualifying Rate

2.1 The flood phenomenon

If the conjugate prior distributionπ（θ）for the qualifying rateθis chosen as Beta distribution B（a1，b1）which is taken as below，

According to Bayesian theory，the posterior distribution π（θ｜Yn）is also Beta distribution which is given as below，

where

If the loss function is taken as L（θ，）＝c（θ-）2（c is constant），the value ofshould minimize the posterior risk，which can be proved as E（θ｜Yn）which is the expectation of posterior distributionπ（θ｜Yn）.So the Bayesian point estimation for qualifying rateθis

Otherwise，the equivalence property between the simulation test information（n1，s1）and the mean value of prior distributionis given as

Further，

where kis the scale coefficient.If Eq.（7）is substituted in Eq.（5），the other transformation of Bayesian point estimation for indexθcan be solved as

We can conclude from Eq.（8）that the point estimationis between the mean value of field data and the expectation of prior distribution，and its value is decided by coefficient k.Specially，when kis equal to 1，the prior distribution is ordinary and without respect to its reliability.Otherwise，if kis not reasonably chosen，it's more probable that the point estimationθ∧levels off to the mean value of prior distributionθ-，which makes field data “flooded”［6-7］with prior information.

2.2 The reliability of prior information

Reliability of prior knowledge should be taken into consideration before applying field information to choose prior distribution.Prior information which has higher reliability should play more important role in Bayesian estimation.Prior information which is out of reliability should be rejected.The relevant researches on reliability of prior information can be concluded as follows.

2.2.1 The evaluation algorithm to measure reliability of prior distribution

The algorithm is to accurately evaluate the parameters of prior distribution by integrating it into posterior distribution.The common way is based on data consistency test such as rank sum test［3-4］and another.The principle is that if the field data and the prior data are compatible under the confidence coefficient 1-α，the reliabilityξof the prior information can be calculated as

In order to get the reliabilityξ，Bootstrap method［8-9］is applied to calculating the prior distribution and Monte-Carlo method［10］is applied to calculating the probabilityβ.The method is only fit for non-binomial distribution and needs priori probability.The document［4］demonstrates a new method to define reliability of prior information by calculating fusion degree between prior distributionπ（θ）and conditional density function f（Yn｜θ）which is shown as

where ifξapproaches 0，prior information has no effect on posterior distribution，and ifξapproaches 1，the posterior distribution is based on normal Bayesian analysis.However，when the estimation parameter subjects to the binomial distribution，if the field data is too large，the priori information will not work in the posterior distribution at all under all kinds of circumstances.So the definition in the document does not fit for Bayesian estimation of safe-or-failure test.The document［11］defines the reliability of prior information based on the consistency between the prior data and the posterior distribution.But the method is too complex and needs too much calculation.

2.2.2 The evaluation algorithm for hyper-parameters of conjugate prior distribution

The evaluation algorithm is to accurately estimate the hyper-parameters of the prior distribution through all kinds of prior information based on reliability.The common method is to use the prior moment and quantile to compute the hyperparameters.But the method will get higher estimation error if the prior information is not enough.The document［12］demonstrates the prior distribution calculation algorithm based on the information content，which builds equivalence between the prior information and the field test information to calculate hyper-parameters.But the course of calculation strongly depends on the parameterω-which is bound up with reliability of the prior information.Improper choose for parameterω-will get inaccurate estimation.

2.2.3 The fast calculation algorithm for prior distribution

Since the second category of maximum likelihood method takes the reliability of prior information into consideration based on the field test data，we can apply the method to choosing prior distribution.The second category of maximum likelihood method can be described as

where，mπ（Yn）is the marginal distribution of cluster.If the sample Ynprovides more support for prior distributionπ（θ），the value of mπ（Yn）is bigger.So the prior distribution which maximizes mπ（Yn）is the solution we need.In the case of Beta distribution，if the simulation test information（n1，s1）and the field test data（n2，s2）is given，the marginal distribution mπ（Yn）is described as

We can apply genetic algorithm toolbox in Matlab to optimizing the marginal distribution mπ（Yn）and get the last solution k＊.So the prior distributionπ＊（θ）which is chosen is

Table 1demonstrates some optimization results for the scale coefficient k＊.From data in the table，we draw conclusions as below.

（1）If the prior information is not consistent with the field test information，the scale coefficient k＊will descend extremely and approach 0along with widening of difference between the prior mean-value and the posterior mean-value.

（2）If the prior mean-value is consistent with the posterior mean-value to some extent，the scale coefficient k＊will also descend in proportion with growth of total number n2of the simulation test.

Therefore，the coefficient k＊can adjust the weight of simulation information in Bayesian estimation for the posterior distribution.

Table 1 Some optimization results for scale coefficient

2.3 The fast calculation formula for unilateral confidence lower limit

With the classical evaluation algorithm for safe-or-failure test，if the field test information（n2，s2）and the prior distribution B（a1，b1）is given，it can be concluded from the posterior distributionπ（θ｜Yn）and the definition of confidence coefficient 1-αas below，

Completely solving the unilateral confidence lower limit problem（14）is difficult and obviously beyond the scope of this article.In the sequel，we give a fast calculation algorithm as defined by the following formula，

The following lemma is necessary to proof this formula above.

Lemma If parameterθfollows Beta distribution B（a，b），bθ／［a（1-θ）］follows F distribution F（2a，2b）.

So it can be concluded from Lemma and the posterior distributionπ（θ｜Yn）as follows，

Further，the confidence coefficient 1-αis also defined as

We can draw a conclusion from Eqs.（16）and（17）as

therefore a fast solution for the unilateral confidence lower limit can be described as

3 Simulation Example

In this section，we provide some simulation results［13-14］to validate the efficiency and robustness of the fast Bayesian evaluation algorithm for qualifying rate under some conditions as：

（a）the weaponry test for qualifying rate is safe-or-failure；

（b）the minimum acceptable value for the unilateral confidence lower limit of qualifying rateθLis 0.8；

（c）total number of field test n2is given as 14，27，44，and total number of simulation test n1is given as 270.

In order to investigate how much the simulation test information influence on the unilateral confidence lower limit of qualifying rate based on the fast Bayesian evaluation algorithm，three cases are given as follows.

（a）Classical evaluation algorithm for safe-or-failure test.

（b）Normal Bayesian evaluation algorithm，where k＝1 in Eg.（8）.

（c）Fast Bayesian evaluation algorithm based on the second category of maximum likelihood：in the course of optimization for coefficient k，the genetic algorithm individual is encoded as the real number formation，the genetic generation is 1 000and the population number is 500，the fitness function is given as exp（-mπ（Yn）），and the other parameters are given as the default values.

3.1 Unilateral confidence lower limit

Figures 1-3demonstrate the calculation results of unilateral confidence lower limit of qualifying rate under all simulation test conditions.

Fig.1 Unilateral confidence lower limit when n2is 14

Fig.2 Unilateral confidence lower limit when n2is 27

Fig.3 Unilateral confidence lower limit when n2is 44

3.2 Test design

In order to get the precise solution（n2，f2），the field test can be designed as follows：

（a）initialization：f2＝0，s2＝n2-f2；

（b）the conjugate prior distribution B（a1，b1）is calculated according to the simulation information（n1，s1）and the filed test information（n2，f2），based on the optimization calculation by genetic algorithm toolbox in Matlab；

（c）the unilateral confidence lower limitof qualifying rate is calculated based on the fast calculation formula under the confidence coefficient 1-α；

Figures 4-6demonstrate the calculation results of field test solution（n2，f2）for qualifying rate under all simulation test conditions.

Fig.4 Field test design when n2is 14

Fig.6 Field test design when n2is 44

4 Conclusions

From the figures shown above，we can draw conclusions as follows.

（1）Compared with the classic evaluation algorithm for safe-or-failure，not only fast Bayesian evaluation algorithm，but also normal Bayesian evaluation algorithm all adjust the test program to a certain extent，and have different unilateral confidence lower limits.Fast Bayesian evaluation algorithm has higher confidence coefficient than normal Bayesian evaluation algorithm.

（2）Normal Bayesian evaluation algorithm is instable and maybe make the maximum of failure number f2rise extremely or out of solution.This phenomenon will make program depend excessively on simulation information and ignore the field test completely，which will raise the risk of military.

（3）The fast Bayesian evaluation algorithm is stable to a certain extent，for taking reliability of prior information into consideration based on the field information，which makes field test design program more tolerant and robust.

It is more and more critical to make the best of simulation information reliably and reasonably，along with simulation test plays more and more important role in test and estimation for weaponry equipments.But before using simulation information and Bayesian theory to inspect tactical indexes of weaponry equipments，we should take the reliability of simulation information into consideration.Otherwise，the other key work is to do well in verification，validation ＆accreditation（VV＆A）for simulation system，which will make simulation information more available.

［1］Zhang J H，Tang X M.Bayesian Method［M］.Revised edition.Changsha：National University of Defense Technology Press，1993：1-3，54-57.（in Chinese）

［2］Mao S S.Bayesian Statistics［M］.Beijing：China Statistical Publishing House，1999.（in Chinese）

［3］Yao Z J，Wang G P，Wang G W.Confidence Level of Prior Information and Its Application in Bayesian Method［J］.Fire Control and Command Control，2007，32（7）：51-53.（in Chinese）

［4］Deng H J，Zha Y B.Research on Applying Simulation Credibility into Weapon System Appraisal［J］.Journal of System Simulation，2005，17（7）：1566-1568.（in Chinese）

［5］Han Q，Wang L L，Mao S S.The Choose of Bayesian Test Program Based on Prior Distribution with Expert Information［J］.Probability and Statistics，1990，5（4）：422-427.（in Chinese）

［6］Cao Y Y，Chen G B.Reliability Assessment with Small Sample for Binomial Distribution Equipment［J］.Journal of Naval University of Engineering，2007，19（4）：59-62.（in Chinese）

［7］Zhang J H，Zhang S F.Problem of Large Numbers of Prior Information Obliterating the Small Numbers of Test Information［J］.Journal of Measurement and Control Aircraft，2003，22（3）：1-6.（in Chinese）

［8］Rubin D B.The Bayesian Bootstrap［J］.The Annals of Statistics，1981，9（1）：130-134.（in Chinese）

［9］Hu Z D，Cao Y，Zhang S F，et al.Bootstrap Method for Mis-sile Precision Evaluation under Extreme Small Sample Test［J］.Systems Engineering and Electronics，2008，30（8）：1493-1497.（in Chinese）

［10］Amar J G.The Monte Carlo Method in Science and Engineering［J］.Computer in Science ＆Engineering，2006，8（2）：9-19.

［11］Duan X J，Huang H Y.Bayesian Fusion Evaluation with the Weight Basing on Information Divergence of Prior Sample［J］.Acta Armamentarii，2007，28（10）：75-79.（in Chinese）

［12］WuX Y，Liu Q.Equipment Test and Evaluation［M］.Beijing：National Defense Industry Press，2008：426-427.

［13］Xu X K，Zhou S L，Morozov A K，et al.Per-survivor Processing for Underwater Acoustic Communications with Direct-Sequence Spread Spectrum［J］.The Journal of The Acoustical Society of America，2013，133（5）：2746-2754.

［14］Xu X K，Wang Z H，Zhou S L，et al.Parameterizing both Path Amplitude and Delay Variations of Underwater Acoustic Channels for Block Decoding of Orthogonal Frequency Division Multiplexing［J］.The Journal of The Acoustical Society of America，2012，131（6）：4672-4679.