AIRCRAFT CONCEPT EVALUATION AND EFFECTIVENESS-BASED DECISION-MAKING

Zhao Suozhu（趙鎖珠），Yang Wei（楊偉），2，3 Li Jun（李軍），3，Liu Jifu（劉紀福）

（1.College of Aeronautics，Northwestern Polytechnical University，Xi′an，710072，P.R.China；2.College of Aerospace Engineering，Nanjing University of Aeronautics and Astronautis，Nanjing，210016，P.R.China；3.Chengdu Aircraft Design ＆Research Institute，Chengdu，610041，P.R.China）

INTRODUCTION

Aircraft design includes conceptual design，preliminary design and detailed design.Aircraft top-hierarchy design belongs to conceptual design with the purpose of providing a feasible and optimal general concept for preliminary design.Its outcome directly affects the final combat effectiveness of aircraft.At present，there are extensive researches on combat aircraft top-hierarchy concept evaluation and decision-making.Ref.［1］proposed a synthesized index model to assess combat aircraft，and the evaluation models were perfected synchronously.Refs.［2-6］applied a number of methods，such as ideal point decisionmaking method and spidergram method，to study the design concept selection method from different angles.Ref.［7］proposed a new index system to evaluate air combat effectiveness and established a synthesis multi-index evaluation model.Refs.［8-9］illustrated the multi-criterion decisionmaking method for selection and decision-making among multiple concept options.But the problem is that the evaluation index is repeated or incomprehensive，and the evaluation attribute system and decision-making method is not perfectly synthesized.At the same time，the evaluation results are different due to different decision-making methods.Due to the complexity of design issues and the limitation of decision-making methods，presently there is no perfect and common theory or method for top-hierarchy aircraft concept eval-uation and decision-making［8］.Combat aircraft effectiveness analysis，performing quantitative evaluation or optimization on combat aircraft effectiveness indices，is a more reasonable way to evaluate the merit figure of a combat aircraft.It is essential to study the application of effectiveness analysis and multi-criterion decision-making method to top-hierarchy decision-making for aircraft.

Conventional design concepts are tradeoff of aerodynamics，airframe structure，flight performance，weight and thrust，etc.With the development of aerotechnics and relevant technologies，survivability，reliability，maintainability and supportability（RMS），life cycle cost，improvability，development risk and schedule are also included in combat aircraft design evaluation index system.Aiming at these problems，we investigate combat effectiveness to establish a multi-criterion and multi-hierarchy evaluation index system，introduce weight-assigning method to the system and normalizing methods for different evaluation indices，and comprehensively apply four multi-criterion decision-making methodologies including grey correlation projection method，weighted summation method，weighted product method，and ideal solution decision-making method to evaluate and decision-make eight alternative concepts.The evaluation example shows the validity and feasibility of hierarchy index system and decision-making method.

1 EVALUATION SYSTEM AND DECISION-MAKING APPROACH

1.1 Hierarchy system based on effectiveness

Combat aircraft is a high complexity and high cost weaponry system.When evaluating general concepts，it is necessary to analyze the integrated operation-cost effectiveness which combines the integrated operational effectiveness with the life cycle cost.It can generally reflect the effectiveness-cost feature of modern combat aircraft.The integrated operational effectiveness shows the integrated effectiveness in wartime or peacetime operations.It measures the whole system operation figure of merit of combat aircrafts.It relates to not only the integrated combat effectiveness but also the operational technology features.The combat effectiveness is defined as the ability for the asset to complete a designated mission by typical personnel in predicted or prescribed environment and considered organization，strategy，tactics，survivability and threat conditions.This definition reflects the system＇s availability，confidence level and operational capability，and measures the effectiveness of certain combat forces to complete operational missions by use of weaponry systems.The integrated operational capability usually corresponds to aircraft inherent capability or mission capability.It mainly includes the operational capability，i.e.the capability of modern combat aircraft to perform air-air and air-ground warfare.

The top-down hierarchy structure for evaluation indiex system is：integrated operation—cost effectiveness index—integrated operational effectiveness index—integrated combat effectiveness index—integrated capability index.The layer one indices of integrated operational effectiveness include integrated combat effectiveness，testability，durability， safety， development risk， and growth.The layer one indices of life cycle cost include developing and testing cost，purchase cost，operation and maintenance cost，demilitarization and disposal cost.The lay one indices can be further decomposed into lower layer indices till tactical and technical performance index layer.The layer-by-layer breakdown results in evaluation index system for combat aircraft concepts are shown in Fig.1.

Fig.1 Evaluation index architecture for selection of combat aircraft concept

1.2 Assigning weight to evaluation index

There are many methods to assign weights，and the representatives are analytic hierarchy process （AHP），group weights determination method，rough set etc.AHP is an effective assessing method combining qualitative analysis with quantitative analysis.Therefore，there is certain subjectivity in it.Group weight determination method calculates group negotiation section，index and weights，to determine the weight of each index by negotiation.There is also certain subjectivity in it.Rough set theory explores the index data of the combat aircraft at service or under development，uses the conception of information entropy to determine the attribute significance of each index，and normalizes the weight coefficient of each index.This method excludes the subjectivity in determining weights.The weights can be determined from the following process：different weight deciders use AHP method for weighting，determine subjective weights by group weight determination method，fix objective weights via rough sets method，and then acquire combined weights by organic integration of subjective and objective weights.

When adopting AHP method to determine weight，the following procedure is conducted：（1）Break complex problems into multiple hierarchies and each hierarchy is made up of multiple indices；（2）Compare the indices of the same hierarchy in pairs using the next higher hierarchy element as the criterion to construct the judge matrix for weight vector determination；（3）Solve the judge matrix；（4）Evaluate the matrix′s coherence；（5）Obtain the weight of the base hierarchy［10］，and aggregate the calculated results of this layer by means of weighted summation method or weighted quadrature method，which can be used as the original indices of next higher hierarchy；（6）Repeat this calculation till the top hierarchy indices are worked out.

1.3 Normalization of index value

In view of the discrepancy of index value ranges，we normalize each item of capability.After normalization，index value of each item lies between 0and 1，which makes the values of different items compatible and consistent.This value is used as index value for the current index.For different evaluation indices，we adopt appropriate normalization methods，mainly including linear function，concave function，convex function and S shape function，as shown in Table 1.

Table 1 Normalization method for index value

1.4 Multi-criterion decision-making approach

The method of multi-criterion decision-making allows overall and wholeness evaluation on concepts，and thus prioritization and selection of different concepts.Therefore，it is suitable for aircraft concept decision-making.Commonly used multi-criterion decision-making methods include weighted summation method，weighted quadrature method，grey correlation projection method and ideal solution decision-making method，etc.

（1）Weighted summation method

Assume the index value as xi，and its weight is wi.The weighted summation result Siis

This relation reflects the total contribution of different indices to the next higher hierarchy indices.The indices are complementary，and zero value of any index will not cause zero value of next higher hierarchy index.

（2）Weighted quadrature method

Assume the index value as xi，and its weight is wi.The weighted quadrature result Piis

This relation highlights the indispensability of each index，any zero index can cause the next higher hierarchy index to become zero.

（3）Grey correlation projection method

The grey correlation projection method determines the figure of merit of each concept by calculating the grey correlation degree between alternative concepts and the ideal concept.The concepts can be evaluated by their grey correlation projection values.The evaluation process includes：

Firstly，build a grey correlation judge matrix.Assume that the value of the ideal concept is A0，and its normalized index value is｛x′01，x′02，…，x′0m｝.Consider x′0jas denominator，and x′ijas numerator，the gray correlation degree rijbetween alternative concept and the ideal concept can be written as

The grey correlation judge matrix R ＝［rij］n×mconsists of r，and the count is（n＋1）×m.

Secondly，introduce the weights of the evaluation indices to establish a grey correlation weight judge matrix R′＝［wjrij］n×m.

Thirdly，calculate the grey correlation projection values Gibetween alternative concept Aiand the ideal concept A0，which can be written as

If the value of Giis bigger，the project is better.And the highest grey correlation projection value indicates the best concept.

（4）Ideal solution decision-making

The process is as follows：

First，build normalized weight matrix as R＝［zij］n×m＝［wjxij′］n×m.

Second，calculate the relative closeness［11］for the ideal solution.Assume the weight ideal solution as Z＊＝｛z＊j｜j＝1，2，…m｝，set the weight negative-ideal solution as Z－＝｛zj－｜j＝1，2，…m｝.

The gap between alternative concept and ideal solution Z＊is

The gap between alternative concept and negative-ideal solution Z－is

The relative closeness between alternative concept and ideal solution Z－is

A bigger Diindicates a concept is closer to ideal solution，which means the concept is better.

（5）Integration of multi-criterion decisionmaking methods

Select the factorμj，according to preference for different decision-making methods，assume：μ1＋μ2＋μ3＋μ4＝1.The values of integrated evaluation and decision-making can be obtained via

2 EXAMPLE FOR AIRCRAFT CONCEPT EVALUATION AND DECISION-MAKING

2.1 Clarifying problem

In order to make up the shortage of existing manned combat aircraft and unmanned attack aircraft，a future air dominant unmanned air vehicle（UAV）is required for air-air combat，interception，and air control missions.This air dominant UAV should be a light weight unmanned combat air vehicle（UCAV）adopting a single engine and its takeoff weight should be less than 13tons.The highlights of conceptual design stage are integrated mission capability，development risks and life cycle cost，etc.

2.2 Scheming alternative concept

In view of these objectives，eight top-layer alternative concepts are proposed，as shown in Table 2，and design features of each project are qualitatively described.

Table 2 Eight alternative concepts

2.3 Concept evaluation based on effectiveness

To verify the validity and feasibility of hierarchy index system and decision-making method，eight alternative aircraft concepts are evaluated.Five experts are requested to give every hierarchy′s judge matrix，and decide corresponding weight of each index after consistency check.After that，we adopt group weight determination method to synthesize the weights obtained by the five experts.Then the weight of the evaluation index is obtained，as shown in Fig.1.The value of each index can be obtained via equations in Refs.［12-14］，and the values are normalized using methods given in Table 1.The index values and orders of the eight aircraft concepts are shown in Table 3.

Evaluation results from Table 3show thatthe precedence order of the concepts changes with the increase of considerations.In terms of integrated combat capability，the order is H＞E＞F＞D＞B＞A＞C＞G.Concept E and H are superior to other projects with their excellent flight performance and aerial combat firepower，and concept H shows obvious advantages.Concept F is ahead of D by slight advantages with superior survivability.Concept G ranks last for its poor flight performance and control ability，though its survivability is the best.In terms of integrated combat effectiveness，Concept E and H are still superior to other concepts.Concept H remains optimal.Concept D is ahead of F thanks to its high availability，while concept C lags behind G and ranks last for its poor availability.In respect of the integrated use effectiveness，concept H remains optimal，and D is ahead of E for its mature technology，further reducing its gap with concept H.Regarding the integrated use effectiveness，the order turns into D＞H＞E＞C＞F＞B＞A＞G.Concept D is ahead of H by slight advantages in life cycle cost and becomes the optimal solution.

Table 3 Concept evaluation result

2.4 Concept decision-making based on effectiveness

The general evaluation of concepts includes numerous considerations.As users vary in terms of technology，finance，and operation features，different decision-making results can be produced.Assume evaluation index as ｛integrated combat capability，dependability，research risk，endurance，security，life cycle cost｝is selected in concept evaluation and decision-making layer，the index weight vectors are ｛0.236，0.145，0.096，0.105，0.118，0.300｝，then the normalized data of concept evaluation indices are shown in Table 4.

Table 4 Normalized data of concept evaluation index

（1）Result of weighted summation method

The weighted summation result can be obtained via Eq.（1），as shown in Table 5.

（2）Result of weighted quadrature method

The weighted quadrature result can be obtained via Eq.（2），as shown in Table 5.

（3）Result of grey correlation projection method

Firstly，the grey correlation matrix can be obtained via Eq.（3）.Substituting weight vector Winto Eq.（3），agrey correlation weight judge matrix can be obtained.The grey correlation projection values of each concept can be obtained via Eq.（4），as shown in Table 5.

（4）Result of ideal solution decision-making method

The gap between evaluated concepts and ideal solution is ｛0.152，0.140，0.131，0.067，0.078，0.112，0.169，0.057｝，the gap between evaluated concepts and negative-ideal solution is｛0.310，0.320，0.360，0.388，0.376，0.341，0.285，0.391｝.The relative closeness between the evaluated concepts and ideal solution can be obtained through Eq.（7），as shown in Table 5.

（5）Synthesis results

Assumeμ1，μ2，μ3，μ4as 0.2，0.2，0.3，0.3.The values of integrated evaluation and decisionmaking can be obtained via Eq.（8），as shown in Table 5.

Table 5 Calculation result of multi-criterion decision-making

The final order is H＞D＞E＞C＞F＞B＞A＞G.From the analysis on Table 4－5，concept H is a more excellent scheme，which balances all the key indices better and shows more excellent overall result，though the individual capabilities are not prominent.On the contrary，some performance of concept G shows the best，but its overall effect becomes worst due to other poor key performance indices.

3 CONCLUSIONS

（1）This article introduces the effectiveness to establish the multi-criterion and multi-hierarchy evaluation index system in view of the limitations of existing evaluation systems.The topdown hierarchy structure for evaluation index system is：Integrated operation-cost effectiveness index，integrated operational effectiveness index，integrated combat effectiveness index，integrated combat capability index and performance index.This system covers various aspect of combat aircraft concept evaluation.Users can select interested indices to make a strategic decision.

（2）The weight assigning methods for the multi-criterion and multi-hierarchy evaluation index system and the normalization methods appropriate for different evaluation indices can completely and exactly reflect the importance and effect of each evaluation index.

（3）The best aircraft concept can be selected by applying a multi-criterion decision-making method which combines the grey correlation projection method，weighted summation method，weighted quadrature method and ideal solution decision-making method.The proposed method can help to improve aircraft concept decision-making.

［1］ Dong Yanfei，Wang Liyuan，Zhang Hengxi.Synthesized index model for fighter plane air combat effectiveness assessment［J］.Acta Aeronautical et Astronautic Sinica，2006，27（6）：1084-1087.（in Chinese）

［2］ Cartagena M A，Rosorio J E，Mavris D N.A method for technology identification，evaluation and selection of aircraft propulsion systems［R］.AIAA-2000-3712，2000.

［3］ Jilla C D，Miler D W，Sedwick R J.Application of multidisciplinary design optimization techniques to distributed satellite systems［J］.Journal of Spacecraft and Rockets，2000，37（4）：481-490.

［4］ Clemen R T.Making hard decisions：An introduction to decision analysis［M］.Boston，USA：PWS-Kent，1991.

［5］ Mavris D N，Delaurentis D.A stochastic design approach for aircraft affordability［R］.ICAS Paper 98-6.1.3，1998.

［6］ Hacker K.Comparison of design methodologies in the preliminary design of a passenger aircraft［R］.AIAA，99-0011，1999.

［7］ Wang Liruan，Zhang Hengxi，Xu Haojun.Multi-index synthesize evaluation model based on rough set theory for air combat efficiency［J］.Acta Aeronoutica et Astronautica Sinica，2008，29（4）：880-885.（in Chinese）

［8］ Xie Jianxi.Research on decision theory and engineering applications for top-level design of military aircraft［D］.Xi′an：Northwestern Polytechnical University，2007.（in Chinese）

［9］ Liu Xiaodeng.Selection decision approach and methodology of aircraft design［D］.Xi′an：Northwestern Polytechnical University，2005.（in Chinese）

［10］Xu Shube.Principle of analytical hierarchy process［M］.Tianjin：Tianjin University Press，1988.（in Chinese）

［11］Li Jun.Assessment and software development of modern transport′s integrated capability and effectiveness［D］.Xi′an：Northwestern Polytechnical University，2010.（in Chinese）

［12］Zhu Baoliu，Zhu Rongchang，Xiong Xiaofei.Combat aircraft effectiveness assessment［M］.Second Edition.Beijing：Aviation Industry Press，2006.（in Chinese）

［13］Zhang Hengxi，Guo Jilian，Dong Yanfei.Modern aircraft effectiveness and cost analysis［M］.Beijing：Aviation Industry Press，2007.（in Chinese）

［14］Tan Xiaowei，Fang Weiguo. A new assessment method for fighter＇s fighting effectiveness systems［J］.Engineering-Theory Methodology Applications，2004，13（1）：75-79.（in Chinese）