MF-CFI:A fused evaluation index for camouflage patterns based on human visual perception

Xin Yang,Wei-dong Xu,Qi Jia,Jun Liu

National Key Laboratory of Lightning Protection and Electromagnetic Camouflage,Army Engineering University,Nanjing,Jiangsu,210007,China

Keywords: Camouflage evaluation Visual perception Effect assessment Fused index Eye movement analysis

ABSTRACT The evaluation index of camouflage patterns is important in the field of military application.It is the goal that researchers have always pursued to make the computable evaluation indicators more in line with the human visual mechanism.In order to make the evaluation method more computationally intelligent,a Multi-Feature Camouflage Fused Index (MF-CFI) is proposed based on the comparison of grayscale,color and texture features between the target and the background.In order to verify the effectiveness of the proposed index,eye movement experiments are conducted to compare the proposed index with existing indexes including Universal Image Quality Index (UIQI),Camouflage Similarity Index (CSI) and Structural Similarity (SSIM).Twenty-four different simulated targets are designed in a grassland background,28 observers participate in the experiment and record the eye movement data during the observation process.The results show that the highest Pearson correlation coefficient is observed between MF-CFI and the eye movement data,both in the designed digital camouflage patterns and largespot camouflage patterns.Since MF-CFI is more in line with the detection law of camouflage targets in human visual perception,the proposed index can be used for the comparison and parameter optimization of camouflage design algorithms.

1.Introduction

Camouflage technology is a very important means for military protection.It has been widely used by various countries and forces in important military targets like military personnel,weapons and equipment,and defense engineering facilities.The purpose of designing camouflage pattern is to make the military targets and the background highly integrated,so that the investigators or equipment cannot effectively identify the location of the targets[1,2].For fixed military targets,camouflage is easy to achieve;for mobile military targets,it is difficult to achieve high integration of targets and background in any part of the visual field,which often only reduces the saliency of the targets[3].Therefore,how to use an objective method to measure the reduction degree of this saliency is of great significance.

In traditional evaluation methods,the designed camouflage patterns are often placed in the field background and its camouflage effect is evaluated by the statistical data of observers [4],which is time-consuming and labor-intensive and cannot be widely promoted and applied due to weather factors.Researchers have proposed a large number of methods for the evaluation of camouflage effects in engineering practice [5-7].Cui et al.[8].decomposed the exposed features of a target into statistical features,shape features and texture features,and graded the camouflage effect through neural network fitting.Jia et al.[9].extracted features by studying the human visual perception mechanism and used neural networks to fit the detection probability.Lin et al.[10].fitted the four image features to the target detection function according to the mechanism of stimulation in psychology and related experiments.All of the above methods use the coarser index of detection probability as the final measure.

The existing research [11] has shown that eye movement measuring apparatus can detail the observer′s detection process.Through the eye movement measuring apparatus,both the subjective test results given by the observer and the instinctive response obtained by the stimulation can be obtained.These more detailed data include the data generated by the actions such as gaze,saccade,retrograde,and pupillary changes in the target searching process [12,13].Lin et al.[14].used different camouflage patterns to test the characteristics of eye movement.The results showed that eye movement data could be more sensitive to the effect of different camouflage patterns.Based on this conclusion,Brunyé et al.[15,16].investigated the significance of camouflage patterns during movement process.Volonakis et al.[17].proposed an evaluation model that conformed to human visual characteristics through the observation experiments by a camouflage helmet.It can be found from the above research that using eye movement data instead of detection probability as the standard of model testing can reduce the experimental cost and improve the accuracy.

The evaluation of the effect of camouflage patterns is affected by various conditions,such as observation distance,background characteristics,etc.So,it is a simple and easy solution to use the fused evaluation index between camouflage patterns and background.Lin et al.successively proposed the similarity evaluation indexes such as Quality Index(Q-index)[18],Camouflage Similarity Index(CSI)[19]and Universal Image Quality Index (UIQI) [20],and verified the rationality of the indexes through eye movement experiments.Among these indexes,Q-index and UIQI only compare the correlation between grayscale images,and CSI index only compares the fusion between image colors.However,these indexes cannot fully reflect the perception of visual detection to the features of color,structure,texture and shape in a comprehensive way.Lin et al.[21].designed a camouflage optimization method in La*b*color space based on CSI index.Xue et al.[22,23].designed an evaluation index for camouflage patterns based on the fusion of visual saliency features and mosaic features,and further proposed a camouflage design model with effect feedback.However,they did not fully verify the relationship between the evaluation index and vision.Therefore,it can be seen that the evaluation index of camouflage patterns is very important as it can provide a basis for the horizontal comparison of camouflage design schemes,the optimization of camouflage patterns and the evaluation problems in engineering practice.Based on this,this paper proposes an evaluation index for camouflage patterns,i.e.MF-CFI.The proposed index is developed based on the improvement of UIQI and combines the human visual perception process of color,texture,shape and structure.The effectiveness of the proposed index is verified by eye movement experiments.

2.Method

The evaluation index proposed in this paper is developed based on UIQI.So,UIQI is introduced first,and then the MF-CFI is proposed based on visual features.

2.1.Related research

UIQI was first proposed by Wang et al.[24].and applied to the evaluation of image distortion.UIQI combines three aspects of distortion,i.e.correlation loss,brightness distortion and contrast distortion.UIQI is more universal than the indexes like Mean Square Error(MSE)and Peak Signal to Noise Ratio(PSNR),due to its independence of the characteristics of the tested signal.Eqs.(1)-(6) provide the calculation steps of UIQI,wherebijandcijcorrespond to pixel values of the target and background image region,respectively,andMandNare the length and width of the image,respectively.In the calculation process,target and background images need to be scaled to the same size.

Lin et al.[14].demonstrated through eye movement experiments that there was a significant correlation between UIQI and each eye movement data when evaluating the effect of camouflage patterns.Compared with MSE and PSNR,UIQI accords more with the characteristics of eye movement.However,UIQI is only suitable for grayscale images,and it does not reflect the understandings by human vision for advanced features such as color,texture,and structure.Therefore,this paper makes an improvement to UIQI and proposes the MF-CFI to apply it to the evaluation of the fusion effect of camouflage patterns.

2.2.The approach of the MF-CFI

For camouflage patterns,the background region that affects the fusion effect is usually limited to the region around the target,and the influence of far region is greatly reduced.In addition,since the fusion of the target in the background image is only related to the local background,the comparison between full-frame background and small-scale camouflage pattern has little value.This paper adopts the commonly-used eight-connected domain method[2,8],as shown in Fig.1.That is,the pixel where the target stays is taken as the origin,and the surrounding eight regions of the same size are used as the background.When calculating the final result,the average of the eight regions is calculated as the partial effect of camouflage patterns.

Studies have shown that the underlying visual perception mechanism of human eyes decomposes an image into features such as color,shape,structure and texture before the further processing of high-level features [25,26].Itti [27] decomposed an image into multi-scale features of color,texture and grayscale when researching visual attention mechanism.This feature decomposition method only considers some of the elements in the process of human visual perception,but it has achieved good results in practice.Referring to this process,this paper proposed the MF-CFI where the similarity between target and background is compared in terms of the decomposed features of gray,color and texture.Eq.(7) shows the method for calculating the grayscale features of an image.Theavalue andbvalue in the Lab uniform color space are used as color features [28].When converting an image from RGB color space to Lab space,we need to first transfer the image to the XYZ color space and then to the Lab space[29].The texture features are described using a Gabor filter,as shown in Eqs.(8)-(10).The Gabor filter is the result of cosine modulation of the twodimensional Gaussian function.Biological experiments have shown that it can approximate the receptive field function of single cell [30].Four orientations are selected to extract the texture features for calculation,namely θ∈｛0°，45°，90°，135°｝.The value off0in Equation(8)is 0.1,and σ is set to 56 according to the bandwidth of 1.

Fig.1.Eight-connected domain method that contains the target and background.(a) Original image of target and background.(b) Schematic diagram of target and background,where T represents target and B represents background.

One grayscale feature map,two color feature maps,and four texture feature maps of an image can be obtained by the above calculation.UIQI is used to calculate the fusion of the feature maps of target and background,and then the two color similarities and the four texture similarities are averaged.Considering that the grayscale features,color features and texture features have the same position in the target detection process,the mean value of the three similarities is used as the fused evaluation index for the target and the background.Since the surrounding region of the target is divided into eight backgrounds,we first calculate the fused evaluation value of each background and obtain the average value,so as to the relatively complete evaluation result of MF-CFI in the position where the camouflage pattern stays.The range of MF-CFI value is [-1,1],and the value is 1 if and only if the background region is identical to the target region.The overall calculation process of UIQI is shown in Fig.2.

3.Experiment

In order to verify the effectiveness of the proposed index in evaluating the effects of camouflage patterns,the experiment is designed based on the camouflage search evaluation criteria by NATO seminar SCI-012[31]and the relevant military target search criteria by China.

3.1.Experimental apparatus and participants

The eye movement measuring apparatus used in the experiment is Tobii T120 with a sampling frequency of 60 Hz.The display has a size of 17 inches and resolution of 1920×1080.The computer host is connected to the eye movement measuring apparatus and display screen to record the eye movement and perform related calculations.The error calibration of eye movement should be performed before the experiment to ensure that the average error is not more than 0.5°in the experiment process.The distance of the participant′s eyes from the screen is controlled within 50-60 cm while the height of the eyes is approximately one-third of the top of the display in the vertical direction.

Twenty-eight male undergraduate students aged 19-24 participated in the experiment and they all had certain military literacy and basic military common sense.All participants had normal vision and no history of eye disease.All participants were informed of the experimental process before the experiment began and they were made sure to understand the objectives of the experiment.

3.2.Experimental method

An aerial optical background image with a radius of about 1 km was collected,and the shooting height was about 100 m.Digital camouflage pattern and large-spot camouflage pattern were designed for simulation.The texture generation model based on Markov random field proposed by Jia et al.[32].was used as the algorithm for generating digital camouflage pattern,with the number of main colors being 7.The design algorithm of large-spot camouflage pattern was implemented according to conventional method[33].The main color is obtained from the background color ratio,and the spot shape is manually drawn.When drawing spots of different colors,make sure that the ratio of the camouflage pattern is basically the same as the ratio of the main color in the background.In order to better conform to the visual perception in actual reconnaissance,the designed camouflage pattern was transformed into human figure,tank and vehicle with different postures.Table 1 shows the targets of different forms with camouflage patterns.According to the actual scale,the target was randomly synthesized into the background image as the observation material,as shown in Fig.3.Five images were selected from the acquired background,and 24 target images were synthesized into the selected background images randomly.In order to prove the advantages of MFCFI,UIQI,CSI and SSIM [34] were used for comparison.The correlation between the eye movement data and the indexes was calculated for the 24 targets.The higher the correlation,the more the indexes conformed to the mechanism of human visual perception.

Table 1 Simulation of camouflage targets with different poses.

Fig.2.Flow chart for calculating MF-CFI.

3.3.Experimental process

Fig.3.The collected grassland background and the simulated target to be observed.The red box indicates where the target is located.

Firstly,the experimental participants sat in front of the display to perform the calibration of eye movement to ensure that the error was no more than 0.5°.In this study,the discontinuous experiment method was used,that was,the synthesized simulated targetbackground image was presented for the participants in a random order.The display time of each material was 20 s.The observer needs to observe all 5 background images with the target embedded.If the participant thought that the target was completely found in the image,they could switch to the next group of observation task by pressing the button.During the switch from one group to another,a 10 s black screen was inserted as visual buffer to ensure that the reflection of the previous image would not affect the following observation.During the experiment,the precipitant needs to use the mouse to point out the region that was suspected as the target,and graded the difficulty level of the detection.The difficulty level was divided into 7 levels,and the higher the level,the more difficult to detect(see Fig.4).

In this study,all eye movement data was recorded for performance evaluation,including hit rate,detection time,difficulty level,the first saccade amplitude of region of interest(ROI),and fixation duration of ROI.Hit rate represents the ratio of the number of times the target was clicked by the participant to the number of participants.Detection time is the time taken from the start of observation to the click on the target region.Difficulty level is represented by the numbers of 1-7,with 1 representing the smallest detection difficulty and 7 representing the largest detection difficulty.ROI is defined as the minimum circumscribed rectangle of the target.Previous studies [35] have shown that the large saccade of the target indicated that the target itself could provide effective clues,and longer fixation duration indicated that the target could not be understood by the observer.

4.Results

In order to investigate the performance of different eye movement data in evaluating the indexes,the mean and variance of the eye movement data were analyzed for different forms of camouflage patterns.The results are shown in Table 2.The statistical results of these indexes indicated that the standard deviation of the fixation duration of ROI was relatively small,revealing that this index was relatively stable and it might be related to the detectability of the target.When the simulated target was detected,the observer would not spend too much time on the understanding of the target.This showed that this index was more effective in distinguishing between detectable and undetectable targets,and it was difficult for camouflage technology to achieve the complete fusion of background and moving target which could not be detected by the observer.The standard deviation of detection time was relatively large,indicating that this index was less stable.Analysis on the detection time of each target showed that the detection time had a significant correlation with the location of the target.The closer the target is to the center,the shorter the average detection time,and the smaller the standard deviation;the closer the target is to the edge,the longer the average detection time,and the larger the standard deviation.This indicated that human eyes tended to observe the central region at the initial observation,which confirmed to some extent the viewpoint by Srinivas et al.[36] who considered that the saliency assessment of human eyes had center deviation.According to the observers′reflection after the experiment,the decision to give the difficulty level was substantially influenced by the previously-observed target.It was difficult for the observers to determine how the specific difficulty level should be given,so most of the participants chose to make the decision by comparing the current image with the previous image.In the evaluation of the proposed index,the eye movement data of detection time which had obvious center deviation effect was removed to ensure the accuracy of the evaluation process.

Table 2 Mean and variance analysis of the eye movement data for large-spot camouflage and digital camouflage patterns,where M represents the mean and SD represents the standard deviation.

Compared to large-spot camouflage pattern,digital camouflage pattern had higher difficult to detect the eye movement data,especially for the indexes of difficulty level and the first saccade amplitude of ROI,which was consistent with the intuitive perception.Difficulty level showed certain subjectivity in comparing the current image with the previous one.By contrast,the first saccade amplitude of ROI could more objectively reflect this law.For the large-spot camouflage pattern,the mean values of difficulty level and fixation duration of ROI were both smaller than those for the digital camouflage pattern,while the value of hit rate of large-spot camouflage pattern was higher,indicating that the effect of the digital camouflage pattern was slightly better than the large-spot camouflage pattern from human visual perception.

Fig.4.Experimental steps.

In order to evaluate the effectiveness of the proposed MF-CFI,the Pearson correlation coefficient was used to measure the linear correlation between the proposed index and the eye movement data.The correlation was calculated for digital camouflage pattern and the large-spot camouflage pattern separately,and the results are shown in Tables 3 and 4.The results in the table show that compared with UIQI,CSI and SSIM,the proposed MF-CFI had the highest correlation with the eye movement data.Similar results were found in the digital camouflage pattern in Table 3 and the large-spot camouflage pattern in Table 4.UIQI and CSI performed differently on different eye movement data.This was because the focus of these two indexes was different.Specifically,UIQI was more suitable for structural features and CSI was only for the difference in the color between target and background.SSIM paid more attention to the structural features of the grayscale images,so its correlation with eye movement data was low and not more than 0.35.

Table 3 Correlation between eye movement data and the indexes for digital camouflage pattern.

Table 4 Correlation between eye movement data and the indexes for large-spot camouflage pattern.

Most of the values of correlation indexes for digital camouflage pattern were higher than those of the large-spot camouflage pattern,indicating that the color mixing principle made the fusion effect of digital camouflage pattern better than that of the largespot camouflage pattern.During the experiment,it was found that the hit rate of the large-spot camouflage pattern was mostly 100%,and the difficulty level was also concentrated between 2 and 4.These results were relatively concentrated and could not establish an effective linear relationship with the index.Therefore,the calculation results reflected that the effect of the digital camouflage pattern was better than that of the large-spot camouflage pattern.

Among the four eye movement data,the first saccade amplitude of ROI had the highest correlation.In fact,the first saccade amplitude of ROI was mainly from the stimulus by the underlying visual characteristics,which was an unconscious process for the observers.The results also showed that this eye movement data was evenly dispersed,and it could more acutely capture the significant changes in the target.However,in the experiment,the process of recording eye movement had an angular error,which would increase with the increase in the distance from the target to the center of the screen.The fixation duration of ROI had the lowest correlation.The experimental results showed that this index emphasized the understandability of the target,i.e.,the further recognition status after the target was detected.This process was closely related to the observer′s prior knowledge and had little to do with the background.Therefore,the fixation duration of ROI could be used as an evaluation index for target recognition,e.g.evaluating the effect of camouflage techniques such as obstacle masking and target camouflage.

5.Conclusion

This paper proposes a new evaluation index for camouflage pattern,namely MF-CFI.This index is based on the comparison between the target and the background in eight-connected domain,and conducts similarity evaluation for the features of grayscale,color and texture in the observation process.The results of eye movement experiments show that the proposed MF-CFI conforms more to the observation process than UIQI,CSI and SSIM,and has the highest correlation with eye movement data as indicated by Pearson correlation coefficient.In addition,experimental results show that the adopted eye movement data have different performances in evaluating the camouflage effects.Specifically,detection time is related to the distance from the target to the observation center,while the fixation duration of ROI is only related to the identifiability of the target,and has little correlation with the background.At the same time,the first saccade amplitude of ROI is more sensitive to reflecting the significance of the target.

As the proposed MF-CFI has good conformity to human visual perception,it can be applied to the optimization and evaluation of camouflage patterns as well as the comparison and evaluation of design algorithms.In addition,the use of eye movement data to develop camouflage patterns is an efficient and sophisticated method.Compared with the detection probability model,eye movement data can represent the details of the observation process in a more objective and accurate way.Future work needs to focus on the norms of eye movement experiment and the deep meanings of eye movement data,in order to provide a standard for the scientific evaluation of camouflage patterns.

Declaration of competing interest

The author claims that there is no conflict of interest.

Acknowledgments

This research was funded by Natural Science Foundation of Jiangsu Province &Key Laboratory Foundation,grant number is BK20180579 &6142206180204 respectively.