Dipper Throated Optimization Algorithm for Unconstrained Function and Feature Selection

Ali E.Takieldeen, El-Sayed M.El-kenawy,2, Mohammed Hadwanand Rokaia M.Zaki

1Faculty of Artificial Intelligence, Delta University for Science and Technology, Mansoura, 35712, Egypt

2Department of Communications and Electronics, Delta Higher Institute of Engineering and Technology, Mansoura,35111, Egypt

3Department of Information Technology, College of Computer, Qassim University, Buraydah, 51452, Saudi Arabia

4Department of Computer Science, College of Applied Sciences, Taiz University, Taiz, Yemen

5Intelligent Analytics Group (IAG), College of Computer, Qassim University, Buraydah, Saudi Arabia

6Higher Institute of Engineering and Technology, Kafrelsheikh, Egypt

7Department of Electrical Engineering, Shoubra Faculty of Engineering, Benha University, Egypt

Abstract: Dipper throated optimization (DTO) algorithm is a novel with a very efficient metaheuristic inspired by the dipper throated bird.DTO has its unique hunting technique by performing rapid bowing movements.To show the efficiency of the proposed algorithm, DTO is tested and compared to the algorithms of Particle Swarm Optimization (PSO), Whale Optimization Algorithm (WOA), Grey Wolf Optimizer (GWO), and Genetic Algorithm(GA) based on the seven unimodal benchmark functions.Then, ANOVA and Wilcoxon rank-sum tests are performed to confirm the effectiveness of the DTO compared to other optimization techniques.Additionally, to demonstrate the proposed algorithm’s suitability for solving complex realworld issues, DTO is used to solve the feature selection problem.The strategy of using DTOs as feature selection is evaluated using commonly used data sets from the University of California at Irvine (UCI) repository.The findings indicate that the DTO outperforms all other algorithms in addressing feature selection issues, demonstrating the proposed algorithm’s capabilities to solve complex real-world situations.

Keywords: Metaheuristic optimization; swarm optimization; feature selection;function optimization

1 Introduction

Optimization is the process of obtaining the greatest or least objective function value for a setof inputs.It is the subject of various machine techniques that draw on artificial neural networks.Hundreds of famous optimization algorithms have become accessible, and dozens of technologies areavailable inmajor scientific code libraries.Given the problems of optimization, selecting whatmethods can thus be challenging [1].Optimization is how a function has the lowest or highest output of input parameters or arguments.In machines where the input parameters of the tasks like the floating-point values are numerical, continuous functions optimization frequently arises.The function returns an evaluation of the parameter of real-life [2].

Continuous function optimization may be used to distinguish between such problemswith discrete variables, known as combined optimization problems [3].Different techniques may be resolved, organized, and called to optimize the problems involving continuous functions.The information about the objective function utilized and used throughout the optimization process depends on one technique of optimization classification [4].The more information about the target function is generally understood, the easier it is to optimize because knowledge can be applied effectively [5].Perhaps the significant difference between optimization techniques is identifying the destination function in one location [6].It means that the first derivative of the feature may be used to identify a possible solution (gradient or route).It distinguishes itself from other not-calculated gradient data [7].Metaheuristic optimization is the optimization process utilizing metaheuristic techniques.Almost every area of life is involved, much from engineering to business, holiday preparation to internet travel [8].The use of those readily available resources must be maximized due to the continuous scarcity of money, resources, and time.The vast majority are non-linear, multimodal, and quite restrictive in real-life problems [9].Different objectives frequently collide.Even if one goal is set, Optimum solutions are not always available [10].Usually, a faultless or failed response is not simple to find.Many metaheuristic algorithms have been published, including swarm intelligence, anthrax optimization, optimization of the particulate swarm.In previous articles [11].The feature selection issuemay be seen as amulti-objective optimization problem in which two conflicting objectivesmust be met: picking the fewest possible features while attaining maximum classification accuracy [12].The solution with the most distinctive features and the maximum classification accuracy is deemed optimal [13].

2 Literature Review

Meta-heuristics refers to generic methods that normally used to solve complex and challenging combinatorial search problems.Generally, the problems solved by metaheuristic algorithms are challenging for computer scientists due to the need to examining a huge number of combinations that usually exponential with conflicting objectives [14].Many metaheuristic algorithms have been proposed to tackle real-world situation such as image segmentation [15], water allocation and crop planning [16], Nurse Rostering [17], power load dispatch [18], and Parkinson diagnosis [19].Several survey papers are available for more information about metaheuristic algorithms [20-23].

Nature-inspired metaheuristic algorithms to solve complex real-world situation have attractedthe attention of the researchers in the scientific community.Many new nature-inspired metaheuristic algorithms have been developed, including SymbioticOrganisms Search [24], Bat Algorithm (BA) [25], Bacterial Foraging Opt [26], Gravitational Search Algorithm [27], Firefly Algorithm (FA) [28], Krill Herd [29], Grey Wolf Optimization (GWO) algorithm [30,31], Cuckoo Search [32], Harmony search algorithm [33], Whale optimization [34], Social spider optimization [35], and Biogeography-basedOpt [36].

Several research paper can be found in the literature tackling feature selection as in [37-40].When it comes to feature selection, metaheuristic algorithms are instrumental because they deal with the dimensions of the data set to make predictions [38].However, when the dimensionality of the data sets is increased, the performance of classification methods suffers because of this.Furthermore, highdimensional data sets have several drawbacks, including a long model creation time, redundant data,and reduced performance, making data analysis very challenging [40].The feature selection step is a major preprocessing step that is used to resolve this problem.Its goal is to select a subset of features from an extensive data set while also increasing the accuracy of the classification or clustering model,resulting in the removal of noisy, extraneous, and ambiguous data.The following section present the proposed DTO optimizer.

3 Proposed Dipper Throated Optimization Algorithm

Dipper Throated bird is a member of the genus Cinclus in the bird family Cinclidae, so-called because of their bobbing or dipping movements see Fig.1.They are unique among passerines for their ability to dive, swim, and hunts underwater.Besides, it can fly rapidly and straight without pauses or glides because it has short and flexible wings.Dipper Throated bird has its unique hunting technique, it performs rapid bowingmovements, enhanced by the purewhite of the breast.Once the prey is detected, it dives headfirst into the water, even into the turbulent and fast-flowing water.When it became on the bottom, it turns up stones and pebbles, to disturb aquatic invertebrates, aquatic insects, and small fish.the Dipper walks on the bottom by grasping stones.It often walks against the current, with the head downwards to locate prey, it can be stable for a long time with its strong feet, also, it can walk into the water and deliberately submerge, by using its wings effectively and walk along with the bottom keeping its head well down and its body oblique to secures its food.,it dives headfirst into the water, even into the turbulent and fast-flowing water.When it became on the bottom, it turns up stones and pebbles, to disturb aquatic invertebrates, aquatic insects, and small fish.the Dipper walks on the bottom by grasping stones.It often walks against the current, with the head downwards to locate prey, it can be stable for a long time with its strong feet, also, it can walk into the water and deliberately submerge, by using its wings effectively and walk along with the bottom keeping its head well down and its body oblique to secures its food.

3.1 Mathematical Formulation

Mathematically, the Dipper Throated Optimization (DTO) algorithm assumes the birds are swimming and flying to search for food resourcesNfsavailable fornbirds.The birds’locations,BP,and velocities,BV, can be represented by the following matrices:

whereBPi,jindicatesithbirdinthejthdimensionfori∈1,2,3,...,nandj∈1,2,3,...,d.BVi,jindicatesithbird velocity in thejthdimension fori∈1,2,3,...,nandj∈1,2,3,...,d.The initial locations ofBPi,jare uniform distribution within lower and upper bounds.The fitness valuesf=f1,f2,f3,...,fnare calculated for each bird as in the following array

where the fitness value indicates the quality of food source searched by each bird.The optimal value means mother bird.These values are then sorted in ascending order.The first best solution in declared to beBPbest.The remaining solutions are supposed to be normal birdsBPndfor follower birds.The global best solution in declared to beBPGbest.

First DTO mechanism by this optimizer to update the swimming bird position is based on the following equation:

whereBPnd(t) is a normal bird position at iterationtandBPbest(t) is the best bird position.The“.”is pairwise multiplication.BPnd(t+ 1) is the updated bird position for the solution.

TheC1andC2are updated within the iterations by the following

wherecchanges from 2 to 0 exponentially,r1is a random value in [0,1] andTmaxin the total number of iterations.

Second DTO mechanism is based on updating the flying bird position and velocity by the following equations.The flying birds’positions are updated as

whereBPnd(t+ 1) is the new bird position for normal birds, and the updated velocity of each birdBV(t+ 1) is calculated as

whereC3is a weight value,C4andC5are constants.BPGbestis a random number inr2isarandom number in [0;1].

The DTO algorithm can be described by this equation

whereM=C2.BPbest(t) -BPnd(t) andRis a random value in [0,1].

Figure 1: White throated dipper

Algorithm 1: The DTO Algorithm Initialization positions BPi(i = 1,2,...,n) with size n,velocities BVi(i = 1,2,...,n), total number of iterations Tmax,fitness function fn, c, C1, C2, C3, C4, C5, r1, r2, R, t = 1 Calculate objective function fnfor each bird BPi Find best bird BPbest While t≤Tmax do for (i = 1 : i＜n + 1) do if (R＜0.5) then Update position of current swimming bird as BPnd(t + 1) = BPbest(t) - C1.|C2.BPbest(t) - BPnd(t)|else Update velocity of current flying bird as BV(t + 1) = C3BV(t) + C4r2(BPbest(t) - BPnd(t)) + C5r2(BPGbest- BPnd(t))Update position of current flying bird as BPnd(t + 1) = BPnd(t) + BV(t + 1)end if end for Calculate objective function fnfor each bird BPi Update c, C1, C2, R Find best bird BPbest Set BPGbest= BPbest Set t = t + 1 Return best bird BPGbest

3.2 Complexity Analysis

The computational complexity of the DTO algorithm can be expressed as follow.For populationnand iterationstmax, the time complexity will be defined as follows:

■InitializationBPi(i= 1,2,...,n),BVi(i= 1,2,...,n),Tmax,c,C1,C2,C3,C4,C5,r1,r2,R,t= 1:O(1).

■Calculate objective functionfnfor each birdBPi:O(n).

■Finding best birdBPbest:O(n).

■Updating position of current swimming bird:O(tmax×n).

■Updating velocity of current flying bird:O(tmax×n).

■Updating position of current flying bird:O(tmax×n).

■Calculating objective functionfnfor each birdBPi:O(tmax).

■Updatingc,C1,C2,R:O(tmax).

■Finding best birdBPbest:O(tmax).

■SettingBPGbest=BPbest:O(tmax).

■Settingt=t+ 1:O(tmax).

■Producing the best birdBPGbest:O(1)

From this analysis, the complexity of computations isO(tmax×n) andO(tmax×n×d) withddimension.

4 Experimental Results

The experiments in this section are explained in two sets.The first set of experiments is designed to evaluate the proposed DTO algorithm performance.The proposed DTO algorithm is tested compared to the algorithms of Particle Swarm Optimization (PSO), Whale Optimization Algorithm (WOA), GreyWolf Optimizer (GWO), and Genetic Algorithm (GA) based on the seven unimodal benchmark functions [19].Then,ANOVAandWilcoxon rank-sum tests are performed to confirmthe effectiveness of the proposed algorithm compared to other optimization techniques.The second is experimental in feature selection [20].

4.1 Evaluation of DTO Algorithm Unconstrained Function

Tab.1 shows a list of the unimodal benchmark function tested in the first experiment.Tab.1 shows the compared algorithms configuration.To be fair in the comparison, all the algorithms start with 20 agents, same objective function, 100 iterations, same dimensions, and boundaries.

Table 1: Descriptions of unimodal benchmark functions used in our experiments

Table 1: Continued

Fig.2 shows the convergence curves of the proposed DTO algorithm compared to the PSO,WOA, and GWO algorithms for the benchmark mathematical functions.Note that, the best convergence is achieved by the DTO algorithm.Tab.2 shows the mean and the standard deviation results based on the benchmark function, F1-F7, for different algorithms

Figure 2: (Continued)

Figure 2: The sample functions’graphical representations and convergence curves

Table 2: Mean and standard deviation results based on the benchmark function F1: F7

To test the statistical difference between the proposed DTO algorithm and the compared algorithms, a one-way analysis of variance (ANOVA) test is applied.In this test, two hypotheses are considered; the first is null hypothesis (H0:μDTO=μPSO=μWOA=μGWO=μGA) and the alternate hypothesis(H1:Means are not all equal).TheANOVAtest results are shown in Tab.3.Fig.3showsthe ANOVA test for proposed and the compared algorithmsvs.the objective function.From the results,the alternate hypothesisH1is accepted.The ANOVA test confirms the effectiveness of the proposed algorithm compared to other optimization techniques.

Table 3: ANOVA test results based on the benchmark function F1: F7

Table 3: Continued

Figure 3: (Continued)

Figure 3: ANOVA test for the proposed DTO and compared algorithms based on the benchmark functions

Wilcoxon’s rank-sum test is employed between every two algorithms to get the p-values betweenthe proposed DTO algorithm and other algorithms to show that DTO has a significant difference.The two hypotheses are the null hypothesis (H0:μDTO=μPSO,μDTO=μGWO,μDTO=μWOA, and μDTO=μGA)and the alternate hypothesis (H1: Means are not all equal).Tab.4 shows that the p-values are less than 0.05 between the proposed algorithm and other algorithms.This confirms the superiority of the DTO algorithm and that it is statistically significant.Thus, the alternate hypothesisH1is accepted.

Table 4: Wilcoxon’s rank-sum test results based on the benchmark function F1: F7

4.2 Evaluation of DTO Algorithm on Feature Selection Problem

IIn the case of the feature selection issue, the output solution should bemodified from a continuous solution to a binary solution utilizing the numbers 0 or 1.This function is often used to convert the continuous solution of an optimizer to a binary solution in an optimization problem.

4.2.1 Fitness Function

The quality of an optimizer’s solutions is determined by the fitness function that has been given to it.The error rate of classification and regression, as well as the features that have been picked from the input dataset, are the primary determinants of the function.It is advisable to choose a solution based on the collection of characteristics that can provide the bare minimum of features with the lowest classification error rate.The following equation is used in this study to evaluate the quality of the solutions provided.

As part of the experiments and comparative findings, the DTO evaluated our proposed algorithm against six datasets from the UCI machine learning library to determine its effectiveness.The datasets were chosen because they had a diverse range of features and occurrences that were reflective of the many types of problems that the proposed approach would be evaluated against.For more details about the UCI datasets, refer to Tab.5.

Algorithm 2: The bDTO Algorithm Initialization DTO Algorithm configuration, including population and parameters Change solutions to binary (0 or 1)Evaluate fitness function While t≤itersmaxdo Calculate objective function fnfor each bird BPi Find best bird BPbest While t≤Tmax do Apply DTO Algorithm Convert solutions to binary (0 or 1) using Eq.(9)Update parameters and best solution end while Return best solution

Table 5: Datasets description

4.2.2 Evaluation Metrics

The evaluation metrics used in this research are presented in Tab.6 as follows:

Table 6: Evaluation metrics

The average error of several algorithms is shown in Tab.7.Due to the decreased error, the optimizer has identified the optimal collection of features that can train the classifier while also producing a smaller error on the concealed test data.Tab.8 bellow shows the average features selected.Tab.9 for average fitness, Tab.10 for standard deviation fitness, Tab.11 for best fitness, and Tab.12 for worst fitness.The DTO has been able to find the superiority fitness for all datasets.

Table 7: Average error

Table 8: Average select size

Table 9: Average fitness

Table 10: Standard deviation fitness

Table 11: Best fitness

Table 12: Worst fitness

5 Conclusion

In this paper, a novel Dipper Throated Optimization (DTO) is introduced which is inspired bythe throated dipper bird.Six UCI machine learning database datasets and unconstrained function are used to prove the consistency of the suggested optimizer and guarantee that the proposed solution is dependable and stable to evaluate its quality and effectiveness.ANOVA and Wilcoxon rank-sum tests are used to compare the proposed algorithm to different optimization methods.The results showed clearly that, DTO outperformed all other compared methods.For future work, DTO needs more investigation by applying it to solve other well-known real-world combinatorial optimization problems.Based on the great success of DTO, the researchers can investigate the hybridizations betweenDTOand othermetaheuristic optimization algorithms as it is proved that hypermetaheuristic algorithms perform well compared to single metaheuristic algorithms.

Acknowledgement:The researcher would like to thank the Deanship of Scientific Research, Qassim University for funding the publication of this project.

Funding Statement:The authors received no specific funding for this study.

Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.