Mission-driven autonomous perception and fusion based on UAV swarm

You HE

Research Institute of Information Fusion, Naval Aviation University, Yantai 264001, China

KEYWORDS

Abstract Distributed autonomous situational awareness is one of the most important foundation for Unmanned Aerial Vehicle(UAV)swarm to implement various missions.Considering the application environment being usually characterized by strong confrontation, high dynamics, and deep uncertainty, the distributed situational awareness system based on UAV swarm needs to be driven by the mission requirements,while each node in the network can autonomously avoid collisions and perform detection mission through limited resource sharing as well as complementarity of respective advantages. By efficiently solving the problems of self-avoidance, autonomous flocking and splitting, joint estimation and control, etc., perception data from multi-platform multi-source should be extracted and fused reasonably,to generate refined,tailored target information and provide reliable support for decision-making.

1. Introduction

In recent years, Unmanned Aerial Vehicles (UAVs) implemented as flying wireless networks have drawn tremendous attention in many military and civil society areas,such as battlefield situation monitoring,traffic control,border patrol,disaster emergency rescue,agricultural condition detection,aerial search and tracking of specific ground targets. Distributed situational awareness is the important foundation for UAV networks to implement various missions. Due to the diversification of tasks, limited time, limited platform resources,and uncertain targets,which are combined with various factors such as unknown obstacles and interference in the complex application environment, the situation monitoring problem becomes increasingly complicated while the performance of the task that a single UAV can perform will be extremely limited.1

With the rapid deployment of network information technologies, e.g., micro-embedded systems, battery technologies,low-cost Wi-Fi radio interfaces, and Global Positioning Systems(GPS),UAVs are currently equipped with a high computational power, sensors, and actuators, which can not only collect but also process data, make decisions and perform actions following a specific decision. Swarm intelligence has also been applied to UAV networks in the battlefield, where it is called military swarming.2Actually, forming a swarm of UAVs not only just increases the quality of situation awareness, but also extends the application area for UAVs. A UAV swarm can perform longer than a single UAV and cover more regions than a single UAV.Moreover,if one UAV of the swarm becomes compromised, others can continue the assigned mission. UAV swarm can be rapidly deployed to the region of interest in diversified delivery approaches and perform diversified tasks, including target search, tracking and recognition in collaboration with other platforms, and achieving high-precision and real-time surveillance of hot zone and high-value targets.3

Fixed fusion mode which can only produce single situation picture does not meet the needs of all UAV platforms in diverse missions. To support efficient and scientific dynamic decision-making,the corresponding perception and fusion system should implement the idea of ‘from sensor to shooter’ to achieve refined and tailored situation information.Thus,there must be an information processing architecture that can be automatically adjusted according to the specific requirements of the arranged mission and provide customized situation information for different users. Such an architecture is particularly important in resource-constrained complex networks.This will lead to incomplete, discontinuous and unreliable detection data, which will directly affect information fusion and is likely to result in unstable target tracking and track uncertainty. Therefore, situational awareness system based on UAV swarm under the condition of high dynamic networking poses new and greater challenges to the architecture and methods of multi-source information fusion (see Fig.1).

2. Mission-driven Autonomous Perception and Fusion (MAPF)

In a complex and highly dynamic network environment, the autonomous perception and fusion system based on UAV swarm needs to guarantee optimization control of platform motion under the premise of avoiding obstacles, and implement efficient target detection as well as information processing. A feasible way is that the system should be driven by mission requirements and based on resource allocation constraints such as sensors, communications and computing in mission planning. The flowchart of the proposed MAPF scheme is as follows.

Through real-time monitoring and assessment of situational awareness capability,MAPF system needs to start from mission requirement and should be able to quickly process network detection data,and then efficiently solve the problems of data extraction on demand, real-time processing and information sharing in a distributed architecture. The most important step is that the MAPF system can optimize the deployment of information fusion threads and fusion processing methods based on the existing limited resource allocation,so as to adapt to the dynamic adjustment of multi-platform collaboration and the frequent changes of network missions. In this way,we can obtain different situation products that meet different needs and transport them to product library before track quality assessment. In addition, target tracks in the library can be assigned to different application platforms via publish/subscribe mechanism.

The remainder of this paper is devoted to discussing several challenging technologies for the implementation of the abovementioned MAPF system, the goal of which is to generate multi-dimensional, global, real-time and accurate situation information products.As discussed below,the first challenging technology is to build an information requirement model to organically connect mission plan and perception/fusion methods.The significance of the second and third technologies is to guarantee the UAV swarm to carry out purposeful optimal control for sensing tasks. On the basis of the above technical support, the purpose of the remaining four technologies is to achieve accurate target tracking and recognition in complex situations.

2.1. Mission-oriented information requirement model for situation awareness

In the perception and fusion system, the information acquisition method based on mobile sensors is particularly important.The quantity and quality of detection data directly affect the effectiveness of situation awareness.4–5Therefore,it is an effective and energy-efficient way to obtain the necessary information according to the identified mission. First, the specific requirements of different mission segments for target information should be carefully analyzed according to the released mission planning. Then, the requirements are converted into qualitative or quantitative quality indexes, which can be used to set the demanding boundaries for different performance parameters, e.g., false track probabilities, track continuity and accuracy of target identification in different situation products. Reliable and effective optimization methods such as non-linear multi-objective decision-making should be introduced into the information requirement model.

2.2. Autonomous perception and avoidance based on UAV swarm

The technology of automatic avoidance based on UAV swarm has become more and more reliable,but the autonomous control combined with sensing task in complex environment needs further research.6With the constraints of limited detection and communication capabilities, efficient integration of swarm local rules and collision avoidance strategy should be simultaneously carried out with information-driven control method which can be used to drive the emergence of task-oriented swarm behavior of multiple UAVs. In addition, considering sensor detection capability, information processing speed and flight execution time, the dynamic characteristics of UAV networks may help guide the UAVs to escape the local extremum of flight path and find the fly-around way when the swarm encounters unknown obstacles or emergency threats.

2.3. Distributed collaborative coverage control for UAV swarm in key areas

For multi-target monitoring in key areas such as important ports, airports, and streets, we need to establish a distributed sensing mode with automatic coverage to avoid missing suspicious targets.7–9In order to achieve full-time surveillance for intensive targets, it is necessary to introduce a missionoriented information value concept and construct a Fisher information matrix which indicates the content of target information between different nodes in the swarm. Subsequently,an information-driven model with data exchange and computing can be used to control adjacent neighbor UAVs to move to the desired locations gradually.After a short period of time,all targets in key areas can be fully covered with only a few UAVs.

Fig.1 Flowchart of MAPF system.

2.4.Distributed cooperative tracking and identification based on UAV swarm

For the continuous tracking and identification of a few highvalued targets in some special fields such as distributed missile defense and forest fire detection, the main purpose of perception and fusion based on UAV swarm is to estimate the target’s trajectory in the sensing field and to predict the state of target in the near future.10–11The physical restriction on perception is that all the UAVs should maintain collision avoidance while tracking the targets because collision among autonomous UAVs will eliminate them in real life.12–15A feasible approach is to assume that motion control for the UAV swarm with the objective of target tracking and highperformance distributed estimation are coupled problems.And efficient distributed estimation algorithms should be designed to improve the collective information value of sensed data to achieve higher tracking performance and identification accuracy.

2.5. Autonomous flocking and splitting for UAV swarm in maneuverable target environments

Mobility control and coordination which deals with the optimal movement of UAV swarm is even more challenging when the situation awareness system is dealing with multiple maneuverable targets that change their speed and direction suddenly and frequently. Although flocking-based algorithms have demonstrated promising performance in tracking mobile targets,16they are not able to cover maneuvering targets as well as non-maneuvering ones, particularly when there are several flocks around several maneuvering targets. An autonomous flocking and splitting architecture needs to be built to determine when and how to split a cohesive flock into multiple subgroups for tracking different maneuvering targets respectively,and when and how to gather several or all the small flocks to a unified group for high-performance surveillance.

2.6. Dynamic adjustment mechanism of information fusion thread under multi-constraint conditions

Sensor nodes in UAV networks have several resource constraints, such as limited battery power, computation and communication capabilities.17–18However, sensor collaboration and reasonable data processing methods in an autonomous network can solve most of these issues and perform complex tasks more efficiently. On the other hand, MAPF system based on UAV swarm can develop different information fusion threads and perform different algorithm modules respectively to meet different detection needs. Besides,dynamic adjustment of the information fusion threads will be performed automatically according to the change of mission and network structure. There are two key problems to be solved, one is when and how to adjust the existing fusion threads, and the other is when and how to generate a new thread to get better estimates.

2.7. Target track processing in highly dynamic networks

Distributed track association and multi-source evidence fusion identification are the key technologies and also difficult problems of target track processing, which directly affect target track quality in the aspects of continuity, stability and consistency.19–20Especially in the case of highly dynamic changes in the position and attitude of UAVs as well as targets, detection accuracy and stability will be significantly affected, which will easily lead to high track fragmentation,inconsistency of track batch numbers, high conflict of recognition results, etc.21–22Therefore, it is necessary to analyze the characteristics of target detection data under highly dynamic network conditions and design robust track fusion algorithms to solve the problem of consistency of target position and attribute.

3. Summary

The traditional framework of situation awareness based on information fusion lacks the necessary control and optimization methods, and thus it is difficult to adapt to the complex and dynamic application environment. This paper proposes a mission-driven autonomous perception and fusion architecture based on UAV swarm, which can support automatic information acquisition and demand-oriented data processing according to the information requirements of application tasks. Several challenging technologies such as joint perception and avoidance, automatic flocking and splitting, cooperative tracking and identification are also discussed. While efficient collaboration between UAVs leads to better performance, the resulting increase in data processing also increases the communication complexity. Therefore, the tradeoffs between performance and limited resource utilization in information processing should always be considered in situation awareness.