Output-Feedback Based Simplified Optimized Backstepping Control for Strict-Feedback Systems with Input and State Constraints

Jiaxin Zhang, Kewen Li, and Yongming Li,

Abstract—In this paper, an adaptive neural-network (NN)output feedback optimal control problem is studied for a class of strict-feedback nonlinear systems with unknown internal dynamics, input saturation and state constraints. Neural networks are used to approximate unknown internal dynamics and an adaptive NN state observer is developed to estimate immeasurable states. Under the framework of the backstepping design, by employing the actor-critic architecture and constructing the tan-type Barrier Lyapunov function (BLF), the virtual and actual optimal controllers are developed. In order to accomplish optimal control effectively, a simplified reinforcement learning (RL) algorithm is designed by deriving the updating laws from the negative gradient of a simple positive function, instead of employing existing optimal control methods. In addition, to ensure that all the signals in the closed-loop system are bounded and the output can follow the reference signal within a bounded error, all state variables are confined within their compact sets all times. Finally, a simulation example is given to illustrate the effectiveness of the proposed control strategy.

I. INTRODUCTION

IN the last decade, fuzzy logic systems (FLSs) and NNs were widely used in adaptive backstepping recursive control design [1]-[3]. In [1], direct adaptive NN control was presented for a class of nonlinear systems with unknown nonlinearities. The authors focused on adaptive fuzzy tracking control in [2] for a class of nonlinear systems. The result [3]developed two different backstepping NN control approaches for a class of strict-feedback systems with unknown nonlinearities. In [4], the fuzzy logic systems and error transformation-based method were used in online learning of completely unknown dynamics and prescribed performance tracking, respectively. The authors developed a finite-time adaptive fuzzy control strategy for a class of nonlinear strictfeedback systems in [5]. Furthermore, the authors in [6]proposed a global nested PID control method for nonlinear systems with unknown system nonlinearities without linearized approximators. However, it is worth mentioning that the above-mentioned adaptive backstepping control methods all assume that the states of the systems are measurable and can be used for control design directly.

As pointed out in [7]-[10], in practice, state variables were often unmeasured for many nonlinear systems. The authors in[7]-[10] designed different state observers, and some intelligent adaptive output feedback control approaches were developed for a class of uncertain nonlinear systems with immeasurable states. Although the great progress has been made in intelligent adaptive control for nonlinear systems, the constraint problems were not fully considered.

In engineering control, saturation, dead zones and timedelay are common phenomena, all stemming from the existence of control constraints. Once the control is constrained, the stability of the nonlinear system is often difficult to guarantee. In [11]-[18], the control problems for nonlinear systems with full-state constraints and partial state constraints were studied. The stability was guaranteed without violation of any constraints. In order to clarify the effect of control constraints on system stability, many scholars investigated such problems based on the BLF. The authors proposed an indirect adaptive fuzzy controller in [19] for a class of uncertain nonlinear systems with input and output constrains. In [20], an adaptive fault-tolerant control (FTC)scheme was proposed for a class of nonlinear systems with control inputs and system state constraints. The authors designed an adaptive fuzzy control scheme in [21] for a class of uncertain nonlinear systems with input saturation and output constraints. In [22], the authors addressed the cooperative control problem for multiple high-speed trains,which guaranteed that the speed and the position of highspeed trains were confined to specific speed limitations, and allowed distances ratified by the automatic train protection and the moving authority, respectively. Even though various intelligent control strategies [11]-[22] have been devised in the constraints problem for nonlinear dynamics, optimization in control design and stability analysis has not been considered therein.

As the foremost branch of modern control theory, optimal control was developed by Bellman [23] and Chambers [24] 50 years ago. Since then, some significant results were reported,for example in [25]-[33]. In [25], a novel RL-based robust adaptive controller was developed for the continuous-time(CT) uncertain nonlinear systems with input constraints. The authors developed an adaptive RL solution in [26] for the infinite-horizon optimal control problem of constrained-input continuous-time nonlinear systems in the presence of nonlinearities with unknown structures. In [27], an optimal NN control scheme was presented for CT nonlinear systems with asymmetric input constraints. The authors in [28]proposed an integral reinforcement learning (IRL) algorithm on an actor-critic structure for a class of affine nonlinear systems, wherein the partially-unknown constrained-input was considered. The finite-time optimal control problem was studied in [29] for the high-order nonlinear systems whose powers were positive odd ratio numbers. However, all of the above adaptive optimal control methods are limited to affine nonlinear systems and thus cannot be applied to nonlinear systems with strict-feedback. To handle this issue, a control technique called optimized backstepping (OB) was first proposed in [30] by implementing tracking control for a class of strict-feedback systems. Recently, the authors in [31]investigated an adaptive RL optimal control design problem for a class of nonstrict-feedback discrete-time systems. In order to accomplish optimal control effectively, the authors designed a simplified RL algorithm in [32] instead of employing the existing RL-based optimal control methods.

Although an optimized control method was developed in[32] based on the OB technique using simplified RL for nonlinear systems, input saturation and state constraints under unpredictable systems states were not considered. Based on the above results, this paper proposes an optimal control scheme based on NN approximation for a class of strictfeedback systems with unknown dynamics, input saturation and state constraints. Compared with the existing works, the main contributions of this paper are listed in the following.

1) In this paper, an adaptive NN backstepping output feedback simplified optimal control method is proposed for a class of uncertain nonlinear systems with unmeasured states,input saturation and state constraints. The tan-type barrier optimal cost functions are constructed for subsystems. In contrast with [30], the method proposed here does not require priori knowledge due to the utilization of the state observer.

2) By separating the optimal value function into a novel error form, the proposed control strategy can effectively solve the optimal tracking control problem. Unlike [30] and [32],this paper adopts a stepwise optimization strategy to analyze the stability of each step of the system. Each controller is designed in this paper to be the optimal solution for the corresponding subsystem, thus optimizing the control of the whole system.

II. PRELIMINARIES

A. Problem Statement

Consider the following strict-feedback nonlinear systems as:

Control Objective:The control objective of this paper is to obtain a NN backstepping output feedback optimal control that not only stabilizes system (1), but also minimizes the value function, while ensuring that all the closed-loop signals are guaranteed to be uniformly ultimately bounded (UUB).All the system states are ensured not to transgress their constrained sets so that the outputycan track the reference signalyr.

B. Neural Networks

III. MAIN RESULT

A. State Observer Design

In this section, a state observer needs to be designed to estimate the unmeasured states. Then, under the actor-critic architecture, a NN adaptive backstepping output feedback optimal controller will be designed based on the designed state observer. Finally, a stability analysis of the closed-loop system is given to prove our main conclusions. Rewrite system (1) as the following state space expression form:

B. Output Feedback Optimized Controller Design and Stability Analysis

The optimal value function for thesi-subsystem is defined as

From (91), (95), and (96), the approximate Hamiltonian is

The Bellman error is defined as

The critic NN adaptive law is given as

Consider the overall Lyapunov function candidate for the final step as

there are the following equations:

Substituting (104) and (105) into (103), one has

Using Young’s inequality, there is the following fact that:

Substituting (107)-(110) into (106), one has

The simulation results are shown by Figs. 1-8. Fig. 1 shows the control outputyand the reference signalyr, it is clear that an ideal tracking performance can be obtained. Figs. 2 and 3 show the trajectories of statesxiand their estimates, (i=1,2) along with |x1|≤kc1and |x2|≤kc2, respectively.Figs. 4-6 profile the 2-norm of the weights for the critic, actor and observer NN; Figs. 7 and 8 display the trajectories of controlleruwithout input saturation and with input saturation,respectively.

Fig. 1. The trajectories of y and yr.

Fig. 2. The trajectories of x1 and x?1.

Fig. 3. The trajectories of x2 and x?2.

Fig. 4. The trajectories of NN weights and

Fig. 5. The trajectories of NN weights and

Fig. 6. The trajectories of NN weights and

It can be clearly observed from the simulation results that the proposed control method ensures all signals in the closedloop system are UUB, that the system outputycan track the given reference signal, and that all the system states are ensured not to violate any constraints.

Fig. 7. The trajectory of controller u (t) without input saturation.

Fig. 8. The trajectory of controller u (t) with input saturation.

V. CONCLUSION

In this paper, an optimal control has been developed based on the backstepping technique using a simplified RL for a class of uncertain nonlinear systems with unmeasured states,input saturation and state constraints. The immeasurable states were approximated by the state-observer. At the same time,the tan-type BLF has been introduced to vary the constraint boundary. Meanwhile, the control design can also release the condition of persistent excitation. Based on the Lyapunov method, it was proven that the proposed adaptive NN optimal controller can ensure that the closed-loop system is UUB. In addition, the tracking error of the system converges to a small neighborhood of the origin and all states did not violate their constraints. Finally, the simulation further demonstrated the effectiveness of the proposed control method. One possible research point for future research is to extend the SISO system in this work to the MIMO case with milder assumptions.