A two-level measurement-based dynamic optimization strategy for a bioreactor in penicillin fermentation process☆

Qinghu Chi,Weijie Zhng ,Zhengshun Fei,Jun Ling ,*

a State Key Laboratory of Industrial Control Technology,Department of Control Science&Engineering,Zhejiang University,Hangzhou 310027,China

b Department of Measurement Technology and Instruments,Zhejiang University of Science and Technology,Hangzhou 310023,China

Keywords:Dynamic optimization Uncertainty NCO-tracking Solution model

ABSTRACT One measurement-based dynamic optimization scheme can achieve optimality under uncertainties by tracking the necessary condition of optimality(NCO-tracking),with a basic assumption that the solution model remains invariantin the presence of all kinds of uncertainties.This assumption is notsatisfied in some cases and the standard NCO-tracking scheme is infeasible.In this paper,a novel two-level NCO-tracking scheme is proposed to deal with this problem.A heuristic criterion is given for triggering outer level compensation procedure to update the solution model once any change is detected via online measurement and estimation.The standard NCO-tracking process is carried outat the inner level based on the updated solution model.The proposed approach is illustrated via a bioreactor in penicillin fermentation process.

1.Introduction

The optimization of dynamic processes has received growing attentions for years because of the demands in reducing production cost,improving product quality and satisfying safety requirements[1-4].Some new techniques are proposed for solving dynamic optimization problems.A hybrid improved genetic algorithm(HIGA)is proposed to deal with the problem of convergence in dynamic optimization[5].An approach that combined differential evolution(DE)algorithm and control vector parameterization(CVP)is proposed to improve the computing efficiency[6].However,due to the uncertainties from model mismatch and process disturbances,the optimal solutions based on nominal models are usually infeasible in practical applications[7,8].Hence,open-loop optimization is insufficient under uncertainties.

With the rapid development in measurement technology,two measurement-based optimization methods show great potential.As shown in Fig.1(a),the measurements are used to update the process model,and the numerical optimization procedure is implemented based on the updated model.The model refinement and optimization are carried out at each sampling instant,referred to as repeated optimization method[9-11].In Fig.1(b),measurements are used to adjust the optimal inputs directly by tracking the necessary conditions of optimality(NCO)in a closed-loop control scheme based on a special solution model,referred to as the NCO-tracking method[7].Without any explicit model updating or online re-optimization,its computation burden is much less than that with the repeated optimization method.

The NCO-tracking method has received lots of attentions because neither the knowledge about possible uncertainties nor the explicit model updating is needed for its online implement[12].Bonvin et al.used it to implement optimal grade transition for polyethylene reactors[12].Zhang et al.also introduced NCO-tracking into optimal grade transition in polymerization processes under uncertainty and proposed a new method to extract the solution model[13].Srinivasan et al.proposed to change set of active constraints using a barrier-penalty function,so the assumption regarding the active set is not required in NCO-tracking[14].Bonvin and Srinivasan addressed the role of NCO in structuring dynamic real-time optimization schemes[15].By using NCO-tracking in the optimization layer and self-optimizing control(SOC)in the lower control layer,J?schke and Skogestad demonstrated that the two methods complement each other,with SOC giving fast optimal correction for expected disturbances and the model free NCO-tracking procedure compensating other disturbances on a slower time scale[16].

Fig.1.Two schemes of measurement-based dynamic optimization.

In many investigations,the structure of solution model is assumed to be invariant under all kinds of process uncertainties in the NCO-tracking scheme[7].This assumption is considered to be satisfied in most cases for the following two reasons.First,the impact from parameter mismatch and process disturbances is limited,e.g.,a parameter may deviate from its nominal value but the deviation is limited[17].Second,basic understanding of the process,such as mechanism,dynamic characteristics as well as typical uncertainties,is usually provided,so the solution model obtained by numerical optimization can reflect the basic structure of the optimal input profiles for various uncertainties.

However,it is almost impossible to analyze the effects of all kinds of uncertainties because of the high complexity in modern industrial processes.Therefore,the rationality of the assumption cannot be guaranteed.One research about the optimal grade transition in polymerization processes[18]showed that inactive path constraint in the solution model could become active under disturbances.That is,the structure of solution model is changed.

With a possible change of the solution model,a supervisory system must be provided to monitor the change and compensate it timely,otherwise the control scheme based on an inappropriate solution model will not meet the necessary conditions of optimality.A special NCO-tracking approach has been proposed by Kadam et al.[18]to handle this problem,in which several kinds of possible changes in the solution model,e.g.,activation of nominally inactive path constraint,are found by offline analysis.An overriding control scheme is used to adjust the control strategy in real time if any of these changes occurs online.The main drawback of this approach lies in the difficulty to construct a candidate set of possible solution model change since it requires human experience and physical insight into the dynamic process,which is analytically expensive and often impossible to obtain.

In this paper,a two-level strategy for NCO-tracking scheme is proposed,in which the assumption on invariant solution model is not required.The scheme consists of an outer level and an inner level.The outer level is used to update the solution model when it changes under the uncertainties and the inner one is used to implement NCO-tracking based on the current solution model.A trigger unit is embedded into this two-level strategy to decide whether the solution model should be updated.With monitoring function in the trigger unit and compensation procedure in the outer level,the new NCO-tracking scheme can be normally implemented even if the solution model changes.Introducing the basic ideas of the repeated optimization,the two-level NCO-tracking scheme is proposed.The scheme is illustrated via a bioreactor in penicillin fermentation production.

2.Dynamic Optimization and NCO-tracking

Consider the following terminal-cost dynamic optimization problem:

where ?(·)denotes the terminal-cost objective function,x∈?nxdenotes the vector of state variables(states)with initial condition x0,u∈?nudenotes the vector of control variables(inputs),h(x(t),u(t))is the mixed state-input path constraints,e(x(tf))is the terminal constraints,ξ1,ξ2and σ are the dimensions of these constraint vectors,with ξ1+ ξ2= ξ,and t0and tfare the initial time and final time.

Using Pontryagin’s minimum principle,problem(P1)can be reformulated as minimizing the Hamiltonian function in the following form

The optimal solution of dynamic optimization must satisfy the necessary conditions of optimality,including constraint conditions,Eq.(7),and sensitivity conditions

NCO-tracking aims at achieving the optimality under uncertainty by converting the optimization problem to a control problem.Under the closed-loop control framework,conditions of constraints and sensitivities are obtained from process output through measurement or estimation.As shown in Fig.2,they are considered as controlled variables.They can be tracked to be zero to satisfy Eqs.(7)-(9)by adjusting the optimal input profile,which serve as the decision variables of the control framework.The essential work of constructing this control scheme includes appropriately parameterizing the input profile and assigning it to corresponding optimality objectives.After that,a solution model linking the decision variables and controlled variables can be generated.When the structure of the solution model is assumed invariant under uncertainties,the control framework based on the solution model can handle the uncertainties from model mismatch and process disturbance via feedback without the need of solving a dynamic optimization problem in real time.

3.Repeated optimization

In the presence of uncertainties,a measurement-based dynamic optimization problem can be formulated as follows

where Jlis the objective function at the l-th sampling instant,θ∈?nθrepresents the vector of uncertain parameters,which could be time invariant such as parameter mismatch or time-varying such as parameter drift and both of which can be considered as constants in a single sampling period compared with the high sampling frequency,yl∈?nyis the vector of output variables(outputs)at the l-th sampling time,and l=1,…,nl,where nlis the number of sampling taken within[t0,tf].The objective of problem(P3)at the l-th sampling time is to handle the uncertainties in θ and calculate the optimal input profiles u*(t)for the remaining time[tl,tf]by utilizing all the history measurements{y1,…,yl}from t0to current sampling time tl.

In the repeated optimization method,problem(P3)is split into two sub-problems,(P4)and(P5).In sub-problem(P4),the current states and parameters are estimated using online measurements.The estimation problem can be formulated as follows

Fig.2.The scheme of NCO-tracking.

Sub-problem(P5):determine the optimal input profiles in the remaining time[tl,tf]using the optimal solution of problem(P4).to update the model parameters.The optimal input profiles u*(t)in the remaining time[tl,tf]can be obtained by numerical optimization methods.

4.Two-level NCO-tracking Scheme

4.1.Framework of the two-level strategy

A novel two-level strategy of NCO-tracking scheme is proposed to relax the assumption of invariant solution model.With the development of online-measurement technology,states and parameters of dynamic model can be obtained via online measurement or estimation.Only the information of states is utilized to calculate the controlled variables of control scheme in the traditional NCO-tracking,while in this two-level NCO-tracking scheme,the information of states and parameters are utilized to obtain the real-time status of the solution model.Once a change is detected in the solution model,corresponding compensation will be implemented to maintain the feasibility of the NCO-tracking scheme.

As shown in Fig.3,the two-level NCO-tracking scheme decomposes the measurement-based optimization problem into an inner level standard NCO-tracking problem and an outer level repeated optimization problem.At the l-th sampling time tl,outputs of dynamic process ylare

When the outer level is triggered,a re-optimization is activated.Model parameter θ is updated by,the initial conditions of states x0,lare set to be（ tl）,and the corresponding time horizon of the optimization problem is shortened as[tl,tf].Finally,the optimal input profiles in the next sampling period u*(t),t∈[tl,tl+1],could be updated via solving this dynamic optimization problem.More importantly,the solution model can also be updated based on the new calculated optimal input profiles.

When the inner level is activated,the standard NCO-tracking scheme is implemented using the current solution model.In the closed-loop control scheme,NCOs,the feedback of manipulated variables,are obtained from the process outputs ylas well as the estimates（t）and

4.2.Discussion of the two-level NCO-tracking scheme

4.2.1.Online estimation

Industrial applications generally involve a vast amount of model parameters,hence the subset of candidate parameters should be chosen appropriately.A smaller subset of parameters generally requires lower excitation and means better performance in the parameter estimation,but insufficient candidate parameters will lead to completely erroneous model and a false optimum.

Model adequacy is a useful criterion to select candidate parameters[21].Among all the model parameters,the one with higher sensitivity with the objective function should be chosen as a candidate parameter[22].A sensitivity analysis method has been proposed to analyze the effect of single parameter on the optimization objective in the parametric nonlinear programming problem[23,24].Since a dynamic optimization problem can be converted to a nonlinear programming,this sensitivity analysis method can be used as a criterion for determining the subset of candidate parameters.

4.2.2.Trigger unit

Fig.3.The two-level NCO-tracking scheme.

In the two-level NCO-tracking scheme,the trigger unit activates the outer level to update the solution model only if necessary,otherwise it will activate inner level to implement a normal NCO-tracking process.An ideal trigger unit should judge whether the solution model changes with uncertainties in the last sampling period based on estimation information.A sensitivity analysis method is introduced into the NCO-tracking scheme to analyze if any change occurs in the solution model by handling a multi-parametric nonlinear programming problem[25].The main bottleneck of this approach is the high online computational burden,which is not allowed in the time-critical applications.

In the two-level NCO-tracking scheme,the optimal solution of the repeated optimization is not only used to update the solution model,but also used as the optimal input profiles in the next sampling period.In some cases,even if the solution model re-extracted from the latest optimal solution remains the same,the two-level NCO-tracking scheme still works well.Hence,complete accuracy is not an overwhelming concern in the judgment of trigger.On the other hand,online calculation is subject to strict time limit in the industrial applications.Since the judgment process is repeatedly implemented at each sampling time,decreasing the computational burden in the trigger unit will significantly reduce the overall calculation.Therefore,a sample heuristic judgment rule is adopted instead of introducing sensitivity analysis.

A parameter variation index SIlis used as the indicator in the trigger unit

where^θi,lis the estimate of the i-th parameter at the l-th sampling time,^θi,jis its estimation in the j-th updated solution model,wiis the corresponding weight coefficient,which could be obtained with reference to the off-line sensitivity analysis for each candidate parameter,and

Parameter variation index SIlshows the overall deviation of model parameters from their estimations in the latest solution model.With the threshold value of parameter variation index,SIref,the trigger criterion can be formulated as follows:if SIl≥SIref,the re-optimization process in the outer level is activated to calculate the optimal solution and update the solution model;if SIl＜SIref,the standard NCO-tracking process will be executed based on the unchanged solution model.

The heuristic trigger criterion is implemented repeatedly in each sampling time without increasing considerable computational load.It should be noted that,weight coefficient wiand threshold value SIrefshould be set appropriately,with off-line sensitivity analysis and human experience required.

5.Illustrative Example

5.1.Model description

In a fed-batch penicillin fermentation process,equations for a fed batch bioreactor are as follows[7].

where X,S and P represent the concentrations of biomass,substrate and product,respectively,V is the volume,u is feed flow rate,Sinis the substrate concentration of the inlet,μm,Km,Ki,and v are kinetic coefficients,and YXand YPare yield coefficients.

The optimization objective is to maximize the product concentration P at the end of a batch by manipulating the feed flow rate u,with lower bound uminand upper bound umax.There is also a constraint on the biomass concentration X due to the oxygen-transfer limitation.The dynamic optimization problem is formulated as

Initial conditions,model parameters and variable constraints are given in Table 1.

Table 1 Initial conditions,model parameters and variable constraints of the model

5.2.Optimal solution based on the nominal model

The optimal profiles of input and state in problem(P6),obtained by a direct sequential optimization method,are given in Fig.4.The feed flow rate u rises fast at the beginning in order to increase biomass concentration X as quickly as possible,while the substrate concentration S remains in its initial value.When biomass concentration X is about to reach its upper bound Xmax,u is lowered to uminto decrease S to approach zero.Once X=Xmax,u increases to a certain point to keep X at its upper bound,which guarantees the fastest growth rate of product concentration P.

Throughout the batch time,P increases and its growth rate is mainly influenced by X.As shown in Fig.4,P reaches and maintains its maximum growth rate once X reaches its upper bound.It can be concluded that the optimal input of this batch process forces the biomass concentration X to increase to its upper bound as soon as possible and keeps this path constraints active until the end of the batch.We denote the time at which X enters its upper bound as tX,tX=t|X(t)≥(1?γ),γ =0.05.tXis an auxiliary reference to reflect the quality of optimal input profiles.

5.3.Optimization results for the two-level NCO-tracking scheme

The process is simulated under three kinds of uncertainties.Parameters YXand Kmdeviate from their nominal values at t=t0to simulate model mismatch.The value of parameter YXincreases slowly at t＞t0to simulate parameter drift.The substrate inlet concentration Sinfluctuates in the form of sine wave to simulate the process disturbance.Profiles of these three parameters are shown in Fig.5.

Fig.4.Optimal input and state profiles based on the nominal model.

The optimal profiles under uncertainties using repeated optimization method(optimization process run 30 times)are given in Fig.6.Estimations of parameters YX,Sinand Kmat each sampling time are given in Fig.7.

Based on the nominal optimal solutions in Fig.4,the solution model can be extracted and the closed-loop control system can be implemented.Since this bioreactor process has been used as a simulation example to elaborate the NCO-tracking method in some papers[7,26],here we just give the closed-loop control scheme based on the solution model.

where κpsis a PI controller that tracks path sensitivity condition,Sref=S0is the set-point of controlled variable,and κpcis a PID controller that tracks path constraint condition.Implementing the control scheme by Eqs.(27)-(29),the optimal profiles with the standard NCO-tracking method and those with the two-level NCO-tracking scheme are shown in Fig.8.The results of openloop optimization under uncertainties(with the optimal input calculated based on the nominal model)are also shown as a comparison.The performance of different optimization strategies is given in Table 2.The optimal solution based on the nominal model is poor in the presence of uncertainties.The substrate concentration reduces significantly because of the model mismatch in parameter YX.It leads to inadequate growth of biomass and finally affects the fermentation yield.

Repeated optimization is performed with the refined model parameters and initial conditions,so the negative impact from uncertainties is compensated to some extent,as shown in Fig.6 and Table 2.However,within each sampling period,the latest calculated optimal input profile is still affected by the uncertainties from parameter drift and process disturbances.Therefore,a high sampling frequency is an important factor in the repeated optimization method.In this simulation,the reoptimization method is used 2,15 and 30 times during the batch process.It is clear that increasing the number of repetition,i.e.,increasing the sampling frequency,will improve the optimization performance.Unfortunately,high sampling frequency will increase the online computation burden significantly and bring a great risk of failure for the online optimization.

Fig.5.Profiles of three parameters.

Fig.6.Optimal input and state profiles with repeated optimization.

Fig.7.Estimation of the three parameters at each sampling time.

Fig.8.Optimal input and state profiles using different optimization approaches(solid line:two-level NCO-tracking;dash-dotted line:standard NCO-tracking;dotted line:open-loop optimization).

Table 2 Performance of different optimization approaches

As shown in Fig.8 and Table 2,the two NCO-tracking schemes can adjust the input profiles in real time to compensate the impact of uncertainties.The performance of the two-level NCO-tracking scheme is obviously better and fairly close to the repeated optimization method with 30 repetition times.Throughout the whole batch,the solution model in the standard NCO-tracking scheme is unchanged,while in the two-level NCO-tracking scheme,the outer level is activated by the trigger unit at t1=5 h and the solution model is updated.Within the remaining time t∈[t1,tf],the inner level is activated at each sampling time and the NCO-tracking is implemented using the latest solution model.In the new solution model,the value of Srefis increased from the original value Sref=S0=0.5 to Sref=0.71 in order to maintain the maximum growth rate of biomass.Eq.(21)indicates that the coefficient of biomass concentrationreflects the effect of substrate concentration on biomass growth.The relation between S and μ(S)at different sampling times are given in Fig.9.High substrate concentration will inhibit the biomass growth.The optimal substrate concentration S*for biomass growth can be calculated fromIn the nominal model,S*=S0,and this optimal operating point is transferred to S*′=S0+ ΔS due to the parameter mismatch in Km.It is also shown in the optimal solution of repeated optimization method(Fig.6),in which model parameter Kmis updated in the first sampling time.It is noticed that S fluctuates around the optimal operating value S*′during the first half of the batch,due to the disturbance in Sin,not Km.

Fig.9.The relation between S and μ(S)in the two models(dash line:t=t0,with nominal model parameters;solid line:t=t1,with updated model parameters).

6.Conclusions

A novel two-level NCO-tracking scheme is proposed.The outer level is used to update the solution model using online measurements and estimations,while the inner level is a standard NCO-tracking.A trigger unit with a parameter variation index is used to decide which level should be triggered to calculate the optimal input profiles.With this structure,the two-level NCO-tracking scheme can achieve optimality even if the uncertainty is large to change the solution model.This method is an extension of traditional NCO-tracking scheme and its advantages are outlined as follows.

·Relax the assumption that solution model is invariant under all kinds of uncertainties.

·Present better control performance than the standard NCO-tracking scheme when the solution model changes and gives the same performance when the solution model does not change.

·Present comparable performance with repeated optimization without additional computation burden.

The results of the illustrative example demonstrate that the two level online NCO-tracking scheme works effectively and efficiently.The comparisons with standard NCO-tracking scheme and repeated optimization show the superiority of the method.