Optimal control
Optimal control theory is a branch of
Optimal control is an extension of the
General method
Optimal control deals with the problem of finding a control law for a given system such that a certain
We begin with a simple example. Consider a car traveling in a straight line on a hilly road. The question is, how should the driver press the accelerator pedal in order to minimize the total traveling time? In this example, the term control law refers specifically to the way in which the driver presses the accelerator and shifts the gears. The system consists of both the car and the road, and the optimality criterion is the minimization of the total traveling time. Control problems usually include ancillary constraints. For example, the amount of available fuel might be limited, the accelerator pedal cannot be pushed through the floor of the car, speed limits, etc.
A proper cost function will be a mathematical expression giving the traveling time as a function of the speed, geometrical considerations, and initial conditions of the system. Constraints are often interchangeable with the cost function.
Another related optimal control problem may be to find the way to drive the car so as to minimize its fuel consumption, given that it must complete a given course in a time not exceeding some amount. Yet another related control problem may be to minimize the total monetary cost of completing the trip, given assumed monetary prices for time and fuel.
A more abstract framework goes as follows.[1] Minimize the continuous-time cost functional
Linear quadratic control
A special case of the general nonlinear optimal control problem given in the previous section is the
Subject to the linear first-order dynamic constraints
A particular form of the LQ problem that arises in many control system problems is that of the linear quadratic regulator (LQR) where all of the matrices (i.e., , , , and ) are constant, the initial time is arbitrarily set to zero, and the terminal time is taken in the limit (this last assumption is what is known as infinite horizon). The LQR problem is stated as follows. Minimize the infinite horizon quadratic continuous-time cost functional
Subject to the linear time-invariant first-order dynamic constraints
In the finite-horizon case the matrices are restricted in that and are positive semi-definite and positive definite, respectively. In the infinite-horizon case, however, the matrices and are not only positive-semidefinite and positive-definite, respectively, but are also constant. These additional restrictions on and in the infinite-horizon case are enforced to ensure that the cost functional remains positive. Furthermore, in order to ensure that the cost function is bounded, the additional restriction is imposed that the pair is controllable. Note that the LQ or LQR cost functional can be thought of physically as attempting to minimize the control energy (measured as a quadratic form).
The infinite horizon problem (i.e., LQR) may seem overly restrictive and essentially useless because it assumes that the operator is driving the system to zero-state and hence driving the output of the system to zero. This is indeed correct. However the problem of driving the output to a desired nonzero level can be solved after the zero output one is. In fact, it can be proved that this secondary LQR problem can be solved in a very straightforward manner. It has been shown in classical optimal control theory that the LQ (or LQR) optimal control has the feedback form
For the finite horizon LQ problem, the Riccati equation is integrated backward in time using the terminal boundary condition
For the infinite horizon LQR problem, the differential Riccati equation is replaced with the algebraic Riccati equation (ARE) given as
Understanding that the ARE arises from infinite horizon problem, the matrices , , , and are all constant. It is noted that there are in general multiple solutions to the algebraic Riccati equation and the positive definite (or positive semi-definite) solution is the one that is used to compute the feedback gain. The LQ (LQR) problem was elegantly solved by Rudolf E. Kálmán.[9]
Numerical methods for optimal control
Optimal control problems are generally nonlinear and therefore, generally do not have analytic solutions (e.g., like the linear-quadratic optimal control problem). As a result, it is necessary to employ numerical methods to solve optimal control problems. In the early years of optimal control (c. 1950s to 1980s) the favored approach for solving optimal control problems was that of indirect methods. In an indirect method, the calculus of variations is employed to obtain the first-order optimality conditions. These conditions result in a two-point (or, in the case of a complex problem, a multi-point)
The approach that has risen to prominence in numerical optimal control since the 1980s is that of so-called direct methods. In a direct method, the state or the control, or both, are approximated using an appropriate function approximation (e.g., polynomial approximation or piecewise constant parameterization). Simultaneously, the cost functional is approximated as a cost function. Then, the coefficients of the function approximations are treated as optimization variables and the problem is "transcribed" to a nonlinear optimization problem of the form:
Minimize
Depending upon the type of direct method employed, the size of the nonlinear optimization problem can be quite small (e.g., as in a direct shooting or quasilinearization method), moderate (e.g.
Discrete-time optimal control
The examples thus far have shown
Examples
A common solution strategy in many optimal control problems is to solve for the costate (sometimes called the shadow price) . The costate summarizes in one number the marginal value of expanding or contracting the state variable next turn. The marginal value is not only the gains accruing to it next turn but associated with the duration of the program. It is nice when can be solved analytically, but usually, the most one can do is describe it sufficiently well that the intuition can grasp the character of the solution and an equation solver can solve numerically for the values.
Having obtained , the turn-t optimal value for the control can usually be solved as a differential equation conditional on knowledge of . Again it is infrequent, especially in continuous-time problems, that one obtains the value of the control or the state explicitly. Usually, the strategy is to solve for thresholds and regions that characterize the optimal control and use a numerical solver to isolate the actual choice values in time.
Finite time
This section may be confusing or unclear to readers. In particular, the law of evolution mentioned in the example is not mentioned in the article and is probably not the same as evolution. (October 2018) |
Consider the problem of a mine owner who must decide at what rate to extract ore from their mine. They own rights to the ore from date to date . At date there is ore in the ground, and the time-dependent amount of ore left in the ground declines at the rate of that the mine owner extracts it. The mine owner extracts ore at cost (the cost of extraction increasing with the square of the extraction speed and the inverse of the amount of ore left) and sells ore at a constant price . Any ore left in the ground at time cannot be sold and has no value (there is no "scrap value"). The owner chooses the rate of extraction varying with time to maximize profits over the period of ownership with no time discounting.
- Discrete-time version
The manager maximizes profit :
subject to the law of motion for the state variableForm the Hamiltonian and differentiate:
As the mine owner does not value the ore remaining at time ,
Using the above equations, it is easy to solve for the and series
and using the initial and turn-T conditions, the series can be solved explicitly, giving . - Continuous-time version
The manager maximizes profit :
where the state variable evolves as follows:Form the Hamiltonian and differentiate:
As the mine owner does not value the ore remaining at time ,
Using the above equations, it is easy to solve for the differential equations governing and
and using the initial and turn-T conditions, the functions can be solved to yield
See also
- Active inference
- Bellman equation
- Bellman pseudospectral method
- Brachistochrone
- DIDO
- DNSS point
- Dynamic programming
- Gauss pseudospectral method
- Generalized filtering
- GPOPS-II
- CasADi
- JModelica.org (Modelica-based open source platform for dynamic optimization)
- Kalman filter
- Linear-quadratic regulator
- Model Predictive Control
- Overtaking criterion
- PID controller
- PROPT (Optimal Control Software for MATLAB)
- Pseudospectral optimal control
- Pursuit-evasiongames
- Sliding mode control
- SNOPT
- Stochastic control
- Trajectory optimization
References
- ^ OCLC 625106088.
- ISBN 0-471-02594-1.
- OCLC 869522905.
- arXiv:2005.03186 [math.OC].
- ISSN 2405-8963.
- .
- .
- ISBN 978-0-9843571-0-9.
- ^ Kalman, Rudolf. A new approach to linear filtering and prediction problems. Transactions of the ASME, Journal of Basic Engineering, 82:34–45, 1960
- ^ Oberle, H. J. and Grimm, W., "BNDSCO-A Program for the Numerical Solution of Optimal Control Problems," Institute for Flight Systems Dynamics, DLR, Oberpfaffenhofen, 1989
- .
- ISBN 978-0-89871-688-7.
- ^ Gill, P. E., Murray, W. M., and Saunders, M. A., User's Manual for SNOPT Version 7: Software for Large-Scale Nonlinear Programming, University of California, San Diego Report, 24 April 2007
- ^ von Stryk, O., User's Guide for DIRCOL (version 2.1): A Direct Collocation Method for the Numerical Solution of Optimal Control Problems, Fachgebiet Simulation und Systemoptimierung (SIM), Technische Universität Darmstadt (2000, Version of November 1999).
- ^ Betts, J.T. and Huffman, W. P., Sparse Optimal Control Software, SOCS, Boeing Information and Support Services, Seattle, Washington, July 1997
- doi:10.2514/3.20223.
- ^ Gath, P.F., Well, K.H., "Trajectory Optimization Using a Combination of Direct Multiple Shooting and Collocation", AIAA 2001–4047, AIAA Guidance, Navigation, and Control Conference, Montréal, Québec, Canada, 6–9 August 2001
- ^ Vasile M., Bernelli-Zazzera F., Fornasari N., Masarati P., "Design of Interplanetary and Lunar Missions Combining Low-Thrust and Gravity Assists", Final Report of the ESA/ESOC Study Contract No. 14126/00/D/CS, September 2002
- ^ Izzo, Dario. "PyGMO and PyKEP: open source tools for massively parallel optimization in astrodynamics (the case of interplanetary trajectory optimization)." Proceed. Fifth International Conf. Astrodynam. Tools and Techniques, ICATT. 2012.
- OCLC 35140322.
- ^ Ross, I. M., Enhancements to the DIDO Optimal Control Toolbox, arXiv 2020. https://arxiv.org/abs/2004.13112
- ^ Williams, P., User's Guide to DIRECT, Version 2.00, Melbourne, Australia, 2008
- ^ FALCON.m, described in Rieck, M., Bittner, M., Grüter, B., Diepolder, J., and Piprek, P., FALCON.m - User Guide, Institute of Flight System Dynamics, Technical University of Munich, October 2019
- ^ GPOPS Archived 24 July 2011 at the Wayback Machine, described in Rao, A. V., Benson, D. A., Huntington, G. T., Francolin, C., Darby, C. L., and Patterson, M. A., User's Manual for GPOPS: A MATLAB Package for Dynamic Optimization Using the Gauss Pseudospectral Method, University of Florida Report, August 2008.
- ^ Rutquist, P. and Edvall, M. M, PROPT – MATLAB Optimal Control Software," 1260 S.E. Bishop Blvd Ste E, Pullman, WA 99163, USA: Tomlab Optimization, Inc.
- ^ I.M. Ross, Computational Optimal Control, 3rd Workshop in Computational Issues in Nonlinear Control, October 8th, 2019, Monterey, CA
- ^ E. Polak, On the use of consistent approximations in the solution of semi-infinite optimization and optimal control problems Math. Prog. 62 pp. 385–415 (1993).
- S2CID 7625851.
- S2CID 756939.
Further reading
- ISBN 1-886529-11-6.
- ISBN 0-470-11481-9.
- ISBN 0-387-90155-8.
- ISBN 0-444-01609-0.
- ISBN 0-13-638098-0.
External links
- Victor M. Becerra, ed. (2008). "Optimal control". Scholarpedia. Retrieved 31 December 2022.
- Computational Optimal Control
- Dr. Benoît CHACHUAT: Automatic Control Laboratory – Nonlinear Programming, Calculus of Variations and Optimal Control.
- DIDO - MATLAB tool for optimal control
- GEKKO - Python package for optimal control
- GESOP – Graphical Environment for Simulation and OPtimization
- GPOPS-II – General-Purpose MATLAB Optimal Control Software
- CasADi – Free and open source symbolic framework for optimal control
- PROPT – MATLAB Optimal Control Software
- OpenOCL – Open Optimal Control Library
- Elmer G. Wiens: Optimal Control – Applications of Optimal Control Theory Using the Pontryagin Maximum Principle with interactive models.
- On Optimal Control by Yu-Chi Ho
- Pseudospectral optimal control: Part 1
- Pseudospectral optimal control: Part 2
- Lecture Recordings and Script by Prof. Moritz Diehl, University of Freiburg on Numerical Optimal Control