Necessary and sufficient optimality conditions for discrete time-invariant automaton-type systems

Abstract

The optimal control of deterministic discrete time-invariant automaton-type systems is considered. Changes in the system’s state are governed by a recurrence equation. The switching times and their order are not specified in advance. They are found by optimizing a functional that takes into account the cost of each switching. This problem is a generalization of the classical optimal control problem for discrete time-invariant systems. It is proved that, in the time-invariant case, switchings of the optimal trajectory (may be multiple instantaneous switchings) are possible only at the initial and (or) terminal points in time. This fact is used in the derivation of equations for finding the value (Hamilton–Jacobi–Bellman) function and its generators. The necessary and sufficient optimality conditions are proved. It is shown that the generators of the value function in linear–quadratic problems are quadratic, and the value function itself is piecewise quadratic. Algorithms for the synthesis of the optimal closed-loop control are developed. The application of the optimality conditions is demonstrated by examples.