Analysis of the appropriate behavior of various types of automata in the conditions of the placement game

The object of research in this work is homogeneous collectives of automata with the property of purposeful behavior. The subject of this study is a comparison of different designs of such machines in the implementation of the conditions of the game of placement. The aim of the study is to establish the best (or similar) structures in terms of properties in order to optimize the time and computational costs of more complex machine learning models based on the principle of reinforcement learning. In the collectives under consideration, automata perform actions in a given habitat (functioning) with varying degrees of effectiveness. The automata, in accordance with their design, react to the input signal with another action. The evaluation of the effectiveness of the machine is defined as the sum of positive signals (rewards) or negative signals (penalties) received by the machine during the considered period of time. This characteristic depends on both the declared design of the machine and the depth of its memory. It is necessary to determine the simplest designs of automata that allow achieving optimal efficiency in a given environment in the shortest possible way. The formalization of both the properties of the environment and the actions of automata, as well as the processing of the results obtained, is carried out using the apparatus of game theory. In this case, the values of the effectiveness of the functioning of the machines are represented as the cumulative amounts of winnings and losses of the slot machine players. As result of the research the designs of automata that provide a given efficiency of functioning with a minimum depth of memory (the least complex design) are presented. The result obtained makes it possible to trace the influence of the inertial qualities of automata, implemented in the form of appropriate structures, on the efficiency of functioning in a given environment, formalized in the form of a game of placement. An automaton with linear tactics and a Krylov automaton form two marginal implementations of an automaton strategy for approaching the optimum. The first is due to the high speed of changing actions, the second is due to a long stay in states close to optimal. The field of application of the results obtained is further investigation of more complex dynamic environments using the simplest designs of automata, since synchronous collectives of automata in the process of computational implementation are difficult to parallelize, which leads to a significant increase in time and computational costs with the complication of the structure of dynamic environments or with an increase in these optimization tasks.

homogeneous group of machines, appropriate behavior, incentives, fines, reinforcement learning, machine, memory depth, game theory, optimal strategy, placement game