Reaction of CO Oxidation on the Surface of Pd Nanoparticles: Optimization by Reinforcement Learning
- 作者: Lifar M.1,2, Tereshchenko A.1, Bulgakov A.1,2, Guda A.1, Guda S.1,2, Soldatov A.1
-
隶属关系:
- The Smart Materials Research Institute, Southern Federal University
- Vorovich Institute of Mathematics, Mechanics, and Computer Sciences, Southern Federal University
- 期: 编号 3 (2023)
- 页面: 79-86
- 栏目: Articles
- URL: https://journals.rcsi.science/1028-0960/article/view/137727
- DOI: https://doi.org/10.31857/S1028096023030081
- EDN: https://elibrary.ru/LMOGVS
- ID: 137727
如何引用文章
详细
The yield of reaction products depends on the interaction between processes on the catalyst surface: adsorption, activation, reaction, desorption, and others. These processes, in turn, depend on the magnitude of the flows of reaction mixtures, temperature, and pressure. Under stationary conditions, active sites on the surface can be poisoned by reaction by-products or blocked by an excess of adsorbed reactant molecules. Dynamic control of reaction parameters takes into account changes in surface properties and adjusts temperature, flow rates and other parameters accordingly. A reinforcement learning algorithm was applied to control the oxidation reaction of carbon monoxide CO on the surface of palladium nanoparticles. The algorithm was trained to maximize the rate of carbon dioxide production based on information about the magnitude of CO, O2 and CO2 fluxes at each time step. A gradient policy algorithm with a continuous action space was chosen, and observations of the flow rates were extended over several successive time steps, which made it possible to obtain a set of non-stationary solutions. The maximum yield of the product is achieved with a periodic change in gas flows, which ensures a balance between the available adsorption sites and the concentration of activated intermediates. This methodology opens up prospects for optimizing catalytic reactions under nonstationary conditions.
作者简介
M. Lifar
The Smart Materials Research Institute, Southern Federal University; Vorovich Institute of Mathematics, Mechanics, and Computer Sciences, Southern Federal University
Email: guda@sfedu.ru
Russia, 344090, Rostov-on-Don; Russia, 344090, Rostov-on-Don
A. Tereshchenko
The Smart Materials Research Institute, Southern Federal University
编辑信件的主要联系方式.
Email: tereshch1@gmail.com
Russia, 344090, Rostov-on-Don
A. Bulgakov
The Smart Materials Research Institute, Southern Federal University; Vorovich Institute of Mathematics, Mechanics, and Computer Sciences, Southern Federal University
Email: guda@sfedu.ru
Russia, 344090, Rostov-on-Don; Russia, 344090, Rostov-on-Don
A. Guda
The Smart Materials Research Institute, Southern Federal University
编辑信件的主要联系方式.
Email: guda@sfedu.ru
Russia, 344090, Rostov-on-Don
S. Guda
The Smart Materials Research Institute, Southern Federal University; Vorovich Institute of Mathematics, Mechanics, and Computer Sciences, Southern Federal University
Email: guda@sfedu.ru
Russia, 344090, Rostov-on-Don; Russia, 344090, Rostov-on-Don
A. Soldatov
The Smart Materials Research Institute, Southern Federal University
Email: guda@sfedu.ru
Russia, 344090, Rostov-on-Don
参考
- Pakhare D., Spivey J. // Chem. Soc. Rev. 2014. V. 43. № 22. P. 7813. https://doi.org/10.1039/C3CS60395D
- Pareek V., Bhargava A., Gupta R., Jain N., Panwar J. // Adv. Sci. Eng. Med. 2017. V. 9. № 7. P. 527. https://doi.org/10.1166/asem.2017.2027
- Kinoshita K. // J. Electrochem. Soc. 1990. V. 137. № 3. P. 845. https://doi.org/10.1149/1.2086566
- Rojluechai S., Chavadej S., Schwank J.W., Meeyoo V. // Catal. Commun. 2007. V. 8. № 1. P. 57. https://doi.org/10.1016/j.catcom.2006.05.029
- DeSantis C.J., Peverly A.A., Peters D.G., Skrabalak S.E. // Nano Lett. 2011. V. 11. № 5. P. 2164. https://doi.org/10.1021/nl200824p
- Sun C., Cao Z., Wang J., Lin L., Xie X. // New J. Chem. 2019. V. 43. № 6. P. 2567. https://doi.org/10.1039/C8NJ05152F
- Vatti S.K., Ramaswamy K.K., Balasubramanaian V. // J. Adv. Nanomat. 2017. V. 2. № 1. P. 127. https://doi.org/10.22606/jan.2017.22006
- Cuenya B.R. // Thin Solid Films. 2010. V. 518. № 12. P. 3127. https://doi.org/10.1016/j.tsf.2010.01.018
- Schalow T., Brandt B., Laurin M., Schauermann S., Libuda J., Freund H.J. // J. Catal. 2006. V. 242. № 1. P. 58. https://doi.org/10.1016/j.jcat.2006.05.021
- Skorynina A., Tereshchenko A., Usoltsev O., Bugaev A., Lomachenko K., Guda A., Groppo E., Pellegrini R., Lamberti C., Soldatov A. // Rad. Phys. Chem. 2018. V. 175. № 1. P. 108079. https://doi.org/10.1016/j.radphyschem.2018.11.033
- Albers P., Pietsch J., Parker S.F. // J. Mol. Catal. A. 2001. V. 173. № 1–2. P. 275. https://doi.org/10.1016/S1381-1169(01)00154-6
- Gromotka Z., Yablonsky G., Ostrovskii N., Constales D. // Entropy. 2021. V. 23. № 7. P. 818. https://doi.org/10.3390/e23070818
- Armstrong C.D., Teixeira A.R. // React. Chem. Eng. 2020. V. 5. № 12. P. 2185. https://doi.org/10.1039/D0RE00330A
- Cutlip M., Hawkins C., Mukesh D., Morton W., Kenney C. // Chem. Eng. Commun. 1983. V. 22. № 5–6. P. 329.https://doi.org/10.1080/00986448308940066
- Vaporciyan G., Annapragada A., Gulari E. // Chem. Eng. Sci. 1988. V. 43. № 11. P. 2957. https://doi.org/10.1016/0009-2509(88)80049-6
- Schwankner R., Eiswirth M., Möller P., Wetzl K., Ertl G. // J. Chem. Phys. 1987. V. 87. № 1. P. 742. https://doi.org/10.1063/1.453572
- Eiswirth M., Ertl G. // Phys. Rev. Lett. 1988. V. 60. № 15. P. 1526. https://doi.org/10.1103/PhysRevLett.60.1526
- Newton M.A., Ferri D., Smolentsev G., Marchionni V., Nachtegaal M. // Nat. Commun. 2015. V. 6. № 1. P. 8675. https://doi.org/10.1038/ncomms9675
- Fang H., Haibin L., Zengli Z. // Int. J. Chem. Eng. 2009. V. 2009. № 1. P. 710515. https://doi.org/10.1155/2009/710515
- Moghtaderi B. // Energy Fuels. 2012. V. 26. № 1. P. 15. https://doi.org/10.1021/ef201303d
- Yoshida H., Kakei R., Fujiwara A., Tomita A., Miki T., Machida M. // Top Catal. 2019. V. 62. № 1. P. 345. https://doi.org/10.1007/s11244-018-1100-5
- Toyao T., Maeno Z., Takakusagi S., Kamachi T., Takigawa I., Shimizu K.-I. // ACS Catal. 2019. V. 10. № 3. P. 2260. https://doi.org/10.1021/acscatal.9b04186
- Segler M.H.S., Preuss M., Waller M.P. // Nature. 2018. V. 555. № 7698. P. 604. https://doi.org/10.1038/nature25978
- Kaelbling L.P., Littman M.L., Moore A.W. // J. Artif. Intell. Res. 1996. V. 4. P. 237. https://doi.org/10.1613/jair.301
- Sutton R.S., Barto A.G. Introduction to Reinforcement Learning. Cambridge: MIT Press, 1998. P. 380.
- Littman M.L. // Nature. 2015. V. 521. № 7553. P. 445. https://doi.org/10.1038/nature14540
- Neumann M., Palkovits D.S. // Ind. Eng. Chem. Res. 2022. V. 61. № 11. P. 3910. https://doi.org/10.1021/acs.iecr.1c04622
- Watkins C.J. Learning from Delayed Rewards: PhD Thesis. Cambridge: King’s Colledge, 1989. 242 p.
- Watkins C.J., Dayan P. // Mach. Learn. 1992. V. 8. № 3. P. 279. https://doi.org/10.1007/BF00992698
- Lillicrap T.P., Hunt J.J., Pritzel A., Heess N., Erez T., Tassa Y., Silver D., Wierstra D. Continuous Control with Deep Reinforcement Learning; https://arxiv.org/ abs/1509.02971.pdf.
- Alhazmi K., Albalawi F., Sarathy S.M. // Chem. Eng. J. 2022. V. 428. P. 130993. https://doi.org/10.1016/j.cej.2021.130993
- Zhou Z., Li X., Zare R.N. // ACS Cent. Sci. 2017. V. 3. № 12. P. 1337. https://doi.org/10.1021/acscentsci.7b00492
- Engel T., Ertl G. // Elementary Steps in the Catalytic Oxidation of Carbon Monoxide on Platinum Metals. Munchen: Elsevier, 1979. P. 43.
- Chorkendorff I., Niemantsverdriet J.W. // Concepts of Modern Catalysis and Kinetics. Weinheim: John Wiley & Sons, 2017. P. 66.
- Libuda J., Meusel I., Hoffmann J., Hartmann J., Piccolo L., Henry C., Freund H.-J. // J. Chem. Phys. 2001. V. 114. № 10. P. 4669.
- Nelder J.A., Mead R. // The Comput. J. 1965. V. 7. № 4. P. 308. https://doi.org/10.1093/comjnl/7.4.308
- Unni M., Hudgins R., Silveston P. // Can. J. Chem. Eng. 1973. V. 51. № 6. P. 623. https://doi.org/10.1002/cjce.5450510601
- Abdul-Kareem H.K., Silveston P., Hudgins R. // Chem. Eng. Sci. 1980. V. 35. № 10. P. 2077. https://doi.org/10.1016/0009-2509(80)85029-9
- Abdul-Kareem H.K., Hudgins R., Silveston P. // Chem. Eng. Sci. 1980. V. 35. № 10. P. 2085. https://doi.org/10.1016/0009-2509(80)85030-5
- Zhou X., Barshad Y., Gulari E. // Chem. Eng. Sci. 1986. V. 41. № 5. P. 1277. https://doi.org/10.1016/0009-2509(86)87100-7