Cooperative multi-agent reinforcement learning for high-dimensional nonequilibrium control
Abstract
Self-assembly, the process by which interacting components form well-defined and often intricate structures, is typically thought of as a spontaneous process arising from equilibrium dynamics. When a system is driven by external mph{nonequilibrium} forces, states statistically inaccessible to the equilibrium dynamics can arise, a process sometimes termed direct self-assembly. However, if we fix a given target state and a set of external control variables, it is not well-understood i) how to design a protocol to drive the system towards the desired state nor ii) the energetic cost of persistently perturbing the stationary distribution. Here we derive a bound that relates the proximity to the chosen target with the dissipation associated with the external drive, showing that high-dimensional external control can guide systems towards target distribution but with an inevitable entropic cost. Remarkably, the bound holds arbitrarily far from equilibrium. Secondly, we investigate the performance of deep reinforcement learning algorithms and provide evidence for the realizability of complex protocols that stabilize otherwise inaccessible states of matter.
Graduate Student
Shriram is a PhD student in the Department of Chemistry at Stanford University, where he is advised by Grant Rotskoff. He is broadly interested in utilizing both data driven and theoretical approaches to more robustly simulate and characterize biological systems. His current work involves using Reinforcement Learning methods to control systems driven away from equilibrium. Shriram received a B.S. in Biological Chemistry with Honors from The University of Chicago, where he also completed a minor in Computer Science. At UChicago, Shriram worked on quantifying the kinetics of small RNA molecule regulation under the mentorship of Jingyi Fei. Outside the lab, Shriram is an avid Chicago sports fan, enjoys watching movies and exploring the local restaurant scene.