Santiago Paternain, “Safe Learning for Dynamical Systems and Control”

Abstract:

Reinforcement learning has shown great success in controlling complex dynamical systems. However, when training a policy, most algorithms only consider a single objective function. While this may suffice in virtual domains, physical systems must satisfy a set of operational constraints, with safety being of crucial importance. It is natural to express these problems as constrained optimization problems since weighted combinations of different rewards are not guaranteed to find a solution that satisfies all the requirements. Furthermore, these examples are not contrived, and safety-constrained reinforcement learning is a vital area of research that needs to be tackled.

After establishing the need to tackle safety-constrained reinforcement learning, I will shift my focus to solving these generally non-convex problems. I will discuss different approaches that exploit duality theory to pave the way towards algorithms for general constrained reinforcement learning. In particular, I will discuss that (i) despite their non-convexity these problems have zero duality gap, (ii) a state-augmented approach that does not guarantee convergence to an optimal policy but, it guarantees optimality and (iii) a safe policy-gradient theorem that allows us to consider constraints beyond time-averages.

This talk will take place in person at SCIoI.

Photo by Jeswin Thomas on Unsplash