Tutorials

The first part of the tutorial will focus on two recent topics in Markov decision processes (MDPs). We will first discuss the construction of data-driven MDPs that combine the tasks of estimating the system’s behavior and selecting a policy that performs well out of sample. We will then discuss the exploitation of problem structure to scale to large problem sizes via weakly coupled MDPs that combine a potentially large number of MDPs via a small number of linking constraints. The second part of the tutorial will provide an introduction to distributionally robust optimization with Wasserstein ambiguity sets. In particular, we will address robust Linear-Quadratic-Gaussian (LQG) control problems, where the noise distributions are unknown and belong to Wasserstein ambiguity sets centered at nominal Gaussian distributions. We will derive structural properties of the optimal primal and dual solutions and develop an efficient Frank-Wolfe algorithm to solve robust LQG problems.

Autonomous systems impose numerous challenging requirements on the underlying control algorithms. In particular, while ensuring safety is often critical in the design, the complexity and uncertainty of the dynamics and the multifaceted nature of the control objective make it hard to achieve. Classical control methods are often unsuited to apply in such unstructured tasks with potentially conflicting safety specifications. While impressive progress has been achieved through learning-based policies, missing safety certificates often still prohibit the widespread application of these techniques outside of research environments. This tutorial presents safety filter techniques, providing general and modular approaches to augment any control policy with safety guarantees in the form of constraint satisfaction. The first part provides a comprehensive overview of three main classes of safety filters and introduces invariance-based methods relying on Hamilton-Jacobi reachability, control barrier functions, and predictive control techniques. The second part then focuses on extensions towards improved practical applicability including data-driven enhancements and reduced-order models. The tutorial aims at providing a comprehensive introduction for researchers of different backgrounds. It provides the main concepts, theoretical guarantees and addresses practical aspects, accompanied by application examples.