Please note: This seminar will be given online.
Jacopo Panerati, Postdoctoral Fellow
Institute for Aerospace Studies, University of Toronto
The recent advances of deep reinforcement learning (RL) have brought renewed interest for data- and learning-based methods for robot control. Having mastered video games, board games, and protein folding, the promise of RL and learning-based control is to automate and speed up the design of all robotic applications—with the potential to revolutionize transportation and logistics, manufacturing and healthcare. RL research has directed much of its focus towards open challenges such as the exploration-exploitation trade-off and algorithms’ divergence resulting from the combination—required by most real-world scenarios—of bootstrapping, off-policy evaluation, and function approximation.
However, robotic RL research also remains plagued by limited reproducibility and often overlooks other crucial aspects of robotic applications—e.g., unrecoverable states, the need for safety guarantees, violation of ergodicity assumptions, a priori knowledge, and slow/expensive data collection. This talk points at three important bottlenecks for robotic RL, i.e., the (i) fast, (ii) safe, and (iii) reproducible collection of data for the training and evaluation of learning-based robot controllers.
An important stepping stone—leading to RL deployable in the real world—we argue, is the development of software tools to properly gauge the progress of RL in safe robot control—and compare it to the work of the control theory community—by leveraging fast physics-based simulation. In our vision, physics-based simulation will dramatically reduce the design cost of robot controllers and play a role analogous to the datasets (e.g., KITTI, MNIST, Common Crawl) that enabled comparisons and greatly accelerated the progress of other data-based methods (in computer vision, NLP).
Bio: Dr. Jacopo Panerati is a postdoctoral fellow at the University of Toronto Institute for Aerospace Studies (UTIAS) working under the supervision of Prof. Angela P. Schoellig in the Dynamic Systems Lab. Jacopo’s research interests include reinforcement learning for robotics, distributed and multi-robot systems, probabilistic graphical models, computer architectures, and software engineering.
Jacopo holds a Ph.D. degree in computer engineering from Polytechnique Montreal and received the M.Sc. degree in computer science from the University of Illinois at Chicago in 2012, the Laurea Triennale degree in computer engineering from Politecnico di Milano in 2009, and the Laurea Specialistica degree in computer engineering again from Politecnico di Milano in 2011.
In 2015, Jacopo was a visiting researcher at the National Institute of Informatics (Tokyo, Japan) and attended the International Space University (ISU)’s Space Studies Program in Athens, Ohio. In 2017, Jacopo also served as a teaching associate for ISU in Cork, Ireland. In 2019, Jacopo was a visiting postdoctoral fellow at the European Astronaut Centre (Cologne, Germany) and worked as a research associate in the University of Cambridge’s Department of Computer Science and Technology.