Human-Robot Interaction (HRI) and Reinforcement Learning

Human-Robot Interaction. Co-robotics involves humans and robots working together safely in the same shared space as a team. This motivates physical Human-Robot Interaction (HRI) systems that adapt to different humans and have guaranteed robustness and stability properties. For modern interactive HRI systems to be capable of performing a wide range of tasks successfully, it is required to include the effects of both the robot dynamics and the human operator dynamics.

Human performance neuropsychological and human factors studies have shown that in coordinated motion with a robot, human learning has two components. The human operator learns a robot-specific inverse dynamics model to compensate for the nonlinearities of the robot, and simultaneously learns a feedback control component that is specific to the successful performance of the task. We incorporate these foundations in the design of HRI control systems by using a new Two Loop HRI Design - an Inner Design Loop to control the robot dynamics and an Outer Design Loop that includes task and human dynamics information. Three adaptive HRI outer loop designs are given that assist the human operator to perform a given task with minimum human workload demands and improved overall human-robot system performance. Experimental results on a PR2 robot demonstrate the effectiveness of this approach in using the robot to improve the human’s performance of a motion task.

Reinforcement Learning. In the second part of the course, techniques from reinforcement learning are used to design a new family of adaptive controllers based on actor-critic mechanisms that converge in real time to optimal control and game theoretic solutions. Using our technique known as Integral Reinforcement Learning (IRL), we will develop Optimal Control reinforcement learning methods for Continuous-Time Systems that do not require knowledge of the system drift dynamics. In the linear quadratic LQR case, the new RL adaptive control algorithms learn the solution to the Riccati equation by adaptation along the system motion trajectories. In the case of nonlinear systems with general performance measures, the algorithms learn the solutions of Hamilton-Jacobi or HJI equations. New IRL algorithms will be presented for solving online several classes of multi-player games. Finally these new IRL techniques are applied to HRI optimal task-loop design