How can we build autonomous robots that operate in unstructured and dynamic environments such as homes or hospitals? This problem has been investigated under several disciplines, including planning (motion planning, task planning, etc.), and reinforcement learning. While both of these fields have witnessed tremendous progress, each have fundamental drawbacks when it comes to autonomous robots. In general, planning approaches require substantial manual engineering in specifying a model for the domain, while RL is data hungry and does not generalize beyond the tasks seen during training. In this talk, we present several studies that aim to mitigate these shortcomings by combining ideas from both planning and learning. We start by introducing value iteration networks, a type of differentiable planner that can be used within model-free RL to obtain better generalization. Next, we consider a practical robotic assembly problem, and show that motion planning, based on readily available CAD data, can be combined with RL to quickly learn policies for assembling tight fitting objects. Then, we show how deep learning can be used to improve classical planning, by learning powerful image-based heuristic functions for A* search. We conclude with our recent work on generating goal-directed visual plans. Motivated by humans’ remarkable capability to predict and plan complex manipulations of objects, we develop a data-driven method that learns to ‘imagine’ a plausible sequence of observations that transition a dynamical system from its current configuration to a desired goal state. Key to our method is Causal InfoGAN, a deep generative model that can learn features that are compatible with strong planning algorithms. We demonstrate our approach on imagining plausible visual plans of rope manipulation.

Aviv Tamar is a postdoc at UC Berkeley’s Artificial Intelligence Research lab, and will soon start as assistant professor in the Electrical Engineering department at Technion, Israel Institute for Technology. Aviv has received his PhD and MSc in 2015 and 2011, both from Technion. His research intersects reinforcement learning, representation learning, and robotics.