Robustness in Mobile Robot Perception and Action

Despite huge advances in Spatial AI, regarding localisation, dense mapping and scene understanding fuelled by the advent of Deep Learning and powerful processors, robots still have a robustness problem: real-world applicability is limited to restricted tasks and restricted environments. Different paradigms have emerged as to how much the perception-action cycle of a mobile robot should remain somewhat hand-engineered and modular, or at the other extreme, end-to-end learned with rather black-box models, e.g. using Deep Reinforcement Learning from pixels to torques. In my talk, I will go through a couple of examples that sit in the middle. They leverage Deep Learning for sub-tasks in an otherwise modular and more classic approach. We explicitly estimate robot states in the form of e.g. position and orientation, as well as the environment, reconstructed to both geometrical accuracy, and decomposed into semantically meaningful entities, such as 3D objects that may even move. Importantly, the spatial representations need to be chosen for task-specific robust robotic interaction with the environment. In this context, I will be presenting some works on drone navigation and control, with an emphasis on accuracy, robustness, failure identification and mitigation – mostly in the application area of aerial inspection and manipulation.

Mitigating Unknown Unknowns: Challenges in developing and deploying fully autonomous aerial robots operating in extreme conditions at a global scale

This talk will highlight observations and insights acquired during the development, productization, and deployment of fully autonomous robots deployed as products at scale in extreme, diverse, and challenging operating conditions across the planet. The talk will span research conducted within the Resilient Intelligent System Lab (Robotics Institute, CMU) to productization and deployment of fully autonomous aerial robots that are actively employed in diverse and challenging conditions where systems regularly and necessarily operate in conditions that are not readily anticipated.

Real-time Motion Planning for the Masses

Traditional industrial automation is far from Power Up and Go. Setting up and maintaining robot work cells is a labour-intensive process that requires specialists to carefully orchestrate every motion. Robots should be smarter than that: If robots could perceive their surroundings and adapt their plans instantaneously, hours of integration time would be saved, and safe collaboration with humans would be possible. A fundamental requirement to enable these skills is the ability to plan motions almost latency-free. Realtime Robotics’ solution is a combination of precomputation and dedicated computing hardware that enables finding collision-free motions in milliseconds.

In this talk, I will share my experience (as a researcher) in getting hardware-accelerated real-time motion planning into factory floors, and the unexpected challenges and opportunities of industrial automation. I will discuss real world challenges for modelling, calibration, certification, and integration of perception and planning into robot workcells.

Towards Certifiably Robust Spatial Perception

Spatial perception is concerned with the estimation of a world model --that describes the state of the robot and the environment-- using sensor data and prior knowledge. As such, it includes a broad set of robotics and computer vision problems, ranging from object detection and pose estimation to robot localization and mapping. Most perception algorithms require extensive and application-dependent parameter tuning and often fail in off-nominal conditions (e.g., in the presence of large noise, outliers, and incorrect data association). In this talk, I present recent advances in the design of certifiably robust spatial perception algorithms that are robust to extreme amounts of outliers and afford performance guarantees. I show these algorithms can achieve unprecedented robustness in a variety of applications, ranging from mesh registration and image-based object localization, to SLAM.

Resilient and consistent robotic autonomy in unknown environments with extreme conditions

Consistency and robustness under extreme conditions are prerequisites to enable autonomous robotic operations in many application domains, ranging from space exploration, to search and rescue, to natural disaster response missions. Extreme conditions include mobility-stressing terrains, perceptually-degraded setting, and comm-denied environment, to name a few. In this presentation we will discuss some of the challenges and opportunities in addressing the problem of robotic autonomy under such extreme conditions.

We discuss the DARPA Subterranean Challenge as a representative mission that pushes the boundaries of robotic autonomy under extreme conditions. We go over TEAM-CoSTAR’s solution (called NeBula) that won the second phase of this challenge. We discuss NeBula’s algorithmic perspective on enabling robustness in robotic operations that aims at formulating and solving the problem in the “joint space” of traditional autonomy modules, including 1) traversability, 2) state estimation, 3) SLAM and semantic understanding, 4) task allocation, and 5) communication.

To Ignore Humans or to Accept them with Open Arms: Challenges and Opportunities for Efficient, Robust, and Adaptive POGO Robots

The field of robotics and autonomous driving has made a lot of advances over the past few decades, but the question of how we should treat humans (human designers, human operators, human users, or human observers) still remains: Should we assume humans don’t exist in the future of autonomy? Should we assume humans exist but are rational enough to stay away from our robots? Or should we accept them with open arms?

In the past, the general consensus has been to “ignore” humans and work on the “hardcore” robotics problems, i.e., full autonomy. However, avoiding humans in our formalism, software, or hardware design can lead to a number of inevitable roadblocks such as lack of robustness or safety guarantees when robots need to interact in non-stationary environments with humans present.

In this talk, we will discuss how we can start conceding that humans exist and that they are not necessarily fully rational. Specifically, we will go over some of the challenges and opportunities arising due to the presence of humans. Challenges such as planning for robots that interact with humans in high risk situations, and opportunities such as access to a diverse set of data that can be collected from interacting with humans. We end by discussing fast adaptation of robots in the presence of adapting human agents.

POGO robots in the wild: A historical perspective and future outlook

There are very few types of POGO robots in the wild. Why is this? As a civilization, we're pretty good at building POGO systems - and getting better - so what will it take to have more (assuming we want more) POGO robots in the wild? This talk will sketch a history of engineered POGO systems. We'll trace the evolution of how such systems came to be, and what they do (and don't do). We'll give some reasons why we believe building POGO robots is different than building other POGO systems. And finally, we'll make some predictions about the future of POGO robots.