Research
Planning and Reasoning Under Uncertainty
A real robot never has perfect sensors or actuators. Instead, an intelligent robot needs to be able to solve the tasks assigned to it while handling uncertainty about the environment as well as about the effects of its own actions. This is a challenging computational problem, but also one that humans solve on a routine basis (we don't have perfect sensors or actuators either!).
We are developing new methods for efficiently expressing and solving problems where the agent has limited, incomplete information about the quantities and identities of the objects that it may encounter.
Papers:
Autonomous Agents That Are Easy to Understand and Safe to Work With
How would a non-expert assess what their AI system can or can’t do safely? Today’s AI systems require experts to evaluate them, which limits the deployability and safe usability of AI systems.
We are developing approaches for autonomous, user-driven assessment of the capabilities of black-box taskable AI systems, even as the AI systems learn and adapt. These methods would enable users to continually evaluate and understand their AI systems in their own idiosyncratic deployments. They would prevent performance failures and accidents that can arise when AI systems are used beyond their dynamic envelopes of safe applicability. We are also designing approaches for computing user-aligned explanations of AI behavior. Together, these approaches improve the safety and usability of AI systems and enable autonomous, on-the-fly training paradigms for AI systems.
Papers:
Synthesis and Analysis of Abstractions for Autonomy
In order to solve complex, long-horizon tasks such as doing the laundry, a robot needs to compute high-level strategies (e.g., would it be useful to put all the dirty clothes in a basket first?) as well as the joint movements that it should execute. Unfortunately, approaches for high-level planning rely on task-planning abstractions that are lossy and can produce “solutions” that have no feasible executions.
We are developing new methods for computing safe task-planning abstractions and for dynamically refining the task-planning abstraction to produce combined task and motion plans that are guaranteed to be executable. We are also working on utilizing abstractions in sequential decision making (SDM) for evaluating the effect of abstractions on models for SDM, as well as to search for abstractions that would aid in solving a given SDM problem.
Papers:
Contextual Abstractions for Generalization in RL
How can reinforcement learning agents scale to solve complex, real-world problems with limited interaction and delayed feedback? Despite significant progress, existing methods continue to face critical challenges in practical applications—most notably, sample inefficiency, poor scalability, particularly when addressing long-horizon tasks with sparse rewards and non-stationary, unknown dynamics. Abstraction has emerged as a powerful mechanism for scaling RL, yet autonomously learning abstract representations-of states and actions-remains a difficult, unsolved problem, hindering the generalization and adaptability of RL agents.
Our research tackles this challenge through novel algorithms based on top-down abstraction refinement. These methods dynamically invent, represent, and utilize abstractions during the learning process. By leveraging these abstractions for planning and transferring them across related tasks, our methods support efficient decision-making in continual learning scenarios.
Papers:
