Embodied AI
Embodied AI is at the forefront of building an ever-improving foundation of automated industrial Artificial Intelligence though Natural Interaction in Physical Space and with Language.
We target the creating of AI that can be operated easily by general users and continuously pushes the boundary of task complexity. We facilitate easy deployment of equipment to new sites and tasks by learning and using problem/task/environment-specific knowledge and skills.
Fast Adaptation allows for minimal effort/cost to deploy systems in new environments through interaction with human and the environment.
Continuous Learning expands functionality by generalizing past experiences into “common sense” through knowledge extraction.
Embodied AI refers to agent-based AI systems that can manipulate objects and communicate with people, assisting us in physical tasks. These agents learn interactively from users in their environment, adapting quickly, and continuously expanding their capabilities. CRL’s research will strive to yield technologies capable of natural interaction with the physical environment and human operators, building a dynamic catalog of skills that encompasses perception, reasoning, and action.
The classical paradigm of perception and manipulation systems is to understand to act – e.g. segmentation is used to detect pedestrians so that ADAS can operate. In the new framework of Embodied AI, we instead act to understand. Specifically, Embodied AI has fast adaptation and continuous learning through task execution at its core. The system interacts to improve perception, reasoning, and its action towards future task completion. Human interaction will enable seamless reprogramming (teaching) of physical systems, while autonomous actions in physical space enable us to learn about new environments. The need for expert knowledge to redeploy classical AI to new tasks is eliminated.
Embodied AI is critical to addressing the real-world challenges of the next generation industry. Adaptation and resilience is at the core of embodied AI as it ensuring a workforce that can truly leverage the power of AI through simple human and machine interaction in ever changing tasks and deployments. Adaptation and learning also ensures sustainability as we constantly improve the applicability of existing hardware through ever learning software algorithms. CRL’s Embodied AI will enable a versatile assistive system for a multitude of tasks. Since Embodied AI focuses on the software, we can learn from many modalities and deployments ensuring a seamless integration of disparate systems.
Language & Interaction Group
The Langauge & Interaction Group (LIG) focuses on learning from interaction with natural language. Dialogue driven deployment-specific interaction with human operators allow for long-horizon planning, refinement and learning with human in the loop. Fast adaptation utilizes local knowledge bases to refine the execution.
Vision & Learning Group
The Vision & Learning Group (VLG) focuses on learning from interaction in physical environments. Complex and safe manipulation and navigation technology leverage precise 3D geometry and scene understanding in conjunction with strong world-aware action selection frameworks. Learned concepts are effectively transfered to new domains.
Latest Publications
Information contained in news and other announcements is current on the date of posting, but subject to change without notice.
F Logothetis, I Budvytis, R Cipolla
S Morad, C Lu, R Kortvelesy, S Liwicki, J Foerster, A Prorok
Norbert Braunschweiler, Rama Doddipatla, Tudor-Catalin Zorila
Youmna Farag, Svetlana Stoyanchev, Mohan Li, Simon Keizer, Rama Doddipatla
Cong Thanh Do, Yuan Li, Simon Keizer, Mohan Li, Rama Doddipatla, Kate Knill
Cong Thanh Do, Shuhei Imai, Rama Doddipatla, Thomas Hain
A. Frummet, A. Papenmeier, M. Fröbe, J. Kiesel, V. Adlakha, N. BraunschweilerT, M.Dubiel, S. Ghosh, M. Gohsen, C. Kreutz, M. Momeni, M. Nilles, S. P. Cherumanal, A.Pirmoradi, P. Thomas, J. R. Trippas, I. Zelch, and O. Zendel.
Abigail Sticha, Norbert Braunschweiler, Rama Sanand Doddipatla, Kate M Knill
Improving Retrieval-Augmented Response Generation in Goal-Oriented Dialogue Question Answering
N. BraunschweilerT, A. Sticha, R. Doddipatla, K. Knill
F Logothetis, I Budvytis, R Cipolla
Lexin Zhou, Youmna Farag, Andreas Vlachos.
L. Domingo, P. Piwek, M. Wermelinger, S. Stoyanchev
M. Li, S. Keizer and R. Doddipatla
C. Li, C. Zhang , S. Teufel, R. Doddipatla , S.Stoyanchev
T. Cord-Landwehr, C. Boeddeker, T-C Zoril?a, R. Doddipatla and R. Haeb-Umbach