FieldAI Research Institute (FAIRI) is Leveraging the NVIDIA Omniverse™ Platform for Research in Embodied Q&A in Dynamic Environments

‍The Field AI Research Institute (FAIRI), in collaboration with NVIDIA, is conducting research on robotic Q&A in dynamic environments. This initiative focuses on evaluating how robots can reason and conduct embodied Q&A in evolving conditions. By leveraging the NVIDIA Omniverse™ platform, FAIRI is developing a dataset that serves as a foundation for studying robotic interactions in simulated settings.

‍

‍

This research studies a long-term embodied Q&A dataset for featuring dynamic environments and tasks, planned to be open-sourced. Field AI has utilized a suite of NVIDIA tools to create an embodied Q&A dataset for research purposes. FAIRI’s dataset is designed to explore interactions enabling new research into durative robotic interactions and missions.

‍

‍

Augmenting Long-Term Synthetic Data

‍

In collaboration with NVIDIA, FAIRI researchers are using NVIDIA Omniverse platform technologies, reference applications, and APIs, including NVIDIA Isaac Sim™,  NVIDIA Omniverse Replicator, and USD Search API to create physically accurate 3D environments. These scenarios simulate environmental changes over time, and will allow researchers to benchmark such EQ&A settings. 

‍

‍

‍

This comprehensive dataset focuses on example environments and EQ&A interactions in simulation, enabling evaluation of various models' performance and adaptability in such scenarios. Additionally, it supports research and testing efforts, such as developing a method that can efficiently organize longer-duration data and analyze information to answer questions (e.g., finding certain items or identifying missing ones).

‍

‍

These videos showcase Field AI’s FFM autonomy in action, automatically exploring and collecting datasets within digital twin environments.

‍

‍

‍

‍

Through this collaboration, FAIRI aims to advance the understanding of long-term robotic reasoning in dynamic environments, leveraging NVIDIA’s platform to push the boundaries of research in persistent, adaptive, and context-aware AI.

‍

‍