Perception and Navigation

Multimodal Perception & Open-World Scene Understanding

Robust robotic perception relies on fusing information across multiple complementary modalities, including vision (RGB-D), LiDAR, and inertial measurements (IMUs). Moving beyond closed-set classifications and purely geometric representations, our research focuses on open-world scene understanding in unstructured environments. We investigate anomaly segmentation and novel class discovery to enable robots to identify unmodeled objects, flag unexpected hazards, and categorize unseen elements dynamically. By combining deep learning architectures with open-vocabulary vision-language models, we aim to bridge raw sensory streams with rich, queryable spatial semantics.

Open-Vocabulary Semantic Mapping

To make intelligent decisions, autonomous agents require spatial representations that go beyond physical occupancy to capture what objects and regions actually represent. Our research explores open-vocabulary scene representations, such as object-centric maps and 3D scene graphs, that embed rich semantic concepts and spatial relationships. For walking robots operating in feature-scarce, cluttered, or changing environments, these semantics-rich maps serve as critical priors to help resolve geometric ambiguities, track scene dynamics, and support downstream reasoning tasks.

Semantic Path Planning & Long-Horizon Autonomy

Traditional mobile navigation frameworks depend primarily on geometric traversability and clearance. We advance navigation research by embedding semantic context directly into global and local motion planning algorithms. Semantic path planning allows robots to make context-aware navigation decisions such as prioritizing specific functional areas, respecting domain-specific constraints, or executing object-goal navigation. By linking low-level locomotion with high-level semantic reasoning, our goal is to support long-horizon autonomy across complex indoor settings and dynamic outdoor environments.

Event-Based Vision for Dynamic Robotic Perception

Event-based vision offers a promising approach to enhancing robotic perception in dynamic and unstructured environments. Unlike conventional cameras, event cameras capture changes in brightness asynchronously, providing high temporal resolution, low latency, and high dynamic range. By integrating event-based vision into quadruped robots, we aim to improve motion estimation, object tracking, and environment understanding, supporting robust navigation, human-robot interaction, and autonomous operation in challenging real-world scenarios.