offroad optimization

Author: MsFriendly

_posts/2025-03-26-offroad.md

@@ -1,9 +1,9 @@
 ---
 layout: post
-title: "Off-road driving by learning from interaction and demonstration"
+title: "Autonomous Off-road driving | Learning-Based Navigation"
 date: 2025-03-26 10:17:07
 categories: research
-description: "We develop algorithms and tools to push robot autonomy in challenging environments"
+description: "CMU AirLab develops autonomous off-road driving systems that combine self-supervised learning, multi-modal perception, foundation models, and online adaptation of cost and dynamics. Our off-road autonomy stack enables resilient navigation in unstructured environments."
 author: "Wenshan Wang and Off-road Team"
 published: true
 sidebar: false
@@ -15,15 +15,15 @@ link-new-tab: true
 tags: Learning, Planning, Perception
 ---
 
-Off-road driving is an important robotics task with applications in agriculture, mining, exploration, and defense. While off-road driving has many similarities to driving in urban areas, a major difference is a lack of an obstacle/no obstacle dichotomy. That is, in off-road scenarios, not all objects are obstacles, and identifying which objects are traversable in a reliable way is critical.
+Autonomous off-road driving, a key challenge in robotics, has broad applications in agriculture, mining, search and rescue, exploration, and defense. While off-road has many similarities in urban areas, a major difference is a lack of an obstacle/no obstacle dichotomy. That is, in off-road scenarios, not all objects are obstacles, and identifying which objects are traversable in a reliable way is critical.
 
 {% youtube MBwNJh9Ghm4 %}
 
 Our research covers a wide range of topics that aim at expanding the robot’s capability as well as improving its robustness in challenging environments. We utilize modern machine learning techniques while eliminating the exhausting hand-labeling process. In specific, we explore self-supervised learning and learning-from-demonstration to understand the terrain traversability cost and vehicle dynamics from large-scale interaction data, online adapt the cost and dynamics model to overcome the out-of-distribution failures. Our system doesn’t require human labeled data, instead, it relies on its own experiences of interacting with the environment, while being aware of the uncertainty in each model, and online adapt the model in novel situations. We will explain our design philosophy in more detail and introduce the key components in the following sections. 
 
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/overview.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/overview.png" alt="Overview of CMU AirLab's autonomous off-road driving system" />
 </figure>
 
 ## Our Design Philosophy: Moving to a No-Hand Labeling Paradigm
@@ -36,7 +36,7 @@ However, there are some limitations to this approach that are unique to the off-
 - **Semantic segmentation labels are time-consuming**: It takes minutes to label a single image frame for semantic segmentation. Furthermore, the boundaries between semantic classes are not well-defined.
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/mud.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/mud.png" alt="Example of intra-class variance in muddy offroad terrain affecting autonomous driving traversability" />
 </figure>
 
 
@@ -47,19 +47,19 @@ In order to design methods that scale more effectively, **we have been developin
 ## Hardware System and Dataset
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/vehicle_hardware.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/vehicle_hardware.png" alt="Sensor payload on ATV for offroad autonomy, including LiDARs, cameras, and other sensors" />
 </figure>
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/dataset.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/dataset.png" alt="Sample images from CMU AirLab's off-road driving dataset showing diverse terrains" />
 </figure>
 
 Our ATV is equipped with a multi-modal sensor payload, which includes LiDARs, cameras, IMUs, shock travel sensors, wheel encoders and more. We have released two large-scale off-road driving datasets: [TartanDrive](https://github.com/castacks/tartan_drive) and [TartanDrive-V2](https://theairlab.org/TartanDrive2/).  
 
 To battle the extreme conditions such as smoke, fog and darkness, we added thermal cameras to the sensor payload. New data will be released. 
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/offroad_atv_thermal_sensors.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/offroad_atv_thermal_sensors.png" alt="Off-road ATV equipped with thermal cameras for robust perception in challenging conditions" />
 </figure>
 
 ## 3D Multi-modal Semantic Mapping for High-Speed Off-road Driving
@@ -69,7 +69,7 @@ Reliable and high-speed autonomous off-road driving has the potential to better
 We therefore develop a 3D multi-modal semantic mapping module capable of delivering long-range and fine grained traversability information for intelligent high-speed off-road driving. 
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/mapping.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/mapping.png" alt="Diagram of 3D multi-modal semantic mapping system for high-speed off-raod driving, showing input fusion from LiDAR and cameras to generate top-down traversability maps" />
 </figure>
 
 
@@ -90,7 +90,7 @@ For semantics, after an image is inferenced with a trained visual semantic model
 Additionally, we leverage features from visual foundation models (VFMs) such as DinoV2 to give us semantic-level features without requiring semantic annotations for given environments.
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/voxelmapping.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/voxelmapping.png" alt="3D voxel map generated from multi-modal sensor fusion for autonomous off-road navigation" />
 </figure>
 
 
@@ -105,7 +105,7 @@ What we are really interested in capturing is roughness in traversability as the
 In our [ICRA 2023 paper](https://arxiv.org/abs/2209.10788) “How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversability”, we propose a self-supervised method that predicts costmaps that reflect nuanced terrain interaction properties relevant to ground navigation. We approach this problem by learning a mapping from rich exteroceptive information and robot velocity to a continuous traversability cost derived from IMU data.
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/hdif.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/hdif.png" alt="Self-supervised learning pipeline for terrain costmap for offroad vehicle traversability" />
 </figure>
 
 We find that our method outperforms occupancy-based baselines on short-scale and large-scale navigation trials. Our short-scale navigation results show that using our learned costmaps leads to overall smoother navigation, and provides the robot with a more fine-grained understanding of the interactions between the robot and different terrain types, such as grass and gravel. Our large-scale navigation trials show that we can reduce the number of interventions by up to 57% compared to an occupancy-based navigation baseline in challenging off-road courses ranging from 400 m to 3150 m. 
@@ -119,7 +119,7 @@ Another source of unlabeled supervision for learning navigation behaviors are ex
 In order to translate human-driven trajectories into a form consumable by planning and control, we use inverse reinforcement learning (IRL) to learn costmaps from lidar data. Compared to the alternative of learning actions directly from expert data, learning costmaps has the advantage of being human-interpretable.
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/irl.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/irl.png" alt="Diagram of Inverse reinforcement learning to learn costmaps for offroad driving" />
 </figure>
 
 In order to achieve a practical algorithm that runs effectively on our platform, we leverage maximum entropy IRL (MaxEnt IRL) with several extensions, such as sampling-based MPC, risk estimation, and deep neural networks. We find that IRL significantly outperforms occupancy-based baselines on several kilometers of challenging off-road trails (reducing interventions by up to 70%). Furthermore, we find that we can leverage our risk-estimation to modulate how aggressive the ATV is with respect to terrains such as tall grass and slopes. Results are presented in our [ICRA 2023 paper](https://arxiv.org/abs/2302.00134) “Learning Risk-Aware Costmaps via Inverse Reinforcement Learning for Off-Road Navigation''. 
@@ -134,7 +134,7 @@ In addition to using costs as a means of determining where to drive, we also lea
 We find that IRL significantly outperforms occupancy-based baselines on several kilometers of challenging off-road trails (reducing interventions by up to 70%). Furthermore, we find that we can leverage our risk-estimation to modulate how aggressive the ATV is with respect to terrains such as tall grass and slopes. Results can be found in our [CoRL 2024 paper](https://openreview.net/forum?id=AhEE5wrcLU) “Velociraptor: Leveraging Visual Foundation Models for Label-Free, Risk-Aware Off-Road Navigation”.
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/offroad_irl.png" alt="" />
+ <img src="/img/posts/2023-08-08-offroad/offroad_irl.png" alt="Input and prediction visualization of risk-aware off-road navigation using visual foundation models and IRL" />
 </figure>
 
 
@@ -145,7 +145,7 @@ In order to create control rules that guarantee desirable qualities like safety,
 The Kinematic Bicycle Model (KBM), one of the examples of a model developed from first principles (purely physics-driven), is widely used in practice but tends to oversimplify the underlying structure of dynamical systems, resulting in prediction errors that cannot be corrected by optimizing over a few model parameters. 
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/dynamicsmodel.png" alt="" style="width:80%" />
+ <img src="/img/posts/2023-08-08-offroad/dynamicsmodel.png" alt="Diagram of deep learning model for predicting off-road vehicle dynamics" style="width:80%" />
 </figure>
 
 We therefore employ deep learning to address this, which offers very expressive models for function approximation. We leverage real-time information from odometry along with multiple other modalities like, First-Person View Image, Heightmaps and RGB-maps.  By reducing the RMSE loss in the anticipated state - [x, y, yaw, velocity] (See Table 1. For deeper analysis), these models outperform purely physics-driven models. Our [ICRA 2022 work](https://arxiv.org/abs/2205.01791), "TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Models" provides a thorough description of the model. 
@@ -173,11 +173,11 @@ We adopt a philosophy that learned methods for off-road driving should be both s
 
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/salon.png" alt="" style="width:100%" />
+ <img src="/img/posts/2023-08-08-offroad/salon.png" alt="Online adaptation framework for autonomous offroad navigation using minimal human inputs" style="width:100%" />
 </figure>
 
 <figure>
- <img src="/img/posts/2023-08-08-offroad/salon_diagram.png" alt="" style="width:100%" />
+ <img src="/img/posts/2023-08-08-offroad/salon_diagram.png" alt="Diagram of off-road self-supervised system combining visual features, point clouds, and proprioception." style="width:100%" />
 </figure>
 
 ## Online Adaptation for Off-road Long-Range Perception

_team/yifei.md

@@ -8,14 +8,14 @@ image: /img/team/yifeil.png
 link-new-tab: true
 ---
 
-Yifei is a Master’s student in Robotics at Carnegie Mellon University. She graduated from University of California, Irvine with a B.S. in computer science.  
+Yifei is a Master’s student in Robotics at Carnegie Mellon University. She graduated from University of California, Irvine with a B.S. in Computer Science.  
 
-Yifei's currently developing robust 3D perception modules to enhance robot navigation in challenging real-world environments. Her research interests lie at the intersection of computer vision, deep learning, and robotics. 
-
-In her free time, she loves skiing, horseback riding, photography, and learning new languages. 
+Yifei is currently developing robust 3D perception modules for UAV and UGV navigation in challenging real-world environments, including off-road, nighttime, and visually-degraded conditions. Her research interests lie at the intersection of computer vision, deep learning, and robotics, with a particular emphasis on thermal sensing and multi-modal perception.
 
 <br>
 
 **Email**: [yifeil5@andrew.cmu.edu](mailto:yifeil5@andrew.cmu.edu)
 
 **LinkedIn**: [yifei-migo-liu](https://www.linkedin.com/in/yifei-migo-liu/)
+
+**Website**: [migoyliu.github.io](https://migoyliu.github.io/)