You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description:A growing collection of your cool projects.
5
+
description:
6
6
nav: true
7
7
nav_order: 3
8
8
display_categories: [work, fun]
9
9
horizontal: true
10
10
---
11
11
12
-
13
12
We aim to build algorithms for perceptual representations learned by and for interaction, causal understanding of mechanisms, and physically-grounded reasoning in practical settings. An emblematic north star is to enable an autonomous robot to watch an instructional video, or a set of these videos, and then learn a policy to execute the task in a new setting. We build both algorithms and systems that have a broad range of applications in different domains in robot autonomy.
14
-
PAIR group blends ideas in *Causality*, *Perception*, and *Reinforcement Learning* towards this vision.
13
+
PAIR group blends ideas in _Causality_, _Perception_, and _Reinforcement Learning_ towards this vision.
15
14
16
-
As a group we pride ourselves on building and applying learning algorithms on different real robot platforms.
15
+
As a group we pride ourselves on building and applying learning algorithms on different real robot platforms.
{% include figure.liquid path="assets/img/res-planning-robot1-small.gif" class="img-fluid rounded z-depth-1" zoomable=true width="70%" %}
21
20
</div>
22
21
<div class="col-sm mt-3 mt-md-0">
23
-
{% include figure.liquid path="assets/img/res-cavin.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
22
+
{% include figure.liquid path="assets/img/res-cavin.gif" class="img-fluid rounded z-depth-1" zoomable=true width="70%" %}
24
23
</div>
25
24
<div class="col-sm mt-3 mt-md-0">
26
25
{% include figure.liquid path="assets/img/res-multimodal-test.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
@@ -32,10 +31,10 @@ As a group we pride ourselves on building and applying learning algorithms on di
32
31
33
32
<divclass="row mt-3">
34
33
<div class="col-sm mt-3 mt-md-0">
35
-
{% include figure.liquid path="assets/img/res-cutting.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
34
+
{% include figure.liquid path="assets/img/res-cutting.gif" class="img-fluid rounded z-depth-1" zoomable=true width="70%" %}
36
35
</div>
37
36
<div class="col-sm mt-3 mt-md-0">
38
-
{% include figure.liquid path="assets/img/res-suturing.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
37
+
{% include figure.liquid path="assets/img/res-suturing.gif" class="img-fluid rounded z-depth-1" zoomable=true width="70%" %}
39
38
</div>
40
39
<div class="col-sm mt-3 mt-md-0">
41
40
{% include figure.liquid path="assets/img/res-acubot.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
@@ -62,54 +61,52 @@ As a group we pride ourselves on building and applying learning algorithms on di
62
61
63
62
### 1. Generalizable Representations in RL for Robotics
64
63
65
-
A key focus of our work is to understand the role of representations in RL towards efficiency and generalization in skill acquisition. RL is mainly composed of State Space (Input), Action space (Output), a Learning Rule, and Policy (or value) model.
64
+
A key focus of our work is to understand the role of representations in RL towards efficiency and generalization in skill acquisition. RL is mainly composed of State Space (Input), Action space (Output), a Learning Rule, and Policy (or value) model.
66
65
66
+
Structured biases upend contemporary methods in all four dimensions, pointing to a need for deeper analysis of representations in RL.
67
67
68
-
Structured biases upend contemporary methods in all four dimensions, pointing to a need for deeper analysis of representations in RL.
69
-
- States: [Unsupervised Keypoints](http://arxiv.org/abs/2001.09518), [Making Sense of Touch and Vision](https://sites.google.com/view/visionandtouch)
70
-
- Objects Representations: [Task Oriented Grasping](https://sites.google.com/view/task-oriented-grasp), [Affordance for Tool-Use](https://sites.google.com/view/task-oriented-grasp)
- Architectures: [Deep-Dense nets in RL](https://sites.google.com/view/d2rl/home)
74
73
75
74
<divclass="row mt-3">
76
-
<div class="col-sm mt-3 mt-md-0">
75
+
<div class="col-sm-3 mt-3 mt-md-0">
77
76
{% include figure.liquid path="assets/img/res-tog.png" class="img-fluid rounded z-depth-1" zoomable=true %}
78
77
</div>
79
-
<div class="col-sm mt-3 mt-md-0">
78
+
<div class="col-sm-3 mt-3 mt-md-0">
80
79
{% include figure.liquid path="assets/img/res-laser-intro.jpg" class="img-fluid rounded z-depth-1" zoomable=true %}
81
80
</div>
82
-
<div class="col-sm mt-3 mt-md-0">
81
+
<div class="col-sm-5 mt-3 mt-md-0">
83
82
{% include figure.liquid path="assets/img/res-clearning.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
84
83
</div>
85
84
</div>
86
85
- 3D Vision: Object and Scene representations for manipulation.
87
86
- Perceptual Concept Learning
88
87
- Geomteric Deep Learning for discovery of symmetries
89
88
90
-
91
89
### 2. Causal Discovery and Inference in Robotics
92
90
93
-
Causal understanding is one of key pillars of my current and future agenda. A simulator is a generative world model, and similarly follows a system of structural mechanisms. However, model learning focuses solely on statistical dependence, while Causal Models go beyond it to build representations that support intervention, planning, and modular reasoning. These methods provide a concrete step towards bridging vision and robotics through sub-goal inference and counterfactual imagination.
91
+
Causal understanding is one of key pillars of my current and future agenda. A simulator is a generative world model, and similarly follows a system of structural mechanisms. However, model learning focuses solely on statistical dependence, while Causal Models go beyond it to build representations that support intervention, planning, and modular reasoning. These methods provide a concrete step towards bridging vision and robotics through sub-goal inference and counterfactual imagination.
94
92
95
93
<divclass="row mt-3">
96
-
<div class="col-sm mt-3 mt-md-0">
94
+
<div class="col-sm-9 mt-3 mt-md-0">
97
95
{% include figure.liquid path="assets/img/res-vcdn.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
98
96
</div>
99
-
<div class="col-sm mt-3 mt-md-0">
100
-
{% include figure.liquid path="assets/img/res-acgn.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
97
+
<div class="col-sm-3 mt-3 mt-md-0">
98
+
{% include figure.liquid path="assets/img/res-mac.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
### 3. Crowd-Scale Robot Learning with Imitation/Offline/Batch RL
110
107
111
-
Data-driven methods help RL in exploration and reward specification. Robot learning, however, is limited by modest-sized real data.
112
-
Access to data brings new algorithmic opportunities to robotics, as it did in vision and language. However, it also poses challenges due to static nature of data and covariate shifts.
108
+
Data-driven methods help RL in exploration and reward specification. Robot learning, however, is limited by modest-sized real data.
109
+
Access to data brings new algorithmic opportunities to robotics, as it did in vision and language. However, it also poses challenges due to static nature of data and covariate shifts.
113
110
114
111
<divclass="row mt-3">
115
112
<div class="col-sm mt-3 mt-md-0">
@@ -130,10 +127,9 @@ Access to data brings new algorithmic opportunities to robotics, as it did in vi
130
127
131
128
### 4. Structured Biases for Hierarchical Planning
132
129
133
-
Procedural reasoning, such as in robotics, needs both skills and their structured composition for interaction planning towards a higher-order objective.
130
+
Procedural reasoning, such as in robotics, needs both skills and their structured composition for interaction planning towards a higher-order objective.
134
131
However, manual composition of skills via a finite state-machine design is both tedious and unscalable. Thus the need for inductive bias is intensified for cognitive reasoning. I have developed imitation guided policy learning in abstract spaces for hierarchically structure tasks.
135
132
136
-
137
133
<divclass="row mt-3">
138
134
<div class="col-sm mt-3 mt-md-0">
139
135
{% include figure.liquid path="assets/img/res-ntp-small.gif" class="img-fluid rounded z-depth-1" zoomable=true %}
@@ -147,71 +143,67 @@ However, manual composition of skills via a finite state-machine design is both
147
143
148
144
### 5. Applications to Real Robot Systems
149
145
150
-
The algorithmic ideas have been motivated by problems in mobility and manipulation in robotics, and have been evaluated on various physical robot platforms.
146
+
The algorithmic ideas have been motivated by problems in mobility and manipulation in robotics, and have been evaluated on various physical robot platforms.
151
147
152
148
<divclass="row mt-3">
153
149
<div class="col-sm mt-3 mt-md-0">
154
-
{% include figure.liquid path="assets/img/res-robots-garg.png" class="img-fluid rounded z-depth-1" zoomable=true %}
150
+
{% include figure.liquid path="assets/img/real-robots-garg.png" class="img-fluid rounded z-depth-1" width="75%" zoomable=true %}
155
151
</div>
156
152
</div>
157
153
- Personal & Service Robotics: [Tool Use](https://sites.google.com/view/task-oriented-grasp), [Task Planning](https://www.youtube.com/watch?v=OdqJuvAHvGE), [Assembly](https://www.youtube.com/watch?v=NwMukXa8kys&feature=youtu.be), [Pick & place](https://ai.stanford.edu/mech-search/multistep), [Laundry Layout](https://roboturk.stanford.edu/realrobotdataset#tasks), [Assistive Teleoperation](http://arxiv.org/abs/1909.09674), [Mechanical Search](http://arxiv.org/abs/1903.01588)
0 commit comments