Skip to content

Commit 30bcb2e

Browse files
authored
Demo/visualize fixes and docs changes (#273)
* Visualizer and control mode, init mode config description updates in docs * Fix Wrong names in control modes * add configs to drive.c, init_only_controlled and demo bug fixes, fix tables in docs * minor fix * Remove control_tracks_to_predict in another place
1 parent 5bd7fd2 commit 30bcb2e

6 files changed

Lines changed: 223 additions & 34 deletions

File tree

docs/src/interact-with-agents.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,30 @@ then launch:
1818

1919
This will run `demo()` with an existing model checkpoint.
2020

21+
## Arguments & Configuration
22+
23+
The `drive` tool supports similar CLI arguments as the visualizer to control the environment and rendering. It also reads the `pufferlib/config/ocean/drive.ini` file for default environment settings.
24+
25+
### Command Line Arguments
26+
27+
| Argument | Description | Default |
28+
| :--- | :--- | :--- |
29+
| `--map-name <path>` | Path to the map binary file (e.g., `resources/drive/binaries/training/map_000.bin`). If omitted, picks a random map out of `num_maps` from `map_dir` in `drive.ini`. | Random |
30+
| `--policy-name <path>` | `Path to the policy weights file (.bin).` | `resources/drive/puffer_drive_weights.bin` |
31+
| `--view <mode>` | Selects which views to render: `agent`, `topdown`, or `both`. | `both` |
32+
| `--frame-skip <n>` | Renders every Nth frame to speed up simulation (framerate remains 30fps). | `1` |
33+
| `--num-maps <n>` | Overrides the number of maps to sample from if `--map-name` is not set. | `drive.ini` value |
34+
35+
### Visualization Flags
36+
37+
| Flag | Description |
38+
| :--- | :--- |
39+
| `--show-grid` | Draws the underlying nav-graph/grid on the map. |
40+
| `--obs-only` | Hides objects not currently visible to the agent's sensors (fog of war). |
41+
| `--lasers` | Visualizes the raycast sensor lines from the agent. |
42+
| `--log-trajectories` | Draws the ground-truth "human" expert trajectories as green lines. |
43+
| `--zoom-in` | Zooms the camera mainly on the active region rather than the full map bounds. |
44+
2145
### Controls
2246

2347
**General:**

docs/src/simulator.md

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,28 @@ A high-performance autonomous driving simulator in C with Python bindings.
2222

2323
- `control_vehicles`: Only vehicles
2424
- `control_agents`: All agent types (vehicles, cyclists, pedestrians)
25-
- `control_tracks_to_predict`: WOMD evaluation mode
25+
- `control_wosac`: WOSAC evaluation mode (controls all valid agents ignoring expert flag and start to goal distance)
2626
- `control_sdc_only`: Self-driving car only
2727

2828
> [!NOTE]
29-
> `control_vehicles` filters out agents marked as "expert" and those too close to their goal (<2m). For full WOMD evaluation, use `control_tracks_to_predict`.
29+
> `control_vehicles` filters out agents marked as "expert" and those too close to their goal (<2m). For full WOMD evaluation, use `control_wosac`.
30+
31+
> [!IMPORTANT]
32+
> **Agent Dynamics:** The simulator supports three types of agents:
33+
> 1. **Policy-Controlled:** Stepped by your model's actions.
34+
> 2. **Experts:** Stepped using ground-truth log trajectories.
35+
> 3. **Static:** Remain frozen in place.
36+
>
37+
> In the simulator, agents not selected for policy control will be treated as **Static** by default. To make them follow their **Expert trajectories**, you must set `mark_as_expert=true` for those agents in the jsons. This is critical for `control_sdc_only` to ensure the environment behaves realistically around the policy-controlled agents.
38+
39+
### Init modes
40+
41+
- **`create_all_valid`** (Default): Initializes every valid agent present in the map file. This includes policy-controlled agents, experts (if marked), and static agents.
42+
43+
- **`create_only_controlled`**: Initializes **only** the agents that are directly controlled by the policy.
44+
45+
> [!NOTE]
46+
> In `create_only_controlled` mode, the environment will contain **no static or expert agents**. Only the policy-controlled agents will exist.
3047
3148
### Goal behaviors
3249

docs/src/visualizer.md

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,11 +25,45 @@ bash scripts/build_ocean.sh visualize local
2525

2626
If you need to force a rebuild, remove the cached binary first (`rm ./visualize`).
2727

28-
## Run headless
29-
Launch the visualizer with a virtual display and export an `.mp4`:
28+
## Rendering a Video
29+
Launch the visualizer with a virtual display and export an `.mp4` for the binary scenario:
3030

3131
```bash
3232
xvfb-run -s "-screen 0 1280x720x24" ./visualize
3333
```
3434

3535
Adjust the screen size and color depth as needed. The `xvfb-run` wrapper allows Raylib to render without an attached display, which is convenient for servers and CI jobs.
36+
37+
## Arguments & Configuration
38+
39+
The `visualize` tool supports several CLI arguments to control the rendering output. It also reads the `pufferlib/config/ocean/drive.ini` file for default environment settings(For more details on these settings, refer to [Configuration](simulator.md#configuration)).
40+
41+
### Command Line Arguments
42+
43+
| Argument | Description | Default |
44+
| :--- | :--- | :--- |
45+
| `--map-name <path>` | Path to the map binary file (e.g., `resources/drive/binaries/training/map_000.bin`). If omitted, picks a random map out of `num_maps` from `map_dir` in `drive.ini`. | Random |
46+
| `--policy-name <path>` | Path to the policy weights file (`.bin`). | `resources/drive/puffer_drive_weights.bin` |
47+
| `--view <mode>` | Selects which views to render: `agent`, `topdown`, or `both`. | `both` |
48+
| `--output-agent <path>` | Output filename for agent view video. | `<policy>_agent.mp4` |
49+
| `--output-topdown <path>` | Output filename for top-down view video. | `<policy>_topdown.mp4` |
50+
| `--frame-skip <n>` | Renders every Nth frame to speed up generation (framerate remains 30fps). | `1` |
51+
| `--num-maps <n>` | Overrides the number of maps to sample from if `--map-name` is not set. | `drive.ini` value |
52+
53+
### Visualization Flags
54+
55+
| Flag | Description |
56+
| :--- | :--- |
57+
| `--show-grid` | Draws the underlying nav-graph/grid on the map. |
58+
| `--obs-only` | Hides objects not currently visible to the agent's sensors (fog of war). |
59+
| `--lasers` | Visualizes the raycast sensor lines from the agent. |
60+
| `--log-trajectories` | Draws the ground-truth "human" expert trajectories as green lines. |
61+
| `--zoom-in` | Zooms the camera mainly on the active region rather than the full map bounds. |
62+
63+
### Key `drive.ini` Settings
64+
The visualizer initializes the environment using `pufferlib/config/ocean/drive.ini`. Important settings include:
65+
66+
- `[env] dynamics_model`: `classic` or `jerk`. Must match the trained policy.
67+
- `[env] episode_length`: Duration of the playback. defaults to 91 if set to 0.
68+
- `[env] control_mode`: Determines which agents are active (`control_vehicles` vs `control_sdc_only`).
69+
- `[env] goal_behavior`: Defines agent behavior upon reaching goals (respawn vs stop).

docs/theme/extra.css

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -463,3 +463,8 @@ blockquote {
463463
margin: 1rem 0;
464464
border-radius: 0 8px 8px 0;
465465
}
466+
467+
/* Fix table visibility - remove alternating row colors */
468+
table tr:nth-child(2n) {
469+
background-color: transparent !important;
470+
}

pufferlib/ocean/drive/drive.c

Lines changed: 136 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
11
#include "drivenet.h"
2+
#include "error.h"
3+
#include "libgen.h"
4+
#include "../env_config.h"
25
#include <string.h>
36

47
// Use this test if the network changes to ensure that the forward pass
@@ -31,37 +34,58 @@ void test_drivenet() {
3134
free(weights);
3235
}
3336

34-
void demo() {
37+
int demo(const char *map_name, const char *policy_name, int show_grid, int obs_only, int lasers, int show_human_logs,
38+
int frame_skip, const char *view_mode, const char *output_topdown, const char *output_agent, int num_maps,
39+
int zoom_in) {
3540

36-
// Note: The settings below are hardcoded for demo purposes. Since the policy was
37-
// trained with these exact settings, that changing them may lead to
38-
// weird behavior.
41+
// Parse configuration from INI file
42+
env_init_config conf = {0};
43+
const char *ini_file = "pufferlib/config/ocean/drive.ini";
44+
if (ini_parse(ini_file, handler, &conf) < 0) {
45+
fprintf(stderr, "Error: Could not load %s. Cannot determine environment configuration.\n", ini_file);
46+
return -1;
47+
}
48+
49+
char map_buffer[100];
50+
if (map_name == NULL) {
51+
srand(time(NULL));
52+
int random_map = rand() % num_maps;
53+
sprintf(map_buffer, "%s/map_%03d.bin", conf.map_dir, random_map);
54+
map_name = map_buffer;
55+
}
56+
57+
// Initialize environment with all config values from INI [env] section
3958
Drive env = {
40-
.human_agent_idx = 0,
41-
.action_type = 0, // Discrete
42-
.dynamics_model = CLASSIC, // Classic dynamics
43-
.reward_vehicle_collision = -1.0f,
44-
.reward_offroad_collision = -1.0f,
45-
.reward_goal = 1.0f,
46-
.reward_goal_post_respawn = 0.25f,
47-
.goal_radius = 2.0f,
48-
.goal_behavior = 1,
49-
.goal_target_distance = 30.0f,
50-
.goal_speed = 10.0f,
51-
.dt = 0.1f,
52-
.episode_length = 300,
53-
.termination_mode = 0,
54-
.collision_behavior = 0,
55-
.offroad_behavior = 0,
56-
.init_steps = 0,
57-
.init_mode = 0,
58-
.control_mode = 0,
59-
.map_name = "resources/drive/map_town_02_carla.bin",
59+
.action_type = conf.action_type,
60+
.dynamics_model = conf.dynamics_model,
61+
.reward_vehicle_collision = conf.reward_vehicle_collision,
62+
.reward_offroad_collision = conf.reward_offroad_collision,
63+
.reward_goal = conf.reward_goal,
64+
.reward_goal_post_respawn = conf.reward_goal_post_respawn,
65+
.goal_radius = conf.goal_radius,
66+
.goal_behavior = conf.goal_behavior,
67+
.goal_target_distance = conf.goal_target_distance,
68+
.goal_speed = conf.goal_speed,
69+
.dt = conf.dt,
70+
.episode_length = conf.episode_length,
71+
.termination_mode = conf.termination_mode,
72+
.collision_behavior = conf.collision_behavior,
73+
.offroad_behavior = conf.offroad_behavior,
74+
.init_steps = conf.init_steps,
75+
.init_mode = conf.init_mode,
76+
.control_mode = conf.control_mode,
77+
.map_name = (char *)map_name,
6078
};
6179
allocate(&env);
80+
if (env.active_agent_count == 0) {
81+
fprintf(stderr, "Error: No active agents found in map '%s' with init_mode=%d. Cannot run demo.\n", env.map_name,
82+
conf.init_mode);
83+
free_allocated(&env);
84+
return -1;
85+
}
6286
c_reset(&env);
6387
c_render(&env);
64-
Weights *weights = load_weights("resources/drive/puffer_drive_weights_carla_town12.bin");
88+
Weights *weights = load_weights((char *)policy_name);
6589
DriveNet *net = init_drivenet(weights, env.active_agent_count, env.dynamics_model);
6690

6791
int accel_delta = 2;
@@ -134,6 +158,7 @@ void demo() {
134158
free_allocated(&env);
135159
free_drivenet(net);
136160
free(weights);
161+
return 0;
137162
}
138163

139164
void performance_test() {
@@ -177,9 +202,93 @@ void performance_test() {
177202
free_allocated(&env);
178203
}
179204

180-
int main() {
205+
int main(int argc, char *argv[]) {
206+
// Visualization-only parameters (not in [env] section)
207+
int show_grid = 0;
208+
int obs_only = 0;
209+
int lasers = 0;
210+
int show_human_logs = 0;
211+
int frame_skip = 1;
212+
int zoom_in = 0;
213+
const char *view_mode = "both";
214+
215+
// File paths and num_maps (not in [env] section)
216+
const char *map_name = NULL;
217+
const char *policy_name = "resources/drive/puffer_drive_weights.bin";
218+
const char *output_topdown = NULL;
219+
const char *output_agent = NULL;
220+
int num_maps = 1;
221+
222+
// Parse command line arguments
223+
for (int i = 1; i < argc; i++) {
224+
if (strcmp(argv[i], "--show-grid") == 0) {
225+
show_grid = 1;
226+
} else if (strcmp(argv[i], "--obs-only") == 0) {
227+
obs_only = 1;
228+
} else if (strcmp(argv[i], "--lasers") == 0) {
229+
lasers = 1;
230+
} else if (strcmp(argv[i], "--log-trajectories") == 0) {
231+
show_human_logs = 1;
232+
} else if (strcmp(argv[i], "--frame-skip") == 0) {
233+
if (i + 1 < argc) {
234+
frame_skip = atoi(argv[i + 1]);
235+
i++;
236+
if (frame_skip <= 0) {
237+
frame_skip = 1;
238+
}
239+
}
240+
} else if (strcmp(argv[i], "--zoom-in") == 0) {
241+
zoom_in = 1;
242+
} else if (strcmp(argv[i], "--view") == 0) {
243+
if (i + 1 < argc) {
244+
view_mode = argv[i + 1];
245+
i++;
246+
if (strcmp(view_mode, "both") != 0 && strcmp(view_mode, "topdown") != 0 &&
247+
strcmp(view_mode, "agent") != 0) {
248+
fprintf(stderr, "Error: --view must be 'both', 'topdown', or 'agent'\n");
249+
return 1;
250+
}
251+
} else {
252+
fprintf(stderr, "Error: --view option requires a value (both/topdown/agent)\n");
253+
return 1;
254+
}
255+
} else if (strcmp(argv[i], "--map-name") == 0) {
256+
if (i + 1 < argc) {
257+
map_name = argv[i + 1];
258+
i++;
259+
} else {
260+
fprintf(stderr, "Error: --map-name option requires a map file path\n");
261+
return 1;
262+
}
263+
} else if (strcmp(argv[i], "--policy-name") == 0) {
264+
if (i + 1 < argc) {
265+
policy_name = argv[i + 1];
266+
i++;
267+
} else {
268+
fprintf(stderr, "Error: --policy-name option requires a policy file path\n");
269+
return 1;
270+
}
271+
} else if (strcmp(argv[i], "--output-topdown") == 0) {
272+
if (i + 1 < argc) {
273+
output_topdown = argv[i + 1];
274+
i++;
275+
}
276+
} else if (strcmp(argv[i], "--output-agent") == 0) {
277+
if (i + 1 < argc) {
278+
output_agent = argv[i + 1];
279+
i++;
280+
}
281+
} else if (strcmp(argv[i], "--num-maps") == 0) {
282+
if (i + 1 < argc) {
283+
num_maps = atoi(argv[i + 1]);
284+
i++;
285+
}
286+
}
287+
}
288+
181289
// performance_test();
182-
demo();
290+
demo(map_name, policy_name, show_grid, obs_only, lasers, show_human_logs, frame_skip, view_mode, output_topdown,
291+
output_agent, num_maps, zoom_in);
183292
// test_drivenet();
184293
return 0;
185294
}

pufferlib/ocean/drive/drive.h

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -971,7 +971,7 @@ int collision_check(Drive *env, int agent_idx) {
971971
int index = -1;
972972
if (i < env->active_agent_count) {
973973
index = env->active_agent_indices[i];
974-
} else if (i < env->num_actors) {
974+
} else if (i < env->num_actors && env->static_agent_count > 0) {
975975
index = env->static_agent_indices[i - env->active_agent_count];
976976
}
977977
if (index == -1)
@@ -1791,7 +1791,7 @@ void compute_observations(Drive *env) {
17911791
int index = -1;
17921792
if (j < env->active_agent_count) {
17931793
index = env->active_agent_indices[j];
1794-
} else if (j < env->num_actors) {
1794+
} else if (j < env->num_actors && env->static_agent_count > 0) {
17951795
index = env->static_agent_indices[j - env->active_agent_count];
17961796
}
17971797
if (index == -1)
@@ -2803,7 +2803,7 @@ void c_render(Drive *env) {
28032803
handle_camera_controls(env->client);
28042804
draw_scene(env, client, 0, 0, 0, 0);
28052805

2806-
if (IsKeyPressed(KEY_TAB)) {
2806+
if (IsKeyPressed(KEY_TAB) && env->active_agent_count > 0) {
28072807
env->human_agent_idx = (env->human_agent_idx + 1) % env->active_agent_count;
28082808
}
28092809

0 commit comments

Comments
 (0)