[New Scenario] Road Traffic (#99)

* Add road traffic scenario

* Adjust kinematic bicycle model

* Adjust README

* Adjust debug file for kinematic bicycle model

* Add "road_traffic" to the list of scenarios

* Format the code to meet the requirements

* [BugFix] Change class name

* [Minor] Default agent number 20

* Code refactoring
- Scenario class appers first, helper function apper later
- Split some mega functions (make_world, reset_world_at, and reward) to subfunctions to improve readability
- Add more detailed function description

* Amend minor
- Readme: corrected the DIO of the paper
- test_vmas: removed debugging code pushed by mistake

* Amend - major
* Added recap to the Scenario class, with a special focus on the explanation of the parameters
* Added more detailed description to the function done
*  Removed some unnecessary parameters such as max_steps, shared_reward, and step_duration
* Put r_p_normalizer outside .get
* Add ValueError if n_agents does not meet requirements for scenario_type. The same for n_nearing_agents_observed
* Adjusted some default parameters

* Amend - minor
- Readme: add some useful info
- road_traffic: change from kwargs.get to kwargs.pop
- road_traffic: explain prioritized replay buffer

* Amend - minor
- Explain scenario_type 4 in more detail

* Amend - minor
- Change scenario_type to map_type
- Add more info about map_type to docstring

* Amend - minor
- Add traffic scenario map data to MANIFEST.in

* Amend - minor
- Add vmas/scenario_data to MANIFEST.in

Author: Jianye-Xu

MANIFEST.in

@@ -1 +1,2 @@
 include vmas/simulator/secrcode.ttf
+include vmas/scenarios_data

README.md

@@ -345,8 +345,7 @@ To create a fake screen you need to have `Xvfb` installed.
 | **<p align="center">joint_passage_size</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/joint_passage_size.gif?raw=true"/> | **<p align="center">flocking</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/flocking.gif?raw=true"/>             | **<p align="center">discovery</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/discovery.gif?raw=true"/>                       | 
 | **<p align="center">joint_passage</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/joint_passage.gif?raw=true"/>           | **<p align="center">ball_passage</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/ball_passage.gif?raw=true"/>     | **<p align="center">ball_trajectory</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/ball_trajectory.gif?raw=true"/>           |
 | **<p align="center">buzz_wire</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/buzz_wire.gif?raw=true"/>                   | **<p align="center">multi_give_way</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/multi_give_way.gif?raw=true"/> | **<p align="center">navigation</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/navigation.gif?raw=true"/>                     |
-| **<p align="center">sampling</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/sampling.gif?raw=true"/>                     | **<p align="center">wind_flocking</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/wind_flocking.gif?raw=true"/>   |                                                                                                                                                                           |
-
+| **<p align="center">sampling</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/sampling.gif?raw=true"/>                     | **<p align="center">wind_flocking</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/wind_flocking.gif?raw=true"/>   | **<p align="center">road_traffic</p>** <br/> <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/road_traffic_cpm_lab.gif?raw=true"/>                         |
 #### Main scenarios
 
 | Env name                | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | GIF                                                                                                                                          |
@@ -371,7 +370,7 @@ To create a fake screen you need to have `Xvfb` installed.
 | `navigation.py`         | Randomly spawned agents need to navigate to their goal. Collisions can be turned on and agents can use LIDARs to avoid running into each other. Rewards can be shared or individual. Apart from position, velocity, and lidar readings, each agent can be set up to observe just the relative distance to its goal, or its relative distance to *all* goals (in this case the task needs heterogeneous behavior to be solved). The scenario can also be set up so that multiple agents share the same goal.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/navigation.gif?raw=true" alt="drawing" width="300"/>         |
 | `sampling.py`           | `n_agents` are spawned randomly in a workspace with an underlying gaussian density function composed of `n_gaussians` modes. Agents need to collect samples by moving in this field. The field is discretized to a grid and once an agent visits a cell its sample is collected without replacement and given as reward to the whole team (or just to the agent if `shared_rew=False`). Agents can use a lidar to sens each other. Apart from lidar, position and velocity observations, each agent observes the values of samples in the 3x3 grid around it.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/sampling.gif?raw=true" alt="drawing" width="300"/>           |
 | `wind_flocking.py`      | Two agents need to flock at a specified distance northwards. They are rewarded for their distance and the alignment of their velocity vectors to the reference. The scenario presents wind from north to south. The agents present physical heterogeneity: the smaller one has some aerodynamical properties and can shield the bigger one from wind, thus optimizing the flocking performance. Thus, the optimal solution to this task consists in the agents performing heterogeneous wind shielding. See the [SND paper](https://matteobettini.github.io/publication/system-neural-diversity-measuring-behavioral-heterogeneity-in-multi-agent-learning/) for more info.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/wind_flocking.gif?raw=true" alt="drawing" width="300"/>      |
-
+| `road_traffic.py`         | This scenario provides a MARL benchmark for Connected and Automated Vehicles (CAVs) using a High-Definition (HD) map from the Cyber-Physical Mobility Lab ([CPM Lab](https://cpm.embedded.rwth-aachen.de/)), an open-source testbed for CAVs. The map features an eight-lane intersection and a loop-shaped highway with multiple merge-in and -outs, offering a range of challenging traffic conditions. Forty loop-shaped reference paths are predefined, allowing for simulations with infinite durations. You can initialize up to 100 agents, with a default number of 20. In the event of collisions during training, the scenario reinitializes all agents, randomly assigning them new reference paths, initial positions, and speeds. This setup is designed to simulate the unpredictability of real-world driving. Besides, the observations are designed to promote sample efficiency and generalization (i.e., agents' ability to generalize to unseen scenarios). In addition, both ego view and bird's-eye view are implemented; partial observation is also supported to simulate partially observable Markov Decision Processes. See [this paper](http://dx.doi.org/10.13140/RG.2.2.24505.17769) for more info.                                                                                                                                                                                                                                                                                                                          | <img src="https://github.com/matteobettini/vmas-media/blob/main/media/scenarios/road_traffic_cpm_lab.gif?raw=true" alt="drawing" width="300"/>         |
 #### Debug scenarios
 
 | Env name               | Description                                                                                                                                                                                                                                                                                                                                                                             | GIF                                                                                                                                         |

tests/test_vmas.py

@@ -114,7 +114,11 @@ def test_global_reset(scenario, num_envs=10, n_steps=10):
 
 @pytest.mark.parametrize("scenario", vmas.scenarios + vmas.mpe_scenarios)
 def test_vmas_differentiable(scenario, n_steps=10, n_envs=10):
-    if scenario == "football" or scenario == "simple_crypto":
+    if (
+        scenario == "football"
+        or scenario == "simple_crypto"
+        or scenario == "road_traffic"
+    ):
         pytest.skip()
     env = make_env(
         scenario=scenario,

vmas/__init__.py

@@ -42,6 +42,7 @@
         "navigation",
         "sampling",
         "wind_flocking",
+        "road_traffic",
     ]
 )
 """List of the vmas scenarios (excluding MPE and debug)"""