Skip to content

Latest commit

 

History

History
182 lines (163 loc) · 8.94 KB

README.md

File metadata and controls

182 lines (163 loc) · 8.94 KB
Raw

Data-driven Methods for Network-level Coordination of Autonomous Mobility-on-Demand Systems Across Scales

Official implementation of the Data-driven Methods for Network-level Coordination of Autonomous Mobility-on-Demand Systems Across Scales tutorial, presented at 27th IEEE International Conference on Intelligent Transportation Systems 2024


Join us in moving the first steps toward the creation of publicly available benchmarks, datasets, and simulators for network-level coordination of MoD systems.

We aim to provide:

  • Openly accessible simulation platforms across different fidelity levels (macroscopic, mesoscopic, microscopic)
  • Calibrated scenarios
  • Implementation of fleet coordination algorithms
  • A common validation process that allows for direct comparison between different approaches (i.e., leaderboard-based)
  • Standardized pipeline for contributed algorithms, scenarios, tools, etc.

Prerequisites

  • We recommend using CPLEX, however if you don't have access to a IBM cplex installation, we provide an alternative with the free python package PuLP. If you are a student or academic, IBM is releasing CPLEX Optimization Studio for free. You can find more info here

  • You will need to have a working installation of SUMO (Simulation of Urban MObility). It is an open source microscopic and continous multi-modal traffic simulation package used to handle large networks. The repo is based on the version 1.20.0. Find the information for SUMO installation here

  • To install all required dependencies, run

pip install -r requirements.txt

Contents

RL4AMOD
├── testing.py: test main file
├── train.py: RL agents train main file
├── src/
│ ├── algos/
│ │ ├── sac.py: Pytorch implementation of SAC
│ │ ├── a2c.py: PyTorch implementation of A2C
│ │ ├── base.py: Base class for controller
│ │ ├── ed.py: Equal distribution controller
│ │ ├── ...
│ │ └── registry.py: models registration function
│ ├── nets/
│ │ ├── actor.py: Pytorch implementation of a GNN-based actor
│ │ ├── critic.py: Pytorch implementation of a GNN-based critic
│ ├── envs/
│ │ ├── data/: data for the macroscopic and mesoscopic scenarios
│ │ └── sim/
│ │ │ ├── macro_env.py: implementation of a macroscopic simulator for AMoD systems system
│ │ │ └── sumo_env.py: implementation of a SUMO-based mesoscopic simulator for AMoD systems
│ ├── misc/: helper functions
│ ├── config/: default parameters for the simulator and the models
│ ├── cplex_mod/: CPLEX formulation of Rebalancing, Matching and MPC problems
├── saved_files/: directory for saving results, logging, etc

Configuration parameters

To run a training or a testing, firstly the simulator and the model type must be selected with the config arguments

config arguments:
    simulator           simulator fidelity between 'macro' and 'sumo' (default: macro)
    model               selected model: choose between the models in src/algos/registry.py file (default: sac)

Simulators parameters

You need to pass the following argument to set a simulator parameter:

simulator.{arg}=value

Use the following argument for macroscopic simulator:

simulator=macro

macro simulator arguments:
    seed                random seed (default: 10)
    demand_ratio        ratio of demand (default: 0.5)
    json_hr             hour of the day for JSON configuration (default: 7)
    json_tstep          minutes per timestep (default: 3 min)
    beta                cost of rebalancing (default: 0.5)
    city                city: defines the city (default: 'nyc_brooklyn')
    max_steps           number of steps per episode (default: T=20)
    time_horizon        time horizon for demand and arriving vehicle forecast (default: 6)
    directory           directory where to save files

Use the following argument for mesoscopic simulator:

simulator=sumo

sumo simulator arguments:
    sumocfg_file        define the SUMO configuration file
    net_file            define the city network file
    seed                random seed (default: 10)
    demand_ratio        demand ratio (default: 0.8)
    time_start          simulation start time in hours (default: 7)
    duration            episode duration in hours (default: 2 hr)
    time_horizon        matching steps in the future for demand and arriving vehicle forecast (default: 10 min)
    matching_tstep      minutes per timestep (default: 1 min)
    reb_tstep           minutes per timestep (default: 3 min)
    sumo_tstep          sumo time step (default: 1 s)
    max_waiting_time    maximum passengers waiting time for a ride (default: 10 min)
    beta                cost of rebalancing (default: 1)
    num_regions         number of regions for spatial aggregation (default: 8)
    random_od           demand aggregated in the centers of the regions (default: False)
    acc_init            initial number of taxis per region (default: 90)
    city                (default: 'lux')
    directory           defines directory where to save files

Models parameters

You need to pass the following argument to set a simulator parameter:

model.{arg}=value

Use the following argument for a2c agent:

model=a2c

a2c model arguments:
    agent_name          agent name for training or evaluation (default: today's date + '_a2c_gnn')
    cplexpath           defines directory of the CPLEX installation
    directory           defines directory where to save files
    max_episodes        number of episodes to train agent (default: 16k)
    max_steps           number of steps per episode (default: T=120)
    no_cuda             disables CUDA training (default: true)
    batch_size          defines batch size (default: 100)
    p_lr                define policy learning rate (default: 1e-3)
    q_lr                defines q-value learning rate (default: 1e-3)
    hidden_size         defines hidden units in the MLP layers (default: 256)
    clip                clip value for gradient clipping (default: 500)
    checkpoint_path     path where to save model checkpoints (A2C)

Use the following argument for sac agent:

model=sac

sac model arguments:
    agent_name          agent name for training or evaluation (default: 'SAC'+'{city}')
    cplexpath           defines directory of the CPLEX installation
    max_episodes        number of episodes to train agent (default: 16k)
    no_cuda             disables CUDA training (default: true)
    batch_size          batch size (default: 100)
    p_lr                policy learning rate (default: 1e-3)
    q_lr                q-value learning rate (default: 1e-3)
    alpha               entropy coefficient (default: 0.3)
    auto_entropy        use automatic entropy tuning (default: false)
    hidden_size         hidden units in the MLP layers (default: 256)
    clip                clip value for gradient clipping (default: 500)
    checkpoint_path     path where to save model checkpoints (SAC)
    rew_scale           reward scale (default: 0.01)
    use_LSTM            use LSTM in the model (default: false)
    input_size          number of node features (defalut: 13)
    test_episodes       number of episodes to test agent (default 10)

Important: Take care of specifying the correct path for your local CPLEX installation. Typical default paths based on different operating systems could be the following. If model.cplexpath = None, the PulP solver will be automatically called.

Windows: "C:/Program Files/ibm/ILOG/CPLEX_Studio128/opl/bin/x64_win64/"
OSX: "/Applications/CPLEX_Studio128/opl/bin/x86-64_osx/"
Linux: "/opt/ibm/ILOG/CPLEX_Studio128/opl/bin/x86-64_linux/"

Examples

Training and simulating an agent

  1. To train an agent (with the default parameters) run the following:
python train.py  simulator.name=macro model=sac simulator.city=nyc_brooklyn model.checkpoint_path=SAC_custom
  1. To evaluate a pretrained agent run the following:
python testing.py  simulator.name=macro model=sac simulator.city=nyc_brooklyn model.checkpoint_path=SAC_custom

Testing baselines

  1. To evaluate a baseline (e.g. equal_distribution) run the following (All available models can be found in src/algos/registry.py):
python testing.py  simulator.name=macro model.name=equal_distribution simulator.city=nyc_brooklyn

In case of any questions, bugs, suggestions or improvements, please feel free to contact us at [email protected], [email protected] or [email protected].