Based on PARL and PaddlePaddle, a parallel version of SAC was implemented and achieved high performance in the CARLA environment.
Paper: SAC in Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
Please view Carla simulator to know more about Carla simulator.
- Result was evaluated with mode
Lane
- System: Ubuntu 16.04
- parl>=2.0.0
- paddlepaddle>=2.0.0
- CARLA_0.9.6
# Download CARLA_0.9.6, extract it to a folder, cd ${YOUR_FLODER} wget http://carla-assets-internal.s3.amazonaws.com/Releases/Linux/CARLA_0.9.6.tar.gz mkdir CARLA_0.9.6 tar -zxf CARLA_0.9.6. 4172 tar.gz -C CARLA_0.9.6 # and add the folder path to `PYTHONPATH` environment variable $ export PYTHONPATH="${YOUR_FLODER}/CARLA_0.9.6/PythonAPI/carla/dist/carla-0.9.6-py3.5-linux-x86_64.egg:$PYTHONPATH" - gym_carla
$ git clone https://github.com/ShuaibinLi/gym_carla.git $ cd gym_carla $ pip install -r requirements.txt $ pip install -e .
-
Open another(new) terminal, enter the CARLA root folder and launch CARLA service. There are two modes to start CARLA service:
(1) non-display mode$ DISPLAY= ./CarlaUE4.sh -opengl -carla-port=2021(2) display mode
$ ./CarlaUE4.sh -windowed -carla-port=2021- Start three CARLA services (ports: 2021,2023,2025) for data collecting and training, one service (port: 2027) for evaluating.
-
For parallel training, we can execute the following xparl command to start a PARL cluster:
$ xparl start --port 8080check xparl cluster status by
xparl status -
Start training
# Make sure 'localhost:8080' in train.py is the same as xparl address $ python train.py
Open another(new) terminal, enter the CARLA root folder and launch CARLA service with display mode.
$ ./CarlaUE4.sh -windowed -carla-port=2029
Visualize the evaluation with the trained model.
$ python evaluate.py --restore_model model.ckpt