Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model,
Alex X. Lee, Anusha Nagabandi, Pieter Abbeel, Sergey Levine.
In Neural Information Processing Systems (NeurIPS), 2020.
- Linux or macOS
- Python >=3.5
- CPU or NVIDIA GPU + CUDA CuDNN
- Clone this repo:
git clone -b master --single-branch https://github.com/alexlee-gk/slac.git
cd slac- To use the DeepMind Control Suite, follow the instructions in the
dm_controlpackage. - To use OpenAI Gym , follow the instructions in the
gymandmujoco_pypackages. - Modify the
requirements.txtfile if necessary:- Replace
tf-nightly-gpuwithtf-nightlyif using CPU. - Omit
gym,mujoco-py, ordm_controlaccordingly if only using one of the suites.
- Replace
- Install python packages:
pip install -r requirements.txt- Install the
tf_agentspackage:
pip install git+git://github.com/tensorflow/agents.git- Install ffmpeg (optional, used to generate GIFs for visualization in TensorBoard).
- For some python installations, the root directory should be added to the
PYTHONPATH:
export PYTHONPATH=path/to/slac:$PYTHONPATHCUDA_VISIBLE_DEVICES=0 python slac/agents/slac/examples/v1/train_eval.py \
--root_dir logs \
--experiment_name slac \
--gin_file slac/agents/slac/configs/slac.gin \
--gin_file slac/agents/slac/configs/dm_control_cheetah_run.ginTo view training and evaluation information (e.g. learning curves, GIFs of rollouts and predictions), run tensorboard --logdir logs and open http://localhost:6006.
The gin-configurable parameters can be modified using the --gin_param flag, e.g.
CUDA_VISIBLE_DEVICES=0 python slac/agents/slac/examples/v1/train_eval.py \
--root_dir logs \
--experiment_name slac \
--gin_file slac/agents/slac/configs/slac.gin \
--gin_file slac/agents/slac/configs/dm_control_cheetah_run.gin \
--gin_param train_eval.gpu_allow_growth=True \
--gin_param train_eval.sequence_length=8 \
--gin_param train_eval.action_repeat=2No matching distribution found for tf-nightly-gpu==1.15.0.dev20190821 (or similar) when installing packages in requirements.txt.
Upgrade pip: pip install --upgrade pip.
pkg_resources.VersionConflict: (setuptools 40.8.0 (.../lib/python3.7/site-packages), Requirement.parse('setuptools>=41.0.0')) when running tensorboard.
Upgrade setuptools: pip install --upgrade setuptools.
Make sure to exactly use the versions of the python packages in the requirements.txt file and in the installation instructions, e.g.
pip install --upgrade -r requirements.txt
pip install --upgrade git+git://github.com/tensorflow/agents.git
If you find this useful for your research, please use the following.
@inproceedings{lee2020slac,
title={Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model},
author={Alex X. Lee and Anusha Nagabandi and Pieter Abbeel and Sergey Levine},
booktitle={Neural Information Processing Systems (NeurIPS)},
year={2020}
}