🔥 CleanRL's paper has been accepted to ! Introducing at v1.0.0! We have added reworked documentation, JAX support, hyperparameter tuning, and more.
📜 Paper: jmlr.org/papers/v23/21-
💾Release:
Conversation
Replying to
💪 We have completely revamped the documentation site. Now each of our 23 algorithm variants has detailed documentation on usage, an explanation of logged metrics, implementation details, and experiment results.
docs.cleanrl.dev/rl-algorithms/
1
3
🚀 This release comes with exciting support for JAX-based implementations, which makes DQN + Atari 25% faster, DDPG and TD3 2.5-4x faster, and PPO + Atari 3x faster than openai/baselines' PPO!
1
5
✌️ We have added a series of toolchain improvements. Notably, to ensure our code can run without obscure errors, we now use GitHub Action to run our algorithm variant on Windows, Linux, and macOS — things that "just work."
1
3
🔧 We also included a new experimental hyperparameter tuner.
Quote Tweet
@cleanrl_lib now has an experimental hyperparameter tuner which is a tailored fit to publish RL papers.
docs: docs.cleanrl.dev/advanced/hyper
code: github.com/vwxyzjn/cleanr
Here is why
Show this thread
1
2
🤗 What's coming next? We are adding evaluations and integration to save and load models. This will help support recent research efforts that reuse prior computation in RL to bootstrap new agents (e.g., reincarnate RL by )
twitter.com/agarwl_/status
Quote Tweet
tl;dr: What if we didn’t almost always train reinforcement learning (RL) agents from scratch for research? Our work argues for an alternative approach to research, where we build on prior computation, such as learned policies, network weights.
arxiv.org/abs/2206.01626 (1/N)
Show this thread
1
2
11
🧠 We are also adding "RLops" features that help us seamlessly integrate new features and bugs.
Quote Tweet
Experimenting *RLops* at @cleanrl_lib — we will soon be able to compare the library's performance at different versions 
!
Rough idea: tag experiments in @weights_biases with a PR number and build tools to generate analysis!
PR: github.com/vwxyzjn/cleanr
!
Rough idea: tag experiments in @weights_biases with a PR number and build tools to generate analysis!
PR: github.com/vwxyzjn/cleanrShow this thread
1
3
🤩 Moreover, we plan to support 's Gymnasium in the future to replace the now no-longer-maintained openai/gym. Notably, Shimmy will help us support `dm_env` environments such as `dm_control`!
Quote Tweet
Today we're launching the Farama Foundation, a new nonprofit dedicated to open source reinforcement learning, and we're beginning by maintaining and standardizing all the major open source reinforcement learning environments. Read more here: farama.org/Announcing-The
1
3
❤️ Lastly, I would like thank our awesome collaborators , , , , Jeff Braga, @coolRR, Alexander Nikulin, , , Shubham Dhar, Elliot Munro, and Yanxiao Zhao
2
4
This makes me so happy to see this contribution get academic recognition. Congratulations.
1
4
Thank you, Eugene :) open source track is amazing. We got very helpful feedback (e.g., one of which significantly helped us with documentation!) Also want to shout out to our editor for guiding us through the process.
1
2
Show replies