body { -ms-overflow-style: scrollbar; overflow-y: scroll; overscroll-behavior-y: none; } .errorContainer { background-color: #FFF; color: #0F1419; max-width: 600px; margin: 0 auto; padding: 10%; font-family: Helvetica, sans-serif; font-size: 16px; } .errorButton { margin: 3em 0; } .errorButton a { background: #1DA1F2; border-radius: 2.5em; color: white; padding: 1em 2em; text-decoration: none; } .errorButton a:hover, .errorButton a:focus { background: rgb(26, 145, 218); } .errorFooter { color: #657786; font-size: 80%; line-height: 1.5; padding: 1em 0; } .errorFooter a, .errorFooter a:visited { color: #657786; text-decoration: none; padding-right: 1em; } .errorFooter a:hover, .errorFooter a:active { text-decoration: underline; } #placeholder, #react-root { display: none !important; } body { background-color: #FFF !important; }

JavaScript is not available.

We’ve detected that JavaScript is disabled in this browser. Please enable JavaScript or switch to a supported browser to continue using twitter.com. You can see a list of supported browsers in our Help Center.

Terms of Service Privacy Policy Cookie Policy Imprint Ads info © 2023 Twitter, Inc.

Conversation

trlX v0.3.0 is here! We've been working hard on this for the last month and we're super excited :) Highlights: Support for hyper parameter sweeps, refactored losses and models allows for better extendability, optimizations to PPO and ILQL. github.com/CarperAI/trlx 1/5

GitHub - CarperAI/trlx: A repo for distributed training of language models with Reinforcement...

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF) - GitHub - CarperAI/trlx: A repo for distributed training of language models with Reinforcem...

4:30 PM · Nov 21, 2022

Replying to

1) We've added support for Ray Tune, and worked with

@weights_biases

to create programmatic reports from your hparam sweeps 📊 2) We've added PPO normalization among other regularization methods, improves convergence and stability. ⏳ 2/5

1

4

9

3) We've refactored the loss functions to allow for alternative backends, like Megatron NeMo (WIP) and T5X (planned) 👨‍💻 4) PPO-Hydra support for OPT and NeoX 🦑😎 (Merging later today) 3/4

1

4

Blog post here for hyper parameter tuning here wandb.ai/ayut/trlx-ppo- 4/5

Reinforcement Learning from Human Feedback: Hyperparameter Optimization for trlX

In this article, we follow a step-by-step guide for performing hyperparameter optimization using Ray Tune and Weights & Biases, looking at how trlX can help.

1

3

10

Coming soon: 1) Multi-gpu support for Ray Tune. 2) Large critic and reward models. 5/5

1

9

Replying to

mem it

1