Program synthesis from unit tests using #Codex! When I add a new test case, it automatically builds a new function and verifies that the unit tests pass. 🎉
Conversation
Replying to
This goes a step beyond what GitHub #Copilot provides because this is searching through dozens of possible solutions in the background and only picking the ones that work. This saves time but also gives you more confidence that the code is aligned with your original intent.
1
5
In "Evaluating Large Language Models Trained on Code" the authors showed that the % of tasks that #Codex can solve goes to 72% from 29% when they sampled 100 different solutions instead of just one arxiv.org/abs/2107.03374
1
3
2
1
Replying to
can you reshare on youtube/in higher resolution? hard to see what's going on
1
1
Replying to
Quote Tweet
Here's the full resolution version on YouTube: youtube.com/watch?v=rPoqWx
Show this thread
Replying to
Here's the full resolution version on YouTube:
3
1
3
Replying to
this is needed in production-level projects!! really amazing work!
1
1
Replying to
Thanks Mrinal! It's an interesting challenge to make these language models safe enough to use in production. Being able to constrain generation with unit tests is one step towards that but I feel like we're only just scratching the surface of what's possible.
Show additional replies, including those that may contain offensive content
Show