I recently gave a 15 minute talk on "Provably Safe AGI" at a "Mechanistic Interpretability" conference at MIT: youtube.com/watch?v=sp0L-z I believe this approach could lead to human flourishing in an AGI world of abundance.
Steve Omohundro
@steveom
Research Scientist for Beneficial AI.
Steve Omohundro’s Tweets
Smaller LLMs Can Imitate Reasoning of Larger LLMs
-13-billion param model learns from rich GPT-4 signals (explanations, step-by-step, complex instructions) guided by teaching of ChatGPT
-Beats SoTA instruction-tuned LLM (Vicuna-13B) by 100% in reasoning
arxiv.org/abs/2306.02707
10
151
660
5M tokens of context. Let that sink in.
Yes, there's caveats. But consider what's to come:
- Entire codebases in prompts
- Novel-length spec docs as instructions
- k-shots where k = 10K
- Few-shots where each "shot" is 50K LoC → diff
Those who declared the imminent death of… Show more
Quote Tweet
Meet LTM-1: LLM with *5,000,000 prompt tokens*
That's ~500k lines of code or ~5k files, enough to fully cover most repositories.
LTM-1 is a prototype of a neural network architecture we designed for giant context windows.
Show this thread
0:25
52.8K views
20
103
637
Show this thread
RealityGPT: The world’s first wearable that gives you ✨perfect memory✨
🧠 Never forget event details
🤖 Use AI agents in the real world
🌎 Get information from the Internet in real time
🗣️ Speak any language
Be the funniest & smartest person in every room:
82
292
1,076
Show this thread
Published Cryptographic and auxiliary approaches relevant for AI safety, part 4 of a 5 part sequence on security and cryptography approaches relevant for AI safety. Comments welcome!
Sequence: lesswrong.com/posts/FRQNC7rJ
Part 4:
1
6
26
Show this thread
Reading OpenAI's latest paper: "Let's Verify Step by Step". The idea is so simple that it fits in one tweet:
For challenging step-by-step problems, give reward at each step, instead of a single reward at the end. Basically, dense reward signal > sparse.
The Process Reward Model… Show more
40
258
1,498
Show this thread
Our paper "Mostly Automated Proof Repair for Verified Libraries" (with and ) got a Distinguished Paper Award at the upcoming PLDI'23.
To celebrate this, let me tell why I think this is a really cool work and why you should give it a read. (1/n)
13
39
190
Show this thread
New + paper!📜
Symbol tuning is a simple method that improves in-context learning by emphasizing input–label mappings. It improves robustness to prompts without instructions/relevant labels and boosts performance on algorithmic tasks.
arxiv.org/abs/2305.08298
15
184
662
Show this thread
Embodied Agent Experiences Enhance LLMs
-Deploy LLM agent in simulator of physical world to acquire diverse experience via goal-oriented planning
-Finetune LLM on that exp to teach acting in world
-Improves over base on 18 downstream tasks by 64% on avg
arxiv.org/abs/2305.10626
8
86
340
Promising. Everyone should hope that we can throw away tokenization in LLMs. Doing so naively creates (byte-level) sequences that are too long, so the devil is in the details.
Tokenization means that LLMs are not actually fully end-to-end. There is a whole separate stage with… Show more
91
621
3,825
100k tokens. Incredible.
Demos all show long input; that's just low-hanging fruit. Think novel-length instruction — entire employee manuals as prefixes. Think k-shots — embed, take 200 nearest HDBSCAN clusters, sample 3 from each.
"Long prompting" changes everything.
Quote Tweet
Introducing 100K Context Windows! We’ve expanded Claude’s context window to 100,000 tokens of text, corresponding to around 75K words. Submit hundreds of pages of materials for Claude to digest and analyze. Conversations with Claude can go on for hours or days.
Show this thread
0:38
370.1K views
16
53
498
Show this thread
Our Stochastic GFlowNets paper got accepted to 😼
In this work, we propose a novel "model-based" stochastic GFlowNet method for extending GFlowNets to environments with stochastic dynamics🎲, which is essential in many control tasks.
3
16
87
Show this thread
It's a great question. I roughly think of finetuning as analogous to expertise in people:
- Describe a task in words ~= zero-shot prompting
- Give examples of solving task ~= few-shot prompting
- Allow person to practice task ~= finetuning
With this analogy in mind, it's… Show more
29
197
874
Artificial Intelligence is one of the most powerful tools of our time, but to seize its opportunities, we must first mitigate its risks.
Today, I dropped by a meeting with AI leaders to touch on the importance of innovating responsibly and protecting people's rights and safety.
3,902
3,526
13.4K
you people love nothing more than a "leaked internal google memo"
and your breathless "no moats" retweets have compelled me to set you straight with another AI-obsessed megathread 😉🧵
tl;dr: we'll see everything, everywhere, all at once, but OpenAI (& Google) have real moats!
21
93
404
Show this thread
Today we’re releasing “The A.I. Dilemma” – a new talk and I gave on 3/9, a week before GPT4 launched.
*Pls share it widely.* It's critical for institutions to understand how the race between AI labs is accelerating the likelihood of catastrophe:
199
1,120
1,994
Show this thread
Humanity's lackadaisical response to the superintelligence threat makes me feel I'm in the movie "Don't Look Up". I just explained why here:
378
1,251
3,908
New version of Broken Neural Scaling Laws (BNSL) is out with accurate extrapolation results for the scaling behaviors listed in this attached picture:
arxiv.org/abs/2210.14891
arxiv.org/pdf/2210.14891
Plots of all extrapolations are in this 🧵.
Any other extrapolations you want?
5
26
74
Show this thread
Meta-Reasoning Over LLM's Chains of Thought
-Prompts LLMs to reason over multiple chains of thought
-Examines reasoning chains
-Mixes info & selects most relevant facts in generating
answer
-Outperforms strong baselines
-Humans can verify explanations
arxiv.org/abs/2304.13007
3
85
354
Guys I would like to emphasize that A Game of Thrones is ~894,000 tokens
The entire Harry Potter series is ~2.5M
Forget asking an LLM to write a blog post or an essay
Have it write you an entire high-fantasy novel series
We are on the cusp of infinite, personalized media
Quote Tweet
6
30
104
Show this thread
4 Autonomous AI Agents you need to know:
“Westworld” simulation, Camel, BabyAGI, AutoGPT ⭐ with the power of ⭐ Thanks for the guidance ❤️
Video: youtube.com/watch?v=yWbnH6
Blog: sophiamyang.medium.com/4-autonomous-a
21
88
379
🧵1/ Just attended a 🔥 webinar on AI agents, ft. some of the brightest minds in the space! Let's unpack the key takeaways & explore the cutting-edge work being done.
Guests:
6
40
310
Show this thread
A great summary of all the companies and projects which aim to compose large language models into more capable systems:
2
1
Solving Math Problems by Combining LLMs & Symbolic Solvers
-LLM that can incrementally formalize word problems into equations
+
-External symbolic solver for equations
Outperforms PAL by 20% on new dataset of challenging textbook Algebra word problems
arxiv.org/abs/2304.09102
1
83
362
## Can open-source LLMs detect bugs in C++ code?
No:
```
LLaMa 65B (4-bit GPTQ) model: 1 false alarms in 15 good examples. Detects 0 of 13 bugs.
Baize 30B (8-bit) model: 0 false alarms in 15 good examples. Detects 1 of 13 bugs.
Galpaca 30B (8-bit) model: 0 false alarms in 15… ~~~~~~~~~~ hf3f8e3a 992bba08-8399-4bde-ab97-c1305e64876 SSR-I18N f2c6ac64-eb07-4bf8-bb18-52a36cf153b7 hf3f8e3a ~~~~~~~~~~
42
221
1,065
Show this thread
#つぶやきGLSL
float e,i,a,w,x,g;for(;i++<1e2;){vec3 p=vec3((FC.xy-.5*r)/r.y*g,g-3.);p.zy*=rotate2D(.6);i<1e2?p:p+=1e-4;e=p.y;for(a=.8;a>.003;a*=.8)p.xz*=rotate2D(5.),x=(++p.x+p.z)/a+t+t,w=exp(sin(x)-2.5)*a,o.gb+=w/4e2,p.xz-=w*cos(x),e-=w;g+=e;}o+=min(e*e*4e6,1./g)+g*g/2e2;
101
2,073
15K
Show this thread
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
GPT-4 surpasses average human perf on SAT, LSAT, and math competitions, attaining 95% on SAT Math.
repo: github.com/microsoft/AGIE
abs: arxiv.org/abs/2304.06364
6
55
197
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Presents RETRO++, which significantly outperforms retrieval-augmented GPT across different model sizes.
repo: github.com/NVIDIA/Megatro
abs: arxiv.org/abs/2304.06762
5
79
346
Power-Seeking Can Be Probable For Trained AI Agents
-Assuming trained AI agent learns a goal from set of goals consistent w/ training rewards:
-If agent faces a choice to
shut down or avoid shutdown
in a new situation, agent is likely to avoid shutdown
arxiv.org/abs/2304.06528
7
38
243
Self-Refine: Iterative Refinement with Self-Feedback
Presents a novel approach that allows LLMs to iteratively refine outputs and incorporate feedback along multiple dimensions to improve performance on diverse tasks.
proj: selfrefine.info
abs: arxiv.org/abs/2303.17651
GIF
2
89
301
📢New Paper: Self-Refine: Iterative Refinement with Self-Feedback
We use an LLM such as GPT-3.5/4 to:
1. Generate output
2. Provide feedback on its own output
3. Refine its previous output
Improves outputs across 7 tasks without additional data/training!
arxiv.org/pdf/2303.17651
4
19
102
Show this thread
“Emergent autonomous scientific research capabilities of large language models”
is this the holy grail?
arxiv.org/pdf/2304.05332
12
123
508
Show this thread
Why Think Step-by-Step? Using LLMs to Understand Reasoning
-Reasoning is effective when training data has clusters of variables influencing each other strongly
-Enables chaining of local inferences to estimate relationships not seen together in training
arxiv.org/abs/2304.03843
12
133
726
Almost everyone I know working in AI these days feels one step away from total burnout. I took the time to take you behind the curtain and know what people on the state-of-the-art AI are struggling with:
13
186
711
Show this thread
This Google paper basically uses ChatGPT to create a primitive 25 person v1.0 of “The Matrix”
22
161
965
Natural Language Reasoning, A Survey
An overview of natural language reasoning in NLP. Contains lots of discussions around language models, capabilities, limitations, and open questions.
arxiv.org/abs/2303.14725
4
116
473
Generative Agents: Interactive Simulacra of Human Behavior
abs: arxiv.org/abs/2304.03442
project page: reverie.herokuapp.com/arXiv_Demo/
63
760
2,473






























