Opens profile photo
Follow
MIRI
@MIRIBerkeley
MIRI exists to maximize the probability that the creation of smarter-than-human intelligence has a positive impact.
Non-Governmental & Nonprofit Organization Berkeley, CAintelligence.orgJoined July 2013

MIRI’s Tweets

Had an insightful conversation with about AI and catastrophic risks. Two thoughts we want to share: (i) It's important that AI scientists reach consensus on risks-similar to climate scientists, who have rough consensus on climate change-to shape good policy.… Show more
216
3,540
Replying to and
RE “actual mathematical model”: Suppose 10 million years ago, Alien A is looking at Earth, and says: “Descendants of chimps might pose an existential risk to descendants of ground sloths.” Alien B says: “You’re crazy. They’re on different continents, adapted for different…Show more
3
47
So if you ask me to ignore all the particular wrong ideas that people come in with, that need refuting; and focus on your own particular interest in the particular step of the larger argument that 'alignment is superhard'; then *I* might identify these key points: 1. Alignment…Show more
12
64
Replying to
Thanks for replying! Your reply is lacking some detail when it comes to evaluating the proposal for predicted success or potential flaws. I will try to fill in some system details the obvious way, and describe how the resulting system is still difficult to align. If you don't…Show more
21
193
To spell out the object-level argument every time, rather than just shaking my head: Most possible sufficiently intelligent minds with complicated goals, that happen not to care about humanity at all one way or another, will: - Want to use up all of resources we use (eg, extract… Show more
51
397
Show this thread
This is a nice demonstration of the old Yudkowsky argument of how even linear improvement can look like a dramatic transition when the human ability range is fairly narrow. I predict it will however be more revolutionary in supplying problems than solving them.
Quote Tweet
New paper comparing GPT-3.5 & GPT-4 performance on college physics problems. It shows that in just a few months, AI has made a leap from the 39th to the 96th percentile of human level performance. Now imagine were it will be in 10 years.
Image
15
260
Show this thread
Consider that somewhere on the internet is probably a list of thruples: <product of 2 prime numbers, first prime, second prime>. GPT obviously isn't going to predict that successfully for significantly-sized primes, but it illustrates the basic point: There is no law saying… Show more
50
931
Show this thread
I think I'm moderately less excited about mech interp than I used to be, but a world that's on track to not totally drop the ball on AI has a *lot* of people who like computer-puzzle-thinking tasks throwing effort at these problems. If you're such a person, have a look!
Quote Tweet
Thanks to @jaylbailey, 200 Concrete Open Problems in Mechanistic Interpretability now has a database! All, er, 339 problems are listed and can be sorted by difficulty. You can note which you're working on, and reach out to other people doing the same. neelnanda.io/cop-spreadsheet
3
98
The way that the most prominent critics of AI risk totally fail to engage with even the most basic arguments made by people in the field suggests that they don't have any good counterarguments. That's very concerning!
Quote Tweet
“Don’t Fear the Terminator” Artificial intelligence never needed to evolve, so it didn’t develop the survival instinct that leads to the impulse to dominate others. Article by @TonyZador @YLeCun blogs.scientificamerican.com/observations/d
Show this thread
33
486
Show this thread
If I wrote an "AGI ruin FAQ", what Qs would you, yourself, personally, want answers for? Not what you think "should" be in the FAQ, what you yourself genuinely want to know; or Qs that you think have no good answer, but which would genuinely change your view if answered.
309
467
Why doesn't everyone realize how hard it is to avoid AI doom? It's common for scientists to race into a new field without realizing what its 50-year problems are, & become wiser with time. The problem is that we need to get superintelligent AGI right on the first try or we die.
5
127
Show this thread
I'm not a big fan of the "takeoff" analogy for AGI. In real life, AGI doesn't need to "start on the ground". You can just figure out how to do AGI and find that the easy way to do AGI immediately gets you a model that's far smarter than any human. Less "takeoff", more "teleport".
13
173
Show this thread
Any time somebody proposes an alignment solution that says "train an LLM to output..." I suggest substituting the phrase "hire an actress to pretend..." and check if the alignment plan would still make sense.
57
618
Past EAs: Don't be ridiculous, Eliezer, as soon as AIs start to show signs of agency or self-awareness or that they could possibly see humans as threats, their sensible makers won't connect them to the Internet. Reality: lol this would make a great search engine
115
3,864
Show this thread
Replying to
Because you need to be smarter to predict plausible plaintext from hashes, than to write down a hash followed by its plaintext. More generally, it's just not true that you only need to be as intelligent as a human to predict exactly what a particular human will say.
4
64