I am extremely excited to announce (1) I've joined OpenAI to lead a large-scale effort into AI-generating Algorithms research, & (2) I'll be an Associate CS Professor at U. British Columbia in 2021, where I will continue to lead the OpenAI project. Both are dreams come true! 1/2
Follow
Jeff Clune
@jeffclune
Associate Professor, CS, U. British Columbia. Faculty Member, Vector Inst. ML, AI, deep RL, deep learning, AI-Generating Algorithms (AIGAs) More: JeffClune.com
Jeff Clune’s posts
Excited to announce I have joined DeepMind as a Senior Research Advisor! I will be working with many fantastic people there on AI-generating algorithms & open-ended learning, one of many areas where @DeepMind is a leader. My roles at UBC & the Vector Institute will stay the same.
Introducing Video PreTraining (VPT): it learns complex behaviors by watching (pretraining on) vast amounts of online videos. On Minecraft, it produces the first AI capable of crafting diamond tools, which takes humans over 20 minutes (24,000 actions) 🧵👇openai.com/blog/vpt
My favorite example of when AI surprised us, the scientists wielding it. A corner case in our algo challenged AI to learn to walk without touching its feet to the ground. Many more in arxiv.org/abs/1803.03453 from @joelbot3000with Cully et al Nature 2015
GIF
Thrilled to share that "First return, then explore" appears today in Nature! Go-Explore solves all unsolved Atari games*, ending a long quest by the field that began in Nature. Led by & w & myself. nature.com/articles/s4158
1/
Introducing POET: it generates its own increasingly complex, diverse training environments & solves them. It automatically creates a learning curricula & training data, & potentially innovates endlessly! eng.uber.com/poet-open-ende By w/ &
Today is my last day at Uber. It's been a dream to have our startup acquired by Uber &, with wonderful colleagues, create Uber AI Labs. I'm proud of our work, eg. Go-Explore, POET, AI-GAs, GTNs, ANML, Differentiable Plasticity, Backpropamine, Deep Neuroevolution, PPGNs, etc. 1/3
Montezuma’s Revenge solved! 2 million points & level 159! Go-Explore is a new algorithm for hard-exploration problems. Shatters Pitfall records too 21,000 vs 0 Blog:eng.uber.com/go-explore Vid youtu.be/L_E3w_gHBOY By & me
Introducing Thought Cloning: AI agents learn to *think* & act like humans by imitating the thoughts & actions of humans thinking out loud while acting, enhancing performance, efficiency, generalization, AI Safety & Interpretability. Led by
arxiv.org/abs/2306.00323 1/5
Introducing Generative Teaching Networks, which generate entirely synthetic data that is up to 9x faster to train on than real data!, enabling state-of-the-art Neural Architecture Search eng.uber.com/generative-tea Led by w ,, & 1/
Welcoming the era of deep neuroevolution. Simple GA trains DNNs on Atari (competes with DQN, A3C, ES). Random search also competitive!
We're hiring software engineers, research engineers, & research scientists on the multi-agent team ! Interested in AI-generating algorithms, many interacting agents, open-ended algorithms, automatically generating training environments, deep RL etc? openai.com/jobs/
We have open sourced the code behind the neuroevolution papers described in our blog post eng.uber.com/deep-neuroevol This includes the code for the deep genetic algorithm that is competitive for Deep RL. We hope this helps the community. Please let us know if you try them. twitter.com/imormo/status/
This post is unavailable.
Reviewers in my experience almost always insist on SOTA for publication. This is the result. In doing so, we are asking to be lied to via p-hacking and brittle, non-reproducible results. We also choke off promising directions that are not immediately better, which is ridiculous!
Introducing Plug & Play Generative Networks. These are images synthetically generated by #DeepNeuralNetworks. More: evolvingai.org/ppgn
I am recruiting PhD students at UBC. I especially encourage applications from those underrepresented in our field. Interested in AI-GAs, meta-learning, generating learning challenges, deep RL, open-endedness, QD, exploration, generalization, continual learning, etc? Please apply!
We've open sourced the code for Plug & Play Generative Networks (PPGNs), which produced these images: evolvingai.org/ppgn Game on!
Learning to Continuously Learn. ANML meta-learns to reduce catastrophic forgetting, and can learn at least 600 tasks (Omniglot classes) sequentially and performs well on most afterwards. arXiv arxiv.org/abs/2002.09571 NeurIPS talk slideslive.com/38924020/how-a Led by Shawn Beaulieu 1/2
I agree. "I'm amazed that people confidently pronounce these things are not sentient, and when you ask them what they mean by sentient they say well they don't really know. So how can you be confident they're not sentient if you don't know what sentient means? "-Geoff Hinton 1/2
No privacy concerns here. Nope. None at all.
Quote
Full body tracking now possible using only WiFi signals
A deep neural network maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions
The model can estimate the dense pose of multiple subjects by utilizing WiFi signals as the only input
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence arxiv.org/abs/1905.10985
Historically, hand-designed pipelines are ultimately outperformed by entirely learned ones. Will that will be true of creating general AI itself? 1/6
Neural Networks are Surprisingly Modular.
Introducing the Synthetic Petri Dish for architecture search: Creates tiny NN variants in a "dish" & optimizes synthetic data to make variants perform like their full NN counterparts. Then rapid NAS search in the dish. Generalizes better than NN models! arxiv.org/pdf/2005.13092 1/
Here is the final link to my talk at the ICLR BeTR-RL workshop on AI-GAs and Learning to Continually Learn (with ANML). slideslive.com/38926301 Thanks to the workshop organizers and my collaborators! #ICLR2020 #iclr
I think biological development is the most impressive technology in the known universe. Can you think of anything more impressive? A seed (or egg) uses nanotech to gather all the resources needed to self-assemble molecules into a jaguar, hawk, oak tree, whale or human. Amazing!
Quote
A human body is so wonderfully nested. Its ~40T cells descend from individual eukaryotic cells before multi-cellularity. And each has ~1000 mitochondria, which were free-living bacteria before endosymbiosis. And all of it is home to 1-3X as many bacteria in the nooks and crannies
Go-Explore now solves all unsolved Atari games*, handles stochastic training throughout via goal-conditioned polices, reuses skills to intelligently explore after returning, and solves hard-exploration simulated robotics tasks! New paper led by & 1/6
My picks are included. What are yours?
blog.re-work.co/ai-papers-sugg
Mine: LRL by et al. (see also RL^2) and Reversible Learning by . Both papers helped inspire AI-Generating Algorithms & its bet on meta-learning, GTNs, & ANML!
Generating multi-modal robot behavior (neural networks that perform many different tasks well) via the new Combinatorial Multi-Objective Evolutionary Algorithm (CMOEA). Very proud of this work. Led by Joost Huizinga Key idea: the more stepping stones the better!
GIF
Very cool to see a replication of our paper "Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning"! Includes step-by-step instructions for AWS. Great work
Introducing Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions.Adds domain-general metrics of environment novelty and progress in open-ended algorithms + a new environment search space arxiv.org/abs/2003.08536 1/
Delighted to announce my lab was awarded a substantial grant from Open Philanthropy for work on AI alignment, safety, & existential risk. I want to make the development of powerful AI & AGI go as well as possible for humanity, & will allocate more time to these key topics
Yann LeCun et al. publishing evolutionary algorithm tools. Welcoming the era of deep neuroevolution indeed! (eng.uber.com/deep-neuroevol) Great to see the traditional ML community adopt these tools in the cases when they are useful.
Here is a new talk entirely on ANML (a Neuromodulated Meta-Learning Algorithm) ANML meta-learns to reduce catastrophic forgetting, and can learn at least 600 tasks sequentially!
videos.re-work.co/videos/1861-an paper: Learning to Continually Learn (arxiv.org/abs/2002.09571) #AI
AI-GAs++
It's learned optimizers all the way down!🐢🐢🐢
arxiv.org/abs/2101.07367
Training Learned Optimizers with Randomly Initialized Learned Optimizers
My AI-Generating Algorithms paper described "Darwin-complete" environment search spaces: those that can represent any environment. I suggested one then: neural network world models. Another is code (e.g. LM-generated code for environments/simulators/games) arxiv.org/abs/1905.10985
In the ICLR Debate I introduced an alternate path to general AI: AI-generating algorithms (AI-GAs), and the Three Pillars required to make AI-GAs. Full debate (on how much we should learn vs. build in): youtu.be/APcFQoLKVLA w Tenenbaum, Precup, Kaelbling, Thoughts?
Here are the slides from my talk on Go-Explore at the #NeurIPS2018 Deep Reinforcement Learning workshop. cs.uwyo.edu/~jeffclune/sha
Our paper was accepted at ICLR. Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity. Very exciting work led by , a pioneer in this area. With Aditya Rawal, , and myself
Join us! Our team at OpenAI is now hiring research scientists and engineers. Interested in AI-generating algorithms, multiple interacting agents, open-ended algorithms, automatically generating training environments, deep RL, and more? Please apply! openai.com/jobs
I am extremely honored to receive the Presidential Early Career Award for Scientists and Engineers (PECASE) from the White House: urldefense.proofpoint.com/v2/url?u=https Thanks to all of my wonderful collaborators and mentors throughout my career, without whom this would not be possible.
Today we are open sourcing a new tool called VINE that visualizes deep neuroevolution populations learning over time. Project led by Rui Wang and Ken Stanley. We hope the community benefits from it! ubere.ng/2IxoFHF via
Introducing a new, totally different type of meta-learning: gradient-based Hebbian learning (stores info in weights, not activations). Congrats to ! With Differentiable Plasticity: A New Method for Learning to Learn ubere.ng/2qlFZHy
GIF
Agreed. I remember when I told journalists in 2016 AI would be able to generate any image you (or bad actors) want from a text description and they thought I was nuts. I also predicted videos and entire virtual worlds would come later, which they will! Strap in, society.
Slides now available for our ICML Tutorial on Population-Based Methods for Training Deep Neural Networks: Novelty Search, Quality Diversity, Open-Ended Search Algorithms, & Indirect Encoding. w & #icml2019 Video soon. cs.uwyo.edu/~jeffclune/sha
I greatly enjoyed giving a guest lecture in 's meta-learning class at Stanford. Thanks to Chelsea for the invitation & for making all of the lectures available for everyone! Mine is a deep dive into AI-GAs, GTNs, differentiable plasticity/backpropamine, ANML, & POET.
Quote
Want to learn about meta-learning? Lecture videos for CS330 are now online!
youtube.com/playlist?list=
Topics incl. MTL, few-shot learning, Bayesian meta-learning, lifelong learning, meta-RL & more:
cs330.stanford.edu
+ 3 guest lectures from Kate Rakelly, @svlevine, @jeffclune
Nice new work by DeepMind covered in Nature, with comments by yours truly. Congrats to the authors.
Momentum continues to build in Deep Neuroevolution. Another five-paper blog post, this time by Sentient. Congrats to Risto and all on the team!
Evolution is the New Deep Learning sentient.ai/blog/evolution
Deep Curiosity Search introduces a new type of intrinsic motivation for deep RL: intralife exploration. Just rewarding agents to 'go somewhere new' can dramatically increase performance, tying state of the art on Montezuma’s Revenge arxiv.org/abs/1806.00553 Congrats Chris Stanton!
Introducing: First-Explore, then Exploit: Meta-Learning Intelligent Exploration. Led by arxiv.org/abs/2307.02276
Humans are masters at exploring. Unlike RL, we do not explore by trying to maximize reward (with noise), but instead explore to gain information! 1/8
Update: Go-Explore remains state of the art on Montezuma’s & Pitfall when *tested* with sticky actions. Should we *require* stochastic training? Depends on the research objectives. Please read newly added section eng.uber.com/go-explore A healthy debate created by Go-Explore.
Creating a Zoo of Atari-Playing Agents to Catalyze the Understanding of Deep Reinforcement Learning. Great work led by Joel Lehman , w/ many excellent collaborators from Uber AI Labs, OpenAI, & Google Brain. Blog: eng.uber.com/atari-zoo-deep Code: github.com/uber-research/
VPT was accepted at NeurIPS! Congrats to the team for all the hard work, and a huge thanks to the reviewers and organizers! #NeurIPS2022
Quote
Introducing Video PreTraining (VPT): it learns complex behaviors by watching (pretraining on) vast amounts of online videos. On Minecraft, it produces the first AI capable of crafting diamond tools, which takes humans over 20 minutes (24,000 actions) 
openai.com/blog/vpt
openai.com/blog/vpt"API for accessing new AI models developed by OpenAI. Unlike most AI systems which are designed for one use-case, the API today provides a general-purpose “text in, text out” interface, allowing users to try it on virtually any English language task." openai.com/blog/openai-ap
Our paper was accepted ! Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents.Congrats to leads , Edoardo Conti, & the whole team
Introducing Fiber, a new open source platform for distributed machine learning, especially population-based methods like Enhanced POET. Program locally with a standard multiprocessing API then deploy to thousands of workers on any cluster. uber.github.io/fiber/introduc Led by
I often have ideas for startups, research papers, short stories, etc. I am going to start posting them here to share them, see what people think, and in the hopes of maybe inspiring someone to something related (or even implement the idea). I'll tag things with #someBodyDoThis
Just realized our Deep Neuroevolution paper is on the front page of Hacker News! news.ycombinator.com That's fun. Also the most talked about ML paper on Twitter this week according to 's arXiv-sanity. Thanks all for your interest in our work!
Excited to share OMNI: Open-endedness via Models of human Notions of Interestingness. Lead: arxiv.org/abs/2306.01711 Open-ended learning requires a vast space of possible tasks, but search thus can get lost. Agents must focus only on *interesting* tasks, but how?🧵1/
Don't aim for success if you want it; just do what you love and believe in, and it will come naturally. - David Frost
(via the excellent book Why Greatness Cannot be Planned by and ).
Accelerating Deep Neuroevolution: Train Atari in Hours on a Single Personal Computer! What took ~1 hour on 720 CPUs now takes only ~4 hours on a *single* modern desktop. Code is open source. Awesome work by with eng.uber.com/accelerated-ne via
Replying to
I will be hiring at both institutions, so please watch this space for details on those opportunities if interested. A huge thanks to everyone at both OpenAI and UBC for these wonderful opportunities! I could not be more excited about both! 2/2
I often have (what I think are) great ideas for startups (especially AI startups). Has someone created a way to suggest ideas to would-be entrepreneurs and then get a small bit of equity if they work on your idea? If not, is that itself a good idea (a site that does that)?
Here's the video of my ICML Continual Learning Workshop talk: "Learning to Continually Learn" introduces AI-GAs as a path to AGI and focuses on ANML as an example of that research paradigm that works well to minimize catastrophic forgetting slideslive.com/38930882/learn #ICML2020
Thank you very much.
Quote
In honor of @UberAILabs we wanted to share this quote from Jeff Clune (@jeffclune ). We hope all researchers & developers can find a good place to work & continue to research in this pandemic .
Your unique contributions to our field will always be remembered.
#uberlabs
How can I get rid of all the AI clickbait in my feed? It's mostly from blue checkmark folks. Is there a way to ban/ignore/filter anyone who has one? Are the rest of you inundated with annoying click-baity AI Tweets?
Another great example of AI outsmarting us. Here scientists took a robot that could solve tasks by picking up a box and then disabled its gripper to see if it could adapt to push the box around. It instead figured out how to pick the box up anyway! arxiv.org/abs/1803.03453
GIF
Fiber, an open source platform for distributed machine learning, esp population-based methods like Enhanced POET. Program locally with a standard multiprocessing API then deploy to thousands of workers on any cluster. eng.uber.com/fiberdistribut
We are beginning to see how powerful AI can catalyze science. In a future coming to you soon: ask the computer to conduct an experiment, interactively probe the results, and then ask for the next experiment, iterating quickly without writing a single line of code or pipetting!
Quote
GPT-3 Does The Work
on generating SVG charts, with a quick web app I built with @billyjeanbillyj. With a short sentence describing what you want to plot, its able to generate charts with titles, labels and legends from about a dozen primed examples.
cc @gdb
on generating SVG charts, with a quick web app I built with @billyjeanbillyj. With a short sentence describing what you want to plot, its able to generate charts with titles, labels and legends from about a dozen primed examples.
cc @gdb
Introducing a new algorithm that automatically, intelligently adjusts exploration vs. exploitation for #DeepRL. NSRA-ES exploits until stuck, then increasingly explores. It's newly added to 'Improving Exploration in ES' Great work on it Vashisht Madhavan!
Nice new article on novelty search, QD, open-ended algos, and AI-generating algorithms, including connections to Go-Explore, AlphaStar, IT&E, & more. Thanks for writing it and to , , & for nice comments. quantamagazine.org/computers-evol
What is starting to happen to me during quarantine because of no haircuts.
Quote
This sheep escaped a farm and spent 6 years in the mountains, during which time he grew 60 pounds of wool. Wolves tried to eat him, but their teeth could not penetrate the floof. You don't have to turn hard to survive the wolves, just be really, really soft and fluffy.
Great article about AI-generating Algorithms ("getting AI to make itself"), including POET, GTNs, & ANML Also includes work & quotes from (POET), Esteban Real (AutoML), & (LRL), plus great work by Thanks for the fun interview & article !
Quote
Artificial intelligence might not need us to become smarter than we are. This thread is about how AI is learning how to create itself. technologyreview.com/2021/05/27/102
Nice article in Nature about how deep neural networks are easily fooled, covering our work over the years (w & ) and great work by many others (eg , , ). nature.com/articles/d4158 Plus an interesting discussion on future AI.
Generative Teaching Networks (GTNs) are on the homepage of Hacker News right now: news.ycombinator.com/news I wonder why now? Blog post (from December): eng.uber.com/generative-tea
Very interesting new work on open-endedness and AI-Generating Algorithms from et al. @DeepMind, with special emphasis on Pillar 3 (automatically generating environments, like in POET). Congrats. I look forward to reading it in detail!
Quote
Very excited to release our new work: Open-Ended Learning Leads to Generally Capable Agents. tldr; algorithm that dynamically shapes task distributions to train agents on huge task space, resulting in surprisingly general behaviour dpmd.ai/open-ended-blog
Thread: (1/n)
We are deeply honored to receive the Outstanding Paper of the Decade award from ISAL for The Evolutionary Origins of Modularity (Proceedings Royal Society 2013). The work was co-led by & with . Thanks Alife community! royalsocietypublishing.org/doi/full/10.10
A friend of mine has cancer. I generated AI images of her as a superhero fighting cancer and she loved them. She asked if I could produce a 3D-printed version. Anyone know a talented 3D-blender-type artist that also knows how to 3D print their creations that I could hire?
Being an AI researcher as adult means reliving the video game part of my childhood in order. First Atari, then Super Mario, now streetfighter. Looking forward to Goldeneye next!
Quote
The MAME RL Algorithm Training Toolkit: A Python library that can provide a Gym-like API around almost any old arcade game. They show how to set up new ROMS, and provide RL example for “Street Fighter III Third Strike: Fight for the Future (Japan version)” github.com/M-J-Murray/MAM
GIF
Excited to give a talk at Stanford today to 's meta-learning and multi-task learning class (& open to all at Stanford). Thanks Chelsea for the invitation."How Meta-Learning Could Help Us Accomplish Our Grandest AI Ambitions, and Early, Exotic Steps in that Direction"
Source code is now available for Generative Teaching Networks eng.uber.com/generative-tea Code github.com/uber-research/ Courtesy of 's amazing research & engineering. We hope this helps the community. Let us know if you use it! w/
Open source code now available for ANML github.com/uvm-neurobotic From Learning to Continually Learn (Beaulieu et al. ECAI 2020). ANML meta-learns how to learn without catastrophic forgetting, including up to 600 sequential tasks. Talk: videos.re-work.co/videos/1861-an
Secretly our Enhanced POET project was an attempt to move closer to making this picture from 's ES blog post a reality. We use the same algorithm, and now have the cliff jumping! Next up: automating the generation of environments with parachutes! blog.otoro.net/2017/10/29/vis
Watch this if you want to understand how the world will change. :-)
I wish arXiv put affiliations on the html page for a paper. Often that's the only reason I download the PDF, as I am curious where the work came from. I wonder how much bandwidth they'd save by doing that.
Dear people who make utilities (e.g. websites, apps) that are free and helpful: thank you.
Cool to see our evolved virtual creatures highlighted in Nature Machine Intelligence! Thanks to for the kind words in the article. Work with Nick Cheney, Rob MacCurdy, and Hod Lipson.
Quote
Tired of training #NeuralNetworks?
Try optimizing virtual creatures instead.
@nature Machine Intelligence article by Sam Kriegman (@Kriegmerica).
rdcu.be/bTN0S
Deep learning on the cover of PNAS!! Our paper on automatically classifying, counting, and describing the behavior of animals in camera trap images was selected for the cover of the Proceedings of the National Academy of Sciences. pnas.org/content/115/25 We hope it helps team ML
Replying to
To be fair, I'm moving either way. But how often do you get to say this and really mean it? ;-)
Quote
I am extremely excited to announce (1) I've joined OpenAI to lead a large-scale effort into AI-generating Algorithms research, & (2) I'll be an Associate CS Professor at U. British Columbia in 2021, where I will continue to lead the OpenAI project. Both are dreams come true! 1/2
Slides available for my talk: "Endlessly Generating Increasingly Complex and Diverse Learning Environments and their Solutions through the Paired Open-Ended Trailblazer (POET)" jeffclune.com/publications/2 Led by with & Video soon
Seeking a postdoc to join my lab at UBC! Interested in combining deep RL & large language models to advance open-endedness? Candidates should have pubs in one of: RL, large models, open-endedness, or related areas. 1-2+-year positions in paradise! Apply: jeffclune.com/join.html
I am delighted to announce that I have been appointed as Canada CIFAR AI Chair. Thanks to everyone at CIFAR and the Vector Institute. Congrats to the other appointees, including
Quote
Happy to announce the appointments of two new Canada CIFAR AI Chairs from UBC Computer Science: Dr. Jeff Clune and Dr. Vered Shwartz
Read about their research: ow.ly/7oCb50Ln3YF
@jeffclune @VeredShwartz @CIFAR_News @VectorInst
#AI #NLP #UBC #CIFAR
Replying to
Why leave? I love Uber AI Labs and it has been a dream opportunity, but another opportunity was offered to me that was too amazing to pass up. It also works better for my family. I will announce my future plans Monday, but for today I just wanted to thank my friends at Uber. 3/3
My MIT talk is online. Thanks to Josh Tenenbaum for the nice introduction. "Improving Deep RL via Quality Diversity, Open-Ended, and AI-Generating Algorithms" (also covers VPT and how that fits into the AI-GA paradigm). Thanks for organizing!
My keynote at CORL is now online. "Improving Robot and Deep Reinforcement Learning via Quality Diversity, Open-Ended, and AI-Generating Algorithms." Thanks again to the organizers for the invitation!
In line with AI-Generating Algorithms, we should stop trying to hand-design the solutions to ML problems. Instead, with the right problems and enough data/compute, ML will do the heavy lifting and solve the problems for us (and do a better job). arxiv.org/abs/1905.10985
Quote
People have spent a lot of time trying to encode better inductive biases into vision models. This work suggests that such effort may be misdirected. An off-the-shelf Transformer pretrained on a large unlabeled dataset can do just as well at classifying CIFAR-10 as the best CNNs. twitter.com/OpenAI/status/…
Our Generative Teaching Networks paper was accepted to ICML! blog: eng.uber.com/generative-tea paper: arxiv.org/abs/1912.07768 GTNs speed up architecture search via meta-learning, & touch all of AI-GA's Three Pillars Led by w
I've been worried about AI Existential Risk for over ten years.Some of my current views are in this nice New Yorker article by It's great to see the topic go from being ridiculed to seriously discussed by society, but the hard work remains 1/
I am delighted that Ken Stanley is joining us at and that we thus will be able to continue to work together so closely!!! Welcome Ken!
Quote
I'm thrilled to announce that I will be joining the superb team at @OpenAI in June, where I will be starting a group (and indeed hiring) focused on achieving open-endedness in machine learning. Looking forward to exploring a novel path!