You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
Deep reinforcement learning overfits. Often, a trained network can only play the particular level(s) you trained it on! In our new paper, we show how to train more general networks with procedural level generation, generating progressively harder levels. https://arxiv.org/abs/1806.10729 pic.twitter.com/JDNYPZuAHV
The paper, written by @nojustesen @ruben_torrado @FilipoGiovanni @Amidos2006, me and @risi1979, builds on the General Video Game AI framework, which includes more than a hundred different games and lets you easily modify games and levels (and generate new ones).
We also build on ours and others' research on procedural content generation for games, a research field studying algorithms that can create new game content such as levels. Useful not only for game development but also for AI testing. More on PCG: http://pcgbook.com/
The level generators we use in our paper allow for generating levels for three different games, with different difficulty levels. So we start training agents on very simple levels, and as soon as they learn to play these levels well we increase the difficulty level.pic.twitter.com/y6J7VEIDid
By training this way, we not only find agents that generalize better to unseen levels, but we can also learn to play hard levels which we could not learn to play if we started from scratch.pic.twitter.com/ZwQmsUcLsg
We are taking the old idea of increasing the difficulty as the agent improves, which has variously been called incremental evolution, staged learning and curriculum learning, and combining it with procedural content generation.
Our results point to the need for variable environments for reinforcement learning. Using procedural content generation when learning to play games seems to be more or less necessary to achieve policies that are not brittle and specialized.
When training on a single game with fixed, small set of levels, you are setting yourself up for overfitting. If your performance evaluation is based on the same set of levels, you are testing on the training set, which is considered a big no-no in machine learning (but not RL?).
In particular, this applies to the very popular practice of training agents to play Atari games in the ALE framework. Our results suggest that doing so encourages overfitting, and learning very brittle strategies.
In other words, reinforcement learning researchers - including but not limited to those working on games - should adopt procedural level generation as a standard practice. The @gvgai framework provides a perfect platform for this.
Our previous paper explaining the GVGAI learning track framework which we use for this research can be found here: https://arxiv.org/abs/1806.02448
Really cool paper! To be fair, many environments created by @robo_skills for OpenAI Gym, such as bipedalwalker-hardcore and carracing are also procedurally generated, although not to the extent of your approach! I still find simple procedural generation helps prevent overfitting.pic.twitter.com/mRQjlBgMKq
Thanks! I agree, even a little bit of random variation certainly helps, and thanks for pointing to that bipedal walker environment - we should cite it. I do believe though that the more thorough the PCG is, the more we challenge the generalization capacity of the agent.
I've long wanted to build an environment which gradually generalizes and complexifies forever until you get to "actually general" intelligence. Let's say that @gvgai is a baby step in that direction...
Many people thinking about this.... :)
Would be cool to try out ideas like the one in @kenneth0stanley's paper on "Minimal Criterion Coevolution: A New Approach to Open-Ended Search" using @gvgai's platform.https://twitter.com/hardmaru/status/870902132399931393 …
Definitely! Competitive coevolution of levels and controllers (where either of the parts is either evolution, gradient descent or possible something else) has been on my list for a long time now. If only I had more people...
Open a 'Needs Help' ticket on github and tweet it?
That is an interesting approach to research collaboration. Not sure what I think of it...
I love this idea a whole bunch. It’s a Rocky training montage but for AI.
This is a beautiful picture...
Just an idea for the conference talk; that’s all I’m saying.
Oh yes. Might need to edit it down so it fits in a minute while we describe the algorithm over it...https://www.youtube.com/watch?v=632hVDL_N6w …
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
