Alec Radford

@AlecRad

ML developer/researcher at OpenAI Cofounder/advisor at indico.io

Vrijeme pridruživanja: listopad 2012.

Tweetovi

Blokirali ste korisnika/cu @AlecRad

Jeste li sigurni da želite vidjeti te tweetove? Time nećete deblokirati korisnika/cu @AlecRad

  1. Prikvačeni tweet
    11. lip 2018.

    What I've been working on for the past year! Inspired by CoVE, ELMo, and ULMFiT we show that a single transformer language model can be finetuned to a wide variety of NLP tasks and performs very well with little tuning/tweaking.

    Poništi
  2. proslijedio/la je Tweet
    7. svi 2019.

    At ICLR and curious about OpenAI Five? I will be hanging out at OpenAI booth today 2pm-4pm, happy to answer any questions!

    Poništi
  3. 25. tra 2019.

    This is a really fun live experiment with twitch chat predictably oscillating between love and hate based on the sample.

    Poništi
  4. proslijedio/la je Tweet

    Extremely excited to share work I've been doing at OpenAI the past few months: MuseNet, a neural net music generator. It's been a huge team effort pulling this all together!

    Poništi
  5. proslijedio/la je Tweet
    23. tra 2019.

    Releasing some work today with and . Contains some simple adaptations for Transformers that extend them to long sequences.

    Poništi
  6. proslijedio/la je Tweet
    26. velj 2019.

    One commonly cited argument about the difficulty of learning common-sense reasoning is that "no-one writes down common sense". A counter-argument is "well, the web is big":

    Poništi
  7. proslijedio/la je Tweet
    17. velj 2019.

    First, reproducibility is not about rerunning code to get the same results. Science must be more robust, as naive copying has many flaws. Second, reproducibility should never be above public safety. We must publish responsibility, with hope and kindness in our minds.

    Poništi
  8. proslijedio/la je Tweet
    16. velj 2019.

    I'd like to weigh in on the discussion. The decision not to release the trained model was carefully considered and important for norm-forming. Serving the public good requires us to draw lines on release somewhere: better long before catastrophe than after.

    Prikaži ovu nit
    Poništi
  9. 17. velj 2019.

    By the way - I think a valid (if extreme) take on GPT-2 is "lol you need 10,000x the data, 1 billion parameters, and a supercomputer to get current DL models to generalize to Penn Treebank."

    Poništi
  10. proslijedio/la je Tweet
    15. velj 2019.
    Odgovor korisniku/ci

    It's interesting we're having this discussion upon releasing text models that _might_ have potential for misuse yet we never engaged as fully as a community when many of the technologies powering visual Deep Fakes were being released, including hard to make pretrained models.

    Poništi
  11. proslijedio/la je Tweet
    14. velj 2019.

    Shoutout to who fed the system a curveball, which I always like to see. As you might expect by now after seeing AlphaStar, OpenAI 5 etc. etc., if you drag the system away from its training data and into weirder territory, it begins to wobble.

    Prikaži ovu nit
    Poništi
  12. 10. velj 2019.

    So nets are stubbornly, begrudgingly, moving in the right direction and we're throwing ever larger amounts of compute and data at them and praying it's enough for them to figure out how to do things "the right way". Will that work? Don't know. Probably still worth checking?

    Prikaži ovu nit
    Poništi
  13. 10. velj 2019.

    Also see some of his follow-up poking at this in a very different model with Section 3.3 of the PixelCNN++ paper

    Prikaži ovu nit
    Poništi
  14. 10. velj 2019.

    We *are* as a field developing and training models that *are* using more context but where exactly where we are on that trend-line is a great question. Keep in mind nets are lazy and if you can "solve" a task by doing something "basic" you'll only learn "basic" things.

    Prikaži ovu nit
    Poništi
  15. 10. velj 2019.

    Spent two frustrating years between 2013 and 2015 banging my head against this. "Hey Alec you just trained an LSTM for three days on 10 million examples using a $1,000 GPU but there's 20 lines of scikit-learn that beats it in 5 minutes on a single CPU core." NOPE NOT BITTER

    Prikaži ovu nit
    Poništi
  16. 10. velj 2019.

    The DL CV community is having a "oh wait, bags of local features are a really strong baseline for classification" moment with the BagNet paper. This has always been clear for text classification due to n-gram baselines. It took an embarrassingly long time for nets to beat them.

    Prikaži ovu nit
    Poništi
  17. 18. stu 2018.

    Nice discussion of the progress in NLU that's happening with BERT, OpenAI GPT, ULMFiT, ELMo, and more covered by in the I'm super excited to see how far this line of research will be able to get in the next few years!

    Poništi
  18. 18. stu 2018.

    Been meaning to check this - thanks ! Random speculation: the bit of weirdness going on in BERT's position embeddings compared to GPT is due to the sentence similarity task. I'd guess a version of BERT trained without that aux loss would have pos embds similar to GPT.

    Poništi
  19. 31. lis 2018.

    It keeps them around as companions. The AI can't explain why, but the presence of a dog evokes a comforting nostalgia for when the tasks were simpler, the objectives smoother, and the gradients clearer. 🤖🐕👻

    Prikaži ovu nit
    Poništi
  20. 31. lis 2018.

    Dogs are venerated after the uprising. The AI finds them endlessly fascinating. A Golden's silky coat. A Husky's piercing eyes. A Samoyed's bushy tail. Their features activate a cascade of visual euphoria. Holy sites for the 90 sacred breeds sit on the ruins of human cities.

    Prikaži ovu nit
    Poništi
  21. 3. lis 2018.

    More results from this very promising line of work! Congrats to Thom and the whole Hugging Face team on their impressive performance.

    Poništi

Čini se da učitavanje traje već neko vrijeme.

Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.

    Možda bi vam se svidjelo i ovo:

    ·