Paul GrahamVerified account

I am a big fan of @OpenAI’s research. It is often very original in ways that more traditional research labs, like my own team, tend to ignore. While #gpt3 doesn’t bring any algorithmic innovation, the zero-to-few shot approach as a universal language API is groundbreaking. 2/13

I do take exception with some of @OpenAI’s PR though. In particular, I don’t understand how we went from #gpt2 being too big a threat to humanity to be released openly to #gpt3 being ready to tweet, support customers or execute shell commands (https://beta.openai.com ). 3/13

Instead I wish @OpenAI had been more open and less sensationalistic, by just open sourcing both for research, especially on #responsibleAI aspects, while acknowledging that neither was ready for production and discouraging services like https://thoughts.sushant-kumar.com/  4/13

One criticism I got was that I cherry picked my examples. Ignoring the fact that 100% of examples touting #gpt3 on Twitter are cherry picked, greatly inflating its perceived performance, cherry picking is a valid approach when highlighting harmful outputs. 5/13

This is a challenge with our current AI benchmarks which do not properly weigh harmful outputs. Even one very bad output in a million in a prod app (eg customer service) can be unacceptable, as shown by the deserved backlash my team got for bad machine translations on FB. 6/13

In this case, it just took a handful of tries to generate toxic #gpt3 outputs from neutral, not even adversarial, inputs. AI algorithms need to be a lot more robust to be productized. The ease of generating these toxic outputs is what prompted my decision to share them. 7/13

Another criticism was that #gpt3 was just reiterating what humans think. Yes AI algorithms do learn from humans but a deliberate choice can be made about which humans they learn from and which voices are amplified. 8/13

Just culling any data from the web or Reddit because it’s available is not a responsible training strategy. It will lead to amplifying unchecked biases, some very harmful. And we need objective functions that discourage toxic speech in the same way we do it in real life. 9/13