Conversation

This does make you wonder how the ratio of meaningful data that’s human generated vs. machine generated has trended over the years Eventually the complaint may be that the human generated data is polluting the system

Quote Tweet

Andrej Karpathy

@karpathy

· Jul 19, 2020

By posting GPT generated text we’re polluting the data for its future versions

3:08 AM · Jul 19, 2020Twitter for iPhone

Replying to

and yet these AI models still cant finish famous movie quotes and song lyrics

Replying to

We're also just recognizing that it's not as open a loop as we'd like to think. Our minds have been hacked for a long time, just increasingly efficiently.

Replying to

Question - After 17 years working in a computer environment the nonsense is creating data because we can, not because it is useful. If AI is intended to mimic human thought processes, wouldn't that include all the nonsense that is human thought as well?

Dale

@DBSLarsson

Jul 19, 2020

AI perpetuating the Flat Earth thought.. 🤢

Replying to

I used to think untagged data was pollution but this seems to turn that into an advantage Latent spaces are almost impossible to gain intuition on, but do you think this data mixing will reinforce local maxima and lock in falsehoods?

Replying to

weighing by cultural impact could indeed lead to some disappointment, a machine trained on Friends + 50 shades of grey + Blurred Lines + the Twitter account of him who shall remain nameless would be a venerable abomination

Replying to

machines at this stage are like out different kinda dogs yet they're not; you tell the dog to bite a stranger that's cool for some people yet dogs can't get your gun when you throw it in the park.