DALLE-2 has a secret language.
"Apoploe vesrreaitais" means birds.
"Contarra ccetnxniams luryca tanniounons" means bugs or pests.
The prompt: "Apoploe vesrreaitais eating Contarra ccetnxniams luryca tanniounons" gives images of birds eating bugs.
A thread (1/n)🧵
Conversation
Replying to
A known limitation of DALLE-2 is that it struggles with text. For example, the prompt: "Two farmers talking about vegetables, with subtitles" gives an image that appears to have gibberish text on it.
However, the text is not as random as it initially appears... (2/n)
28
279
1,503
We feed the text "Vicootes" from the previous image to DALLE-2. Surprisingly, we get (dishes with) vegetables! We then feed the words: "Apoploe vesrreaitars" and we get birds. It seems that the farmers are talking about birds, messing with their vegetables! (3/n)
13
111
1,474
Another example: "Two whales talking about food, with subtitles". We get an image with the text "Wa ch zod rea" written on it. Apparently, the whales are actually talking about their food in the DALLE-2 language. (4/n)
15
298
1,831
Some words from the DALLE-2 language can be learned and used to create absurd prompts. For example, "painting of Apoploe vesrreaitais" gives a painting of a bird. "Apoploe vesrreaitais" means to the model "something that flies" and can be used across diverse styles. (5/n)
8
97
1,126
The discovery of the DALLE-2 language creates many interesting security and interpretability challenges.
Currently, NLP systems filter text prompts that violate the policy rules. Gibberish prompts may be used to bypass these filters. (6/n)
14
117
1,519
We wrote a small paper with summarizing our findings.
Please find the paper here: giannisdaras.github.io/publications/D
Arxiv version coming soon.
(7/n, n=7).
20
70
1,226
Based on valid comments, we updated our paper with a discussion on Limitations and changed the title to Discovering the Hidden Vocabulary of DALLE-2. Thanks to and others for useful comments.
5
18
414
Responses to some of the criticism can be found here:
Quote Tweet
An update on the hidden vocabulary of DALLE-2.
While a lot of the feedback we received was constructive, some of the comments need to be addressed.
A thread, with some new gibberish text and some discussion 
(1/N)
(1/N)Show this thread
4
7
107
Replying to
For those wondering, it doesn't look like Midjourney speaks that language, but I would be curious to see if Imagen etc, do.
Apoploe vesrreaitais
15
10
208
Replying to
I see some consistency in the generated outputs. It seems to me entirely possible that Midjourney has its' own vocabulary - a set of words that seem random to humans but are consistently mapped to visual concepts. Let us know if you find any!
1
1
17
Show replies