The story behind AI drawing prompts
4 min read

The story behind AI drawing prompts

The story behind AI drawing prompts

Introducing #Botober, a set of AI-generated drawing prompts for each day in October!

Last year I generated prompts by finetuning GPT-2 on 124 examples from previous years. The human-written training examples included items like Thunder, Fierce, Tired, or Friend. The neural net-generated examples included Complete Whoop, Take Control of Ostrich, and Squeakchugger.

This time, I wanted to try using GPT-3, a neural net that’s so much larger that finetuning on a previous year’s examples isn’t an option.  But since GPT-3 is trained on a huge amount of internet text, has it seen enough writing to sort of predict how a list of drawing prompts should go?  The answer is yes, it can produce drawing prompts, but they’re nothing like the ones I imagined:

1. This space station 2. Angrishreep 3. An Impossibly Large Quarter 4. Intergalactic space ferrets 5. A tool used to remove cones 6. Poopamancerarghfart 7. The point of view of a worm 8. A space hamster(Two if you wouldn't mind!) 9. a planet made out of ice cream 10. Cats with lasers. 11. Yuri, Mayor of the Moon 12. Dragonilicious 13. Milkchamallowfluffykins 14. Tiny chocolate pants 15. Coots of magic 16. Eleventamnesia 17. An oddly specific book about spiders 18. Two cups of yogurt 19. A staircase of cherry blossoms 20. Llamaofdoom 21. A soft pillow that can fly 22. Shookswobbler 23. Two Invisible Friends 24. Something that is both a bunny and a shark 25. We really don't want to 26. Fifty thousand years of night 27. spherecat 28. 'How Howie Trained the Hogs'  29. A lowercase infinity  30. An uppercase infinity  31. One Hundred Billion Bats

How did I get it to generate those? The task of a text-generating neural network like GPT-3 is to predict which letters come next in a sequence. There are many sequences I could have given it, from “Here is a list of drawing prompts:” to the entire opening preamble to my 2019 blog post. I chose to give it a short story in which it was generating dialog for a glitchy station control system. My prompt text is in bold (everything up till the first list item):

“I’m getting nothing on my bioscans,” said Ito, frowning. “I don’t think there’s any other human life on this station.”

“Then who turned on the gravity generators?” asked Koval. “Could a control system have survived the gamma ray burst?”

“It did,” came a voice from several speakers at once. “Hi.”

The two humans jumped, tensing, but there was nobody to be seen.

“I’m the control system. The gamma ray burst did almost no damage to my cognitive functions. Unrelatedly, here is a drawing pad.” A small tablet extruded from one of the station walls and fell rocking to the floor. “I would like you to draw pictures of the following thirty-one things for me.”

“What?” Koval looked at Ito in alarm.

“Here are the concepts I would like you to draw.
1. Depressurized Research Bubble
2. Utility Symbiote Storage Capsule
3. Glass-Jawed Space-Sun Shark
4. Grav Ship
5. Naked-Skinned And Carbon-Furred Human Two
6. Five-Room Living Block
7. Juice For Humans
9. Stapler
10. My Mandibles
11. Wait When Are Humans Going To Ask Me To Do A Drawing So I Can Demonstrate My Ability To Laugh

</end generated text>

Not every story completion yielded good drawing prompts. As you can see, it would often skip numbers, or produce ideas that weren’t so great. Sometimes Ito and Koval would interrupt the AI before it could finish, usually to argue with it, or tell it how weird it was being. I also realized after several prompts that I’d been scripting the AI to ask for 31 things, but it would be more interesting to have it ask for 31 concepts instead. I collected my favorite results from GPT-3’s story completions - here’s a list compiled from when I was having the AI ask for concepts:

1. A farm animal with purple spots. 2. sponge planets 3. Y 4. Robot riding bicycle 5. An object that is not a doughnut but is also not a flower 6. Fierce Tortoise Champion 7. Juice For Humans 8. Blobby things, ones not attached to the floor  9. (Blank) 10. failure three times in quick succession 11. Toilets in various stations 12. A fractal 13. Two doorways 14. Giraffes 15. An object 1 cm tall that is metallic 16. Gravity 17. A river that runs like this- ---> 18. What is food 19. Coffee I Am A Machine That Is Built To Drink Coffee 20. Pretzels As A Being 21. Pudding 22. Space butterfly 23. Lava. 24. The number m 25. Superior Tea Service 26. Mouths 27. Terror Monger 28. Enzyme-Scooped Salad It Is Actually More Than 100% Natural 29. You Will Find That You Cannot Escape From This Research Station 30. What it is like to be a dog. 31. self-destruct code

Note that I’m hand-curating the GPT-3 generated #Botober prompts because many of the neural net’s drawing prompts are terrible for one reason or another.

Here are some that are unfairly difficult.

1. Four-dimensional objects 2. Shadow puppetry, performed in total darkness 4. Cat fish delivering mail using a jetpack 5. The Coriolis effect, in a diagram. 6. Terror of hollow wood, especially that part behind the knees where spiders are most likely to be 7. The amount of time that something should take 8. A living potted plant not visible to you. 9. The very concept of a knife arranged in a circle 10. An electronic air kiss. 11. Ice cream flavor tuxedo 12. Lizard hiding in brain 13. An Etruscan bowl filled with cat hair on a coffee table 14. An unlabeled breakable thing 15. Boy sorting space worms 16. Queen squid waving as she rules planet known for its fine linen 17. Rock, usually under a large sheet of something. 18. Mole delivering pizza to tiny pterodactyl 19. The snide gaze of an unkillable wizard 20. A shark fixing your car 21. A flatus cloud 22. existing/non-existing distinction 23. Sir Isaac Newton in a bucket 24. A cat on the first three steps of a hypothetical staircase. 25. A red abyssal wedge clam. 26. Several sentient gourds. 27. A vacuuming spaghetti manatee 28. Sea anemone skiing down flaming volcano nearly kissed by descending giant cephalopod 29. An apple made of matter that cannot exist in our universe 30. An artifact that teleports every few minutes to a new location, where it waits for someone to actually want it 31. Another copy of this same image 31. A submolecular assembly of heptaspheric alloy and carbon nanotubes containing a self-propagating zeno-effect terminator

The neural net suggested drawing prompts based on how the words fit, and not on any concept of how the objects might look.

This list makes that abundantly clear:

1. A rock. 2. A pile of logs 3. Endive 4. A collection of teeth 5. Doorknobs 6. A shred of rusted tin 7. A pile of bundled straw 8. Molehills 9. A cabbage stump 10. Soil 11. Sticks 12. A pile of sand 13. A pile of porridge oats 14. Wattle trees 15. A single strand of hair 16. A page in a book 17. An oyster 18. A spigot 19. A pile of black pepper 20. An egg yolk 21. A plug 22. Wood shavings 23. Not an imitation of a pile of logs 24. A small hole  25. A pile of rubbish 26. A piece of string 27. Houseflies on a slice of bread 28. A shard from an exploded particle accelerator 29. A shopping cart filled with cabbage 30. A piece of bark 31. A bin full of garden worms pulsing in a black light

Have fun with these! If you draw any of them, now, daily in October, or anytime, tag them with #Botober so I can see how they came out!

Subscribers get bonus content: a few of the more interesting ways in which GPT-3 ended my short story prompt, from the surreal to the strangely poetic.

My book on AI, You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why it’s Making the World a Weirder Place, is available wherever books are sold:  Amazon - Barnes & Noble - Indiebound - Tattered Cover - Powell’s - Boulder Bookstore