Did you know you can use emojis and unicode characters as prompts in Midjourney? While working on a project I accidentally entered a single-character prompt. I was expecting an error, what I got was a collection of strange and surreal images. I was curious. So I decided to go deep into the rabbit hole.
If you’re not quite sure what this post is about, know that Midjourney is an AI-powered image generator. It is the same generator I used to illustrate a children’s book back in December. You provide text and images to explain what you want it to create, and it does the rest. The default pattern in Midjourney gives you four images for a single prompt. That’s why below you’ll see lots of grids with four images in them.
The ABCs of Midjourney
I had expected that when I entered a single A into Midjourney, I would get a bunch of stylized character A’s. That doesn’t appear to be what happens though. Instead Midjourney seems to rely on its default 5.2 artistic style and the seed to determine what to draw.
The results are images with wild and varied concepts. The letter must be included in some way though. Perhaps, just weighted less? As you can see letters D, M and Z are included in at least one of the four results Midjourney returns.
Unicode and other characters
I tried several prompts that included non-letter characters. The results largely appeared random other than the one with 5 where one of the images included a 5 in the illustration.
These appear to behave like the letters above. The output relates little to the input and the final results are generated by Midjourney 5.2 styles and the random seed.
Emoji prompts
By far the most interesting results were generated by using single emojis in the prompts. While individual characters seemed to have little influence on the output, most emojis clearly influenced the results.
If I said this post was about single-character prompts—I think I’d be cheating here. Even though an emoji looks like one character, I think the word for that emoji is probably what is being used to drive the prompt. So instead of 🌈, 💩, 🎢 Midjourney sees rainbow, poop, rollercoaster. Regardless I’m leaving these. They produced a ton of gems.
Something interesting—virtually all the portraits generated here are women, while almost all the background characters are men—usually with their backs to the camera. The only exception in the 200 or so outputs here is that two of the four images for flag of the united states have portraits of men. The other two from that prompt are a portrait of a woman with floating Christmas tree ornaments behind her and a steampunk-looking battleship with American flags crushing a small town.
Also, I love the creepy characters generated by 💩 and the horrific floating city faces 😀 produced.
Combining emojis
The next step, of course, was to start combining emoji prompts.
I started to create common words like pineapple pizza 🍍🍕, coffee house ☕️ 🏠 and cookie monster 🍪 👹 using emoji pairs.—those actually worked pretty good. Others like ring bearer 💍 🐻, produced interesting, but nonsensical results.
Update: How is Midjourney interpreting emojis?
After some more reflection, I’m not entirely sure that the text version of the emoji is the only thing being passed to Midjourney. Some examples are extremely close. For example, rainbow and 🌈 produced similar enough results that I figured the text must be being passed in some way or another.
But that isn’t true for others. For example, grinning produces wildly different results than 😀. But why?
I thought we could test this. We can generate an image with the 😀 emoji and then take the seed from those images. Once we have the seed we can apply that seed to various text prompts with the --seed parameter. They should produce very similar results.
But that’s not what happens. The results are equally creepy, but completely different than what I’d expect the output to be when using the same seed.
I tried using grinning face, which I believe is the full name of that emoji. I also tried entering colons around the word, which is how discord allows users to quickly add emojis without leaving the keyboard. Nothing resembles the image created by the emoji.
What about image prompts? Could Midjourney be using the image and name of the emoji to create results?
Discord uses Twitter Twemoji emoji library, so I downloaded the Grinning emoji and created a few image prompts using the same seed as above.
I tried a few examples that added various weights to the emoji, but I still didn’t get anything close. The heaviest weight looked like it had a pixelated style, so I uploaded a larger version of the Twemoji. I got results with more detailed faces, but nothing close to the giant floating heads and houses that 😀 produced.
If anyone has any idea about how Midjourney interprets emoji’s I’d love some insight. Please leave a comment!