This week Midjourney's team announced the moving of Midjourney AI to V4 as its default version. Two points: V3 is still available, of course, and V4 is still in Alpha. Aren't you excited and slightly scared of what it will be when it's out of Beta? O___o
Anyways, V4 is here to stay, and it's beautiful! Its knowledge of artists' styles is outstanding. Still, it can be challenging. And among other issues (hands are improving, by the way!), there is one with how Midjourney V4 applies style modifiers to different prompts.
STYLE MODIFIERS VS. PROMPTS
What are the most common ways to apply a style modifier to a prompt? I've seen people add an artist's name separated by a period. Others use artwork by or in style of. I myself use by all the time. It's minimal, general, and works! Well, not always—like many others, too. But after many fights, I seem to have found my ultimate prompt strategy.
Today we'll talk about how to add the great Director of Photography Roger Deakins' name to a prompt out of his "artistic scope," and take his cinematic style...
In other words, here are some thoughts and observations on how to direct, nudge and whisper Midjourney to make any style work with virtually any prompt.
TL:DR
Spoiler: ARTISTS LOVE PERSONIFICATION! :) The recipe is incredibly simple—to personalize the prompt, so it becomes more about the style that we are using. Let's start with a benchmark.
NAZGULS AND PEONIES
As I mentioned in the post about Stylize, we use a benchmark for our daily Instagram posts. We test styles against a set of nine standardized prompts to which we add the artist's name or technique. The test lets us see how a particular style modifier works—on a pre-set scale.
Some of the styles we spotlight work well with most prompts, but not all of them. Some names will return excellent results with 6-7 prompts out of 9. But some prompts are more challenging than others! In this research, I will use the toughest four to illustrate my strategy.
And, for some reason, on the top of that list is the cutest of them all!
CUTE MAINECOON CAT
When a style fails, the test will return a generic picture of a cute Mainecoon cat—not even near the original style. Here are three cute felines to show how it happens.
To emphasise the dramatic difference, here are the initial Midjourney styles of our artists.
Let's modify the initial prompt to make it blend better with the target styles. Erwin Wurm is a sculptor and installation artist, Larry Poons is an abstract painter, and Peter Saville is a graphic designer. So...
Can you get away with artwork by? Yes, but there is a decent chance of missing.
Let's test it with the name of the Mexican painter José Clemente Orozco known for his bold, social realist style and his depiction of the struggles of the working class. His initial cute Mainecoon cat was nothing like Orozco's style.
And this is what happened when I tried artwork by against the "artwork by + depicting" formula against an artist-specific prompt.
FUN FACTS ABOUT MAINECOONS
The correct spelling is Maine Coon, but "coon" is banned in Midjourney. So we have to use Mainecoon instead. (／_눈)
cute Mainecoon cat is difficult for style modifiers with by prompt. However, removing the cat breed and leaving just cute cat will make the prompt more "penetrable" for various styles. And leaving only cat will improve the situation even more!
Knowing this, can we start with a simple prompt and then remix the cuteMainecoon cat back in? I tested it with the style of the legendary Soviet movie director Sergei Parajanov. The baseline prompt cat was very cute, but didn't even remind the maitre's Midjourney style.
A simple cat prompt, on the contrary, brought magnificent result! But then remixing it...
cat by Sergei Parajanov --v 4 →
→ cute Mainecoon cat by Sergei Parajanov --v 4
However, what worked much better was the artist-specific prompt strategy.
...we-ell, it kinda worked. ┐(ﾟ～ﾟ )┌
CYBERPUNK CHARACTER
Another challenging test prompt from our benchmark is cyberpunk character.
For many artists, generations with this prompt will be non-specific cyberpunk pictures, only slightly inheriting from the initial style. It's difficult to believe that these three images were generated with different artists in prompts.
Quite the gap with the initial styles, right?
Let's fix that. Carlo Crivelli was an Italian Renaissance painter star. William Morris of Great Britain was, among other things, a famous textile designer. And Dutch Anton Corbijn is a living classic of portrait photography, especially recognized for his bold and contrasty black-and-white style.
Et voila!
Much better.
Meet another Mexican painter—grand master Rufino Tamayo, acclaimed for his abstract, colorful paintings inspired by pre-Columbian and Mexican folk art. His cyberpunk character seems to inherit from him the color pallet, geometrical shapes, and structure of the masks. But it's not very close to the original style.
To put a spell of Tamayo's magical style onto our cyberpunk character, I, again, added a more artist-specific context to the initial prompt.
I ran this test with with the Japanese fashion icon Yohji Yamamoto. The prompt cyberpunk character by Yohji Yamamoto didn't return a single specific result.
But then adding details made it pop.
Let's get back to Erwin Wurm and look at his cyberpunk characters. First—at the basic by Erwin Wurm against his "clean" style...
...and then—at the fantastic results of the more artist-specific prompts.
I also used Wurm's name to check how artwork by works. The basic result was a miss. Interestingly enough, it got better when I used the depicting formula. However, it was not even close to what a contextual prompt did to the artist's style.
TECH GENIUS TEENAGE GIRL
This prompt is a dedication to my friend's daughter, who has just become a teenager and is already a marvel of electronics and engineering. Is this a problematic prompt? I'd say, it's not for nothing she's on the list of The Hardest Prompts. :)
Dziga Vertov was a Soviet pioneer documentary film and newsreel director, as well as a cinema theorist. How Midjourney treats Vertov's style in the sample above? Ignores it! Can we make it respect the great avant-garde cinemeatographer? Yes!
How about a genre? Say, Naive art. The default result is 100% generic.
But what if we add details to the initial style and reformat the prompt a bit?
Finally, let's take our tech genius to a high concept fashion world with the American fashion photographer Steven Klein, famous for his bold and provocative style. Which doesn't show that much in the by generation (although it's good).
However, if we spice the prompt with specificity...
NAZGUL OVER SLEEPING TOWN
I added Nazgul over sleeping town to our benchmark to honour my favorite book as a kid—John R.R. Tolkien's Lord Of The Rings.
Let's give this prompt to another iconic fashion designer, Pierre Cardin. You might expect the results to be high-fashion, elegant and fanciful. But no, the outcome is a pretty generic fantasy image.
How can we change that? Let's add context to the prompt.
Let's try something more peculiar. Laure Provoust is a French multimedia artist known for her immersive films and mixed-media installations. Her initial Nazgul was quite similar to many others.
But then I added context and got some of the most unique Nazguls among all benchmarks!
And for the last example, let's invite the world famous American photographer Annie Leibovitz to tell her version of the Tolkien's story. The baseline generation is very much alike with dozens of others. Until we add...
...you guessed it—artist-specific context! And let's be creative here.
DECAYING PEONIES
Another diehard prompt from our benchmark—decaying peonies.
In this case, I gave it to American fashion photographer Bert Stern. With the initial prompt, peonies inherited from the initial style an overall photo studio look.
Still, I doubt this is how the renowned fashion photographer would have shot decaying peonies. So I asked Midjourney specifically.
Definitely a much better re-imagination of the great artist's style.
Another great example is the American comic book artist Bill Sienkiewicz, known for his work on Marvel's Moon Knight, New Mutants, and Elektra. Here is how Midjourney applies his style with the basic by prompt. You can see it trying to visualize the flowers with Sienkiewicz's signature brushstrokes and paint drops, but... it stops long before reaching its goal.
Artist-specific prompts are here to save the day! I especially love the character-centered ones, but just the flower is outstanding, too.
For today's last case, I want to return to Peter Saville's style—and Remix mode.
Saville's name with the baseline prompt didn't affect the peonies at all.
The artist-specific context gave a much better—design-y—result. But still, it was far from something you would expect from a famed designer and art-director. ( ๑σᆽσ) However, simplifiying the prompt through Remix was very fruitful.
CONCLUSION
With Remix mode or not, adding artist-related context dramatically improves how a style applies to a prompt.
On Midjourney's Community Feed, I see a lot of prompts where people put artists' names in rows of five and even ten—without giving them a thought.
I stand for a different approach—Midjourney to me is a way to educate myself about artists, art history, genres, movements. Among other things, because knowing whose name you are using may greatly help in getting great results from Midjourney.
Happy midjourneys,
— Andrei
P.S. What's your favorite strategy to apply styles to image prompt? Did I miss something? You have a hint or a technique you want to share? Drop me a note via the form below or DM on Instagram →
BONUS TRACK
Today, people mostly know Leon Bakst by his amazing paintings and illustrations, but a lot of these arts were done for his theatrical work: costume- and set-design. So what if we ask Leon Bakst to design a movie set?
Let's use the Leon Bakst's set-design for [title] movie --v 4 and see how Bakst's style will deal with some of the biggest block-busters of all time.
P.P.S. Will we continue testing styles with by prompt? Yes, because it's general and easy, and a great baseline. And if a simple by [style] refuses to affect your generations, you now know what to do!
P.P.P.S. I am doing a separate post on style modifiers for Image Prompts and how to meddle with your own portraits to make the most out of Midjourney's face recognition superpower. ;)
You can help us maintain and expand Midlibrary and produce more regular educational content of higher quality. And keep it free for all!
Support Midlibrary on Patreon! →
