BLOG

Regional Prompter

by GymDreams
Using Regional Prompter to control individual characters of a couple.

Have you ever wondered why so many images generated with AI feature a single person? Are you disappointed to find that AI generated couples often look like twins? There’s a reason for that.

Text-to-image engines can’t natively know which part of the text prompt belongs to each person. So when you asked for different outfits for each of your characters, they often mix everything together. Midjourney is one such engine. A follower asked me why I don’t post images of couples often. That’s because I have to generate 100+ images and hope that the RNG gods will bless me with 5 usable images.

Prompt by region

Regional Prompter was created to deal with this issue. You can divide the canvas into regions, then individually prompt for things which don’t collide. I made two sets of images to show the results.

Age Difference

Muscle Mass Difference

Workflow

I divided the canvas into two regions horizontally (1:1) and prompt individually.

To show that the AI was not simply associating proper attire with the expected age group, I swapped the clothes.

Three men, two regions

You can define base and common regions that make interesting results. For example, here I divided into two regions, but asked for 3 men. It creates an image with 3 men with a progression of age, hair color, and clothing.

Two men, two regions

If you asked for two men, the split goes all the way through! Try doing this with just txt2img and you know that this is impossible.

One man, two regions

You can get creative and split a single person to look different for different regions.

Further Examples

Here are some additional renders from the batch.

A follower asked me if I cherry-picked the results. I didn’t. I didn’t post everything from the batch, and of course some images came out looking better than the others — but the reason I didn’t post all the images from the batch had to do with the quality and not the regional division. All of them divided the regions perfectly.

Consistency and productivity. Prayers to RNGods not required.

Text prompts in Stable Diffusion with Regional Prompter. Source image as is. No upscaling, no post-processing, no in-painting. The results are really that good!

  • Model: A-Zovya Photoreal v1 Ultra
  • Sampler: DPM++ 2M Karras
  • Res: 512x512
  • Steps: 30
  • Control Net: None
  • Regional Prompter: Horizontal 1,1, varying base ratio (0.2-0.5) depending on image

Note: Actual implementation uses 0-based region IDs.

Common Issues

I have received some comments that people are having issues with Regional Prompter. Here are some common issues and how to fix them.

  • Enable it. This is by far the most common reason that RP does not seem to be working — remember to turn it on! Many people entered all the parameters inside the Regional Prompter UI but forgot to check the checkbox to enable it. I have personally done that a few times myself. So if RP is not working, check that you have enabled it.
  • Common vs Base. There are two ways to define the common prompt: Common vs Base. Base is the method shared by the scene, where you build on top of it. Common is a shared prompt that will be applied to both regions. It has a subtle difference. For example:
    • Base. If you put two men inside the base prompt, you are saying that you want to see two men in the entire image.
    • Common. If you put two men inside the common prompt, you are saying that you want two men in each other region — meaning you would sometimes end up with four men.
    • Knowing when to use Common vs Base is thus important. For couples, I usually use Base. You can also enable both Base and Common prompts but it can quickly get very complicated if you do.
  • Base Weight. The base weight is set to 0.2 originally. But as your prompts get complex and especially if you meticulously describe the people in your prompts, you might find yourself having to increase the base weight. Often I find myself having to use 0.5 base weight so as to keep two people inside the image and even put (two men:1.8) (2boys:1.2) in the base prompt.
    • Re: 2boys: it’s not asking for two actual boys btw. That’s a Booru tag, which is short for Danbooru tag and e621 tag. Many Stable Diffusion models are trained with these tags as shorthands to describe what’s in the scene: solo man (1boy), male couples (2boys), etc. So if you need to make a scene showing a gay couple, using 2boys will quickly let Stable Diffusion know that‘s what you wanted.
    • Most people won’t know these tags by heart except the common ones. There’s an Automatic1111 plugin that helps you with them with auto completion: a1111-sd-webui-tagcomplete. I use it. Besides being helpful for spelling, sometimes you’d learn new tags that you can use.
    • It‘s conceptually similar to archetype prompting in Midjourney, which is something I use extensively when working with MJ. But that‘s a very large topic so I will write an article about it in the future.
  • Start small. Don’t dump a huge prompt with unnecessary qualifiers (aka text tokens) from your other previously generated images. Start small and slowly. First divide the region and then gradually expand on it for the image. The reason for this is mostly because of the base weight. If you start with a very complex base and figure, often it’s not clear when you need to increase the weight so that your images won’t derail from your intention. It’s also helpful if you gen a couple before tweaking the prompt, or gen in sets of 2 or 4 depending on the resolution and quality.
  • Work quickly. It’s often helpful to work in lower quality until you could get a rough sketch before adding all the hires fix ADetailer. You’ll be saving a lot of time by getting the pose configuration right before working towards the final image.

Related Posts

Instagram Sets

Resources

Tags: