BLOG

Control Net Parameters

by GymDreams
Detailed explanation of what the different parameters in a control net do, illustrated with images from the Sacred Geometry series as examples.
Seed of Life - Demon in Red. Sacred Geometry series. Stable Diffusion and Hunk with Latex, Jockstrap

Sacred Geometry Series

Using historical geometric forms as a structural foundation for male figure compositions. Whereas Da Vinci explored ideal body proportions with Vitruvian Man, I use the geometric forms as a foundational grid to layout the composition of the figure. When imagined with the Canny control net inside Stable Diffusion, the joints of the figures correspond to the intersections of the geometric forms, thus creating a unique visual language and structure throughout the series.

Control Net Series

In-depth examples and guides using control nets with Stable Diffusion.

Overview

This infographic gives an overview of all the parameters used in a control net.

Infographic. Control Nets used in Seed of Life - Demon in Red. Sacred Geometry series. Stable Diffusion and Hunk with Latex, Jockstrap.
Infographic. Control Nets used in Seed of Life - Demon in Red. Sacred Geometry series. Stable Diffusion and Hunk with Latex, Jockstrap.

Control Nets

For the Sacred Geometry series, two control nets are used:

Both Canny and Line Art are used to process lines and edges. The difference is what they are used for. Canny is used to process the lines of the forms, such as the outline of buildings or the outline of shapes an figures. Line Art is generally used to process artworks done as black and white ink drawings, e.g. manga.

The illustrations used to drive the Sacred Geometry series are custom drawn illustrations with Adobe Illustrator.

Custom drawn illustrations overlaid on top of an image from Seed of Life, Demon, in the Sacred Geometry series.
Custom drawn illustrations overlaid on top of an image from Seed of Life, Demon, in the Sacred Geometry series.

Parameters

THe five main parameters governing any control net are:

Weight

Weight is the influence that control net has on your prompt. How strongly should it be applied?

  • 1.0. 100% influence.
  • 0.0. 0% influence, i.e. no influence at all.
  • 0.5. 50% influence. Half.

Starting Step

When should the control net start to influence the prompt? Range of value is 0.0 - 1.0. Values are expressed as the percentage of steps.

For a render with 50 steps:

  • 0.0. Start at step 0.
  • 0.3. Start at step 15.
  • 0.5. Start at step 25.
  • 0.7. Start at step 35.
  • 1.0. Start at step 50.

For a render with 30 steps:

  • 0.0. Start at step 0.
  • 0.3. Start at step 9.
  • 0.5. Start at step 15.
  • 0.7. Start at step 21.
  • 1.0. Start at step 30.

Canny - Start 0

Since Canny controls how the space is divided, if you start Canny early at step 0, then your figures won’t be able to fill out the space, and you will have renders like these:

Seed of Life: Priest Mage

Starting Canny at step 0 prevents the figures from entering the lines, thus making smaller figures on the outskirts of the Seed of Life symbol.
Starting Canny at step 0 prevents the figures from entering the lines, thus making smaller figures on the outskirts of the Seed of Life symbol.

Canny - Start Later

If instead of start later, then the figures will form after the lines are somewhat denoised, then it would be able to place the figures inside the entire image.

Seed of Life: Priest Mage

Starting Canny in later steps allows the figures to form within whether lines are.
Starting Canny in later steps allows the figures to form within whether lines are.

Ending Step

When should the control net stop influencing the prompt? Range of value is 0.0 - 1.0. Values are expressed as the percentage of steps.

For a render with 50 steps:

  • 0.0. End at step 0.
  • 0.3. End at step 15.
  • 0.5. End at step 25.
  • 0.7. End at step 35.
  • 1.0. End at step 50.

For a render with 30 steps:

  • 0.0. End at step 0.
  • 0.3. End at step 9.
  • 0.5. End at step 15.
  • 0.7. End at step 21.
  • 1.0. End at step 30.

Canny - End 1

If you end Canny at 1.0, then the lines will be visibly present in the final render.

Flower of Life: Priest Mage

Ending Canny at 1.0 (100%) makes sure that the lines will be visibly present on the final step.
Ending Canny at 1.0 (100%) makes sure that the lines will be visibly present on the final step.

Canny - End Early

If you end Canny early, e.g. 0.5 - 0.7, then while the lines would have influenced the composition of the figures, but they may not appear in the final render.

Flower of Life: Priest Mage

Ending Canny early (before 1.0) means that there’s a chance that the lines would’ve been denoised completely, thus leaivng no trace of them, even if the lines were used to drive the composition.
Ending Canny early (before 1.0) means that there’s a chance that the lines would’ve been denoised completely, thus leaivng no trace of them, even if the lines were used to drive the composition.

Resolution

Even if you’re rendering at 512x512, you should set your preprocessor at 1024x1024. This allows maximum fidelity of fine lines and details.

Priority

There are three modes for priorities:

  • Balanced. Default
  • Prompt-priroity, aka “My prompt is more important”.
  • Control-net priority, aka “Control net is more important”.

For control nets such as canny and line art, the differences are quite subtle. But for control nets intended for style transfers, for exmaple, Shuffle and Reference Only, generally speaking, you should use the Contorl-net priority so that the styles will be fully applied.

You can set different priorities for different control nets.

For Canny and Line Art in the Sacred Geometry Series, I used Balanced. Usually this works tbe best but you shoudl always experiemnt with this setting together with weight to find the true balance.

The two settings are related but it has some differences. I understand the nuance but putting it into words is difficult. I will try to explain it in the future.

Text Prompt is always the king

Whether you use control nets or not, it’s important to remember that the text prompt is always the king. In order to realize the image, you will need to add prompts to realize it. Unless you put what you wanted in the text prompt, they won‘t be realized.

I will use the following images from the Sacred Geometry series to illustrate this.

Illustration

Seed of Life: Priest Mage

This mage is backed by an illustration of the symbol, because that’s what was asked.

The lines of the symbols are realized by what you prompt. Here expressed also asn illustration, as a star map.
The lines of the symbols are realized by what you prompt. Here expressed also asn illustration, as a star map.

Particles, Magic Dust

Seed of Life: Priest Mage

If instead you ask for magic dust and light particles, then the lines will be able to turn into particles instead.

Putting words like light particles and magic dusts in the prompt will turn some of these lines into particles.
Putting words like light particles and magic dusts in the prompt will turn some of these lines into particles.

Hologram

Metatron’s Cube: Priest Mage

The prompt for this image asked for hologram / homographic projections, creating a sci-fi look of the image.
The prompt for this image asked for hologram / homographic projections, creating a sci-fi look of the image.

Paper Ribbons

Seed of Life: Jock

The prompt asked for translucent paper ribbons. As ribbons can be bent, the lines are bent as well.
The prompt asked for translucent paper ribbons. As ribbons can be bent, the lines are bent as well.

Laser Lights

Seed of Life: Hunk in Red

The prompt asked for laser lights and light shows. As light can’t be bent, the lines are formed within the composition similarly to the supplied symbol.
The prompt asked for laser lights and light shows. As light can’t be bent, the lines are formed within the composition similarly to the supplied symbol.

Red Ropes

Seed of Life: Hunk in Red 2

The thickness of the lines can be realized by using materials that are also thick in nature. Red ropes are used here, and thus give volume in a three-dimensional space.
The thickness of the lines can be realized by using materials that are also thick in nature. Red ropes are used here, and thus give volume in a three-dimensional space.

Drawstrings from Shower Curtain

Seed of Life: Hunk in Red 2

Drawstrings from shower curtains were prompted, and Stable Diffusion tried a few imaginative composition, mixing red shower curtains and the drawstrings.
Drawstrings from shower curtains were prompted, and Stable Diffusion tried a few imaginative composition, mixing red shower curtains and the drawstrings.

Red Rope

Seed of Life: Demon

A mix of red ropes and laser light shows were prompted, leading to a a mix of both.
A mix of red ropes and laser light shows were prompted, leading to a a mix of both.

Black plastic tubes

Seed of Life: Demon

Prompting with thick black plastic tubes give even more volume to the lines.
Prompting with thick black plastic tubes give even more volume to the lines.

Blue Ribbons

Seed of Life: Demon

Changing the laser light to blue and asking for ribbons in blue give a stronger continuity to the two types of materials.
Changing the laser light to blue and asking for ribbons in blue give a stronger continuity to the two types of materials.

Tech

  • 30 steps, DPM++ SDE Karras, 512x512
  • Airfuck’s Wild Mix v1, 0.5 Denoising
  • Adetailer: face_yolov8n.pt
  • CN0: Canny, Weight 1, Start 0.5, End 1, Res 1024
  • CN1: Line Art, Weight 1, Start 0.4, End 1, Res 1024
  • Hires: 1.5, 15 steps, 4x_NMKD-Siax_200k, 768x768
  • Topaz Gigapixel HQ 4x, 3072x3072
  • Adobe Lightroom color correction
  • Adobe Photoshop AI inpainting / limb fix
Tags: