Illustration | LoRA for Stable Diffusion 1.5

This LoRA has been trained on 190 768x768px images created using the Illustration Photo Effect

The LoRA is compatible with stable diffusion 1.5 using the Automatic1111 interface. Make sure you have loaded a stable diffusion 1.5 based model into Automatic1111 before using this LoRA. It is not compatible with stable diffusion 2.x

Using the LoRA

Firstly, ensure you have the latest version of Automatic1111 installed. How to install Stable Diffusion locally using Automatic1111

After installing Automatic1111, copy the “7sIllustration.safetensors” LoRA file found in this download, into your stable diffusion Lora folder directory – E.g.\stable-diffusion-webui\models\Lora

To create the effect, add <lora:7sIllustration:1> anywhere in your prompt – E.g. “Portrait of a woman <lora:7sIllustration:1>“

To adjust the strength of the effect, change the number after the ‘:‘ – E.g. <lora:7sIllustration:0.5>, <lora:7sIllustration:0.6>, <lora:7sIllustration:0.7>, <lora:7sIllustration:0.8>, <lora:7sIllustration:0.9>, <lora:7sIllustration:1>, <lora:7sIllustration:1.1>

The default strength is 1, so make sure to lower it if the effect is coming through too strong.

Upscaling

The upscaling process is very important for enhancing the quality and details of your images.

Below is my workflow for upscaling images:

1. Enable Highres. fix in the txt2img tab
2. Set low resolution dimensions for your image (e.g. 512 x 768)
3. Sampling method: DPM++ 2M Karras
4. Sampling steps: 20-30
5. Upscaler: 4x Ultra Sharp. Place this upscaler in the  ESRGAN folder in your stable diffusion directory (stable-diffusion-webui\models\ESRGAN).
6. Hires steps: 50
7. Upscale by: 1.5-2
8. Denoising strength 0.4-0.5
9. CFG scale: 4-5

Here is a screenshot of the highres. fix settings in the txt2img tab.

Start generating!

To further increase the resolution of your image and enhance details, bring your generated image from the txt2img tab into the img2img tab. 

Below is my workflow:

1. In the img2img tab, upload/drag your image generated from the txt2img tab into the img2img window.
2. Set the prompt and negative prompt the same as the initial prompt used to generate the image in the txt2img tab.
3. Sampling method: DPM++ 2M Karras
4. Steps: 30-50
5. Set the width/height of the image. This should be the same dimensions as the image uploaded into the img2img window.
6. CFG scale: 4-6
7. Denoising strength: 0.3-0.5. Using this range should ensure that when upscaling in the img2img tab, the composition of your image remains in place whilst enhancing details as it upscales. I typically use 0.45. If you set it too high, it will add unwanted extra details to your scene. Set it too low and it wont be able to enhance the quality of the image.
8. Here is a screenshot of the img2img settings.
9. Enable ‘Controlnet’ and set the Preprocessor to tile_resample and the model to control_v11f1e_sd15_tile. 
10. Check ‘Pixel Perfect’.
11. Control Weight: 1
12. Starting Control Step: 0
13. Ending Control Step: 1
14. Control Mode: Controlnet is more important
15. Here is a screenshot of the Controlnet settings.
16. Next either enable ‘SD Upscale’ or ‘Ultimate SD Upscale’. I have been using SD Upscale.
17. In the SD Upscale settings, set the ‘Tile Overlap’ to 32, the ‘Scale Factor’ to 2 and the ‘Upscaler’ to 4x-UltraSharp.
18. Here is a screenshot of the SD Upscale settings.

Next hit generate and your image will then double in resolution while enhancing details.

Negative prompt

The negative prompt can greatly increase the quality of your images. Below are some negative prompt embedding recommendations. Place these inside the ’embeddings’ folder in your stable diffusion directory (stable-diffusion-webui\embeddings):

EasyNegative

Deep Negative

Image Sharpener

After placing these inside your embeddings folder, add these to your negative prompt by using the same name as the file name of the embedding.

Stable Diffusion 1.5 models

Using the right model for the style of image your want to generate is very important. 

Below are some recommendations:

Realism: Realistic Vision 2.0, ICBINP

Fantasy/scifi: Dreamshaper, ReV Animated, GhostMix

Combining LoRA’s

You can add more than one LoRA to your prompt to build new styles.

To do that, simply add the LoRA’s to your prompt, i.e. ‘portrait of a woman <lora:LoRA1:1> <lora:LoRA2:1> <lora:LoRA3:1> ‘. 

Experiment by adjusting the strength of each LoRA to create the style you are looking for.

Support/Tips

If you need assistance using the LoRA, send me a message and I will be in touch.