Victoria 3 Style Art Generator Tutorial

Brief LoRa

Low-rank adaptation (LoRA) is an adapter-based technique for efficiently fine-tuning models. The basic idea is to design a low-rank matrix that is then added to the original matrix. An adapter, in this context, is a collection of low-rank matrices which, when added to a base model, produces a fine-tuned model. It allows for performance that approaches full-model fine-tuning with less space requirement. A language model with billions of parameters may be LoRA fine-tuned with only several millions of parameters.

From Wikipedia

This technique can make model fine-tuning easier. Not only requires fewer datasets, but also consumes less resources of the computer.

The Victoria 3 Style Art model was trained (or more accurately, fine-tuned) via this method.

Usage

Environment

a NVIDIA graphic card with CUDA
CUDA Toolkit

stable-diffusion-webui

Victoria 3 Style Art Model File

Stable Diffusion Base Model - I recommend Real Dream Classic, Note: Please always use SD1.5 based model.

Setup

The setup method of stable-diffusion-webui can be found on its GitHub page.

If you already complete setting up the first 3 things, you can put realDream_*.safetensors into stable-diffusion-webui/webui/models/Stable-diffusion and put vic3_v3.safetensors into stable-diffusion-webui/webui/models/Lora.

Please double-check that you haven’t misplaced them.

Generating

Launch stable-diffusion-webui. (also, you can find launching tutorial on its GitHub page)

You can select your checkpoint (base model) on the upper left on this page. I will show you an example using Real Dream.

For now, use the txt2img tab, other tabs you can explore them later.

Then put in your prompt, let me show you an example. By the way, this model is not really good at generating human. I will just show you how to generate pics without human. Here is the example:

Positive:

(masterpiece, best quality:1.2), vic3, library, indoor, book, shelf, candle, (fire:1.75), burn, ash, smoke, ruin <lora:vic3_v3:0.8>

Negative:

lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts,signature, watermark, username, blurry

The most important thing is including <lora:vic3_v3:0.8> and vic3 into the positive prompt. You can also change the strength of the style by changing the number after <lora:vic3_v3:, for example <lora:vic3_v3:0.5>.

Now, move to the Generation tab. The best method I tested is: