What is a Lora

I am writing these articles because I just learned them myself.  

A lora is an ai model that was trained on top of a huge general purpose model.  It is specialized for a limited subject area. It stands for low rank adaptation.  This area could be an art style, a character, clothing, or a location/setting.  And you can use a few of these lora at the same time.

The big model was created using tens of thousand of enterprise class high RAM GPUs.  This can only be done at great expense, expertise, and it takes millions of hours of time spread across all these machines.  But that work is now done and doesn't need to be repeated.  These general purpose models are awesome, they know a little bit about almost everything.  But they don't know a lot about any one thing.  

This is where lora comes in.  As an individual or small company you might need to make images that the larger model doesn't know much about.  Let's say you are making images for a game you are creating and need to make an image of a beholder.  Now the big ai model might have an idea of a beholder because it ran across a couple of images in the millions of files it was trained on.  But it doesn't make very good beholders.  So on your home computer you can collect up all the images of beholders you can find, convert them to a standard size, tag them with text labels, and then train the new model using stable diffusion in a few hours. Then by invoking the lora in your stable diffusion prompt along with a big main model you can place beholders in all sorts of new situations.  You can use as few as a dozen images, but a few hundred is better.

Because it didn't take much effort to create one of these lora people share them with each other. https://civital.com is one of the main sites this happens on.  Word of warning, that site has a lot of adult content, but it is heavily censored by default, so nobody is forcing you to look at the pictures. 

To use a lora from this site, download it, and put it in stable-diffusion-main/models/Lora

At this point you can also take a couple of minutes to create an information file next to the lora.  Create a file called lora-name.description.txt next to the lora. The lora-name isn't literal, it should match the name of the actual lora, except for the .safetensor ending.  In this file, cut and past the keywords used to activate the lora and maybe an example prompt or two from the web site. People post their own examples and the positive and negative prompts they used to create the image. 

To use the lora now that it is installed in the right place do the following.  If you click on the little show/hide extra networks button under the big generate button it will show you everything you have installed on your machine.  This will open up a new extra networks section in the webui.  There is a reload button in that section so you can see the ones you just put in. You just have to click on the lora icon to include it in your prompt.   The text file you created and populated is visible at the bottom of the icon for the file.

These is even a civitai helper that will download thumbnails and the training info used to create a model. There is another plug in that is supposed to manage the keywords for you, but I haven't figured that out yet.

Here is an example. This image was created with a 36MB beholder lora, the stable diffusion prompt is there.

https://civitai.com/images/1256610?modelVersionId=102225&prioritizedUserIds=935296&period=AllTime&sort=Most+Reactions&limit=20

The bit inside the angle bracket is the lora, <lora:beholder_v1-000092:1>. It has lora, there are other networks too, so you have to tell what kind this is, then the name, then : strength.  Sometimes you have to change the :1 to weaker, :0.8, or stronger, :1.2, for the best effect.  the word beholder is the new keyword that the lora has been trained to respond to.

I hope I have been clear.  This is everything I know about loras right now.