Low Rank Adapted Supervised Fine Tuned Stable Diffusion XL
Comparison
| Prompt | SDXL | Fine Tuned |
|---|---|---|
| a boat in the canals of Venice, painted in gouache with soft, flowing brushstrokes and vibrant, translucent colors, capturing the serene reflection on the water under a misty ambiance, with rich textures and a dynamic perspective | 👁 image/png |
👁 image/png |
| Grainy shot of a robot cooking in the kitchen, with soft shadows and nostalgic film texture. | 👁 image/png |
👁 image/png |
Model description
These are ariG23498/open-image-preferences-v1-sdxl-lora LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.
The weights were trained using DreamBooth using the open-image-preferences-v1-binarized dataset.
Use with diffusers
from diffusers import AutoPipelineForText2Image
import torch
pipeline = AutoPipelineForText2Image.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=torch.bfloat16
).to('cuda')
pipeline.load_lora_weights('ariG23498/open-image-preferences-v1-sdxl-lora', weight_name='pytorch_lora_weights.safetensors')
prompt = "ENTER PROMPT"
image = pipeline(prompt).images[0]
Command to train the model
!accelerate launch examples/dreambooth/train_dreambooth_lora_sdxl.py \
--pretrained_model_name_or_path "stabilityai/stable-diffusion-xl-base-1.0" \
--dataset_name "data-is-better-together/open-image-preferences-v1-binarized" \
--hub_model_id "ariG23498/open-image-preferences-v1-sdxl-lora" \
--push_to_hub \
--output_dir "open-image-preferences-v1-sdxl-lora" \
--image_column "chosen" \
--caption_column "prompt" \
--mixed_precision="bf16" \
--resolution=1024 \
--train_batch_size=1 \
--repeats=1 \
--report_to="wandb"\
--gradient_accumulation_steps=1 \
--gradient_checkpointing \
--learning_rate=1.0 \
--text_encoder_lr=1.0 \
--optimizer="prodigy"\
--lr_scheduler="constant" \
--lr_warmup_steps=0 \
--rank=8 \
--checkpointing_steps=2000 \
--seed="0"
- Downloads last month
- 7
Model tree for ariG23498/open-image-preferences-v1-sdxl-lora
Base model
stabilityai/stable-diffusion-xl-base-1.0