⚡ Flux.1-dev: Upscaler ControlNet ⚡

This is Flux.1-dev ControlNet for low resolution images developed by Jasper research team.

👁 Image

How to use

This model can be used directly with the diffusers library

import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetModel
from diffusers.pipelines import FluxControlNetPipeline

# Load pipeline
controlnet = FluxControlNetModel.from_pretrained(
 "jasperai/Flux.1-dev-Controlnet-Upscaler",
 torch_dtype=torch.bfloat16
)
pipe = FluxControlNetPipeline.from_pretrained(
 "black-forest-labs/FLUX.1-dev",
 controlnet=controlnet,
 torch_dtype=torch.bfloat16
)
pipe.to("cuda")

# Load a control image
control_image = load_image(
 "https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Upscaler/resolve/main/examples/input.jpg"
)

w, h = control_image.size

# Upscale x4
control_image = control_image.resize((w * 4, h * 4))

image = pipe(
 prompt="", 
 control_image=control_image,
 controlnet_conditioning_scale=0.6,
 num_inference_steps=28, 
 guidance_scale=3.5,
 height=control_image.size[1],
 width=control_image.size[0]
).images[0]
image

👁 Image

Training

This model was trained with a synthetic complex data degradation scheme taking as input a real-life image and artificially degrading it by combining several degradations such as amongst other image noising (Gaussian, Poisson), image blurring and JPEG compression in a similar spirit as [1]

[1] Wang, Xintao, et al. "Real-esrgan: Training real-world blind super-resolution with pure synthetic data." Proceedings of the IEEE/CVF international conference on computer vision. 2021.

Licence

This model falls under the Flux.1-dev model licence.

Base model

black-forest-labs/FLUX.1-dev

(580)

A collection of ControlNet models for Flux.1-dev by Jasper Research • 4 items • Updated Sep 24, 2024 • 28