Stable Diffusion in Google Colab without limits using code

[oscarguid]

Hello everyone,

I was trying to generate images in Google Colab from the stable diffusion code, for now only with the text to image function, I managed to incorporate some functions that interest me such as changing the seed manually, or making the seed random by setting the value. of 1. I also applied a function to show me the generated random seed and incorporated the DPM++ 2M Karras sampler.

But when comparing the results generated by the stable diffusion program on my local computer, the result generated by Google Colab is very inferior, I cannot understand the reason. The images were generated with the same seed, the same prompt and neg prompt, as well as with the same model. I searched the webiu.bat files on my computer to find something about the program code but I had no success, perhaps it was due to my inexperience. Could you guys help me? I leave you the code that I am using in Google Colab, and the comparison of the images.

Dependencies:

#DPM++ M2 Karras
!pip install diffusers["torch"] transformers
!pip install accelerate
!pip install xformers
!pip install git+https://github.com/huggingface/diffusers
!pip install diffusers transformers accelerate scipy safetensors

Text to image Generation Code

import torch
from diffusers import StableDiffusionPipeline, DPMSolverSinglestepScheduler, DPMSolverMultistepScheduler

model_id = "stabilityai/stable-diffusion-2-1"



def generate_seed(seed_value):
    if seed_value == -1:
        return torch.randint(0, 2**32, (1,), device="cuda").item()
    else:
        return seed_value

# Use the DPMSolverMultistepScheduler (DPM-Solver++) scheduler here instead
pipe = StableDiffusionPipeline.from_pretrained("Justin-Choo/epiCRealism-Natural_Sin_RC1_VAE", torch_dtype=torch.float16)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config,use_karras_sigmas=True)
pipe = pipe.to("cuda")
pipe.safety_checker = None

prompt = "photo of a blonde young bellow average woman, portrait, cute face, green eyes, cute, sexy"
h = 512
w = 512
steps = 20
guidance = 7
seed_value = 202904339  # Puedes cambiar esto a cualquier valor para usar una semilla específica o dejarlo en -1 para semilla aleatoria

generator = torch.Generator("cuda").manual_seed(generate_seed(seed_value))
neg = "naked, cartoon, cgi, render, illustration, painting, drawing"

# Obtener la semilla generada aleatoriamente
generated_seed = generator.initial_seed()
print(f"Semilla generada aleatoriamente: {generated_seed}")

image = pipe(prompt, height=h, width=w, num_inference_steps=steps, guidance_scale=guidance, negative_prompt=neg, generator=generator).images[0]
image

[patrickvonplaten]

You seem to install diffusers three times here:

#DPM++ M2 Karras
!pip install diffusers["torch"] transformers
!pip install accelerate
!pip install xformers
!pip install git+https://github.com/huggingface/diffusers
!pip install diffusers transformers accelerate scipy safetensors

Can you maybe change this code to just the following:

!pip install git+https://github.com/huggingface/diffusers
!pip install transformers accelerate scipy safetensors

and then print out the following versions (both locally and on colab)

import diffusers
import transformers

print(f"Diffusers {diffusers.__version__}")
print(f"Transformers {transformers.__version__}")

[oscarguid]

Thanks Patrick for the contribution, I had not noticed about the diffusers, I am very new to this, but I am trying to learn. Thank you!

I tell you that I have achieved good results by changing the model for another version of epicRealism. I leave it below: "stablediffusionapi/epicrealism-natural-sin-r"

I show you the results with two different types of sampler:

On the other hand, I tell you that the following post helps me to improve the code: https://github.com/redromnon/stable-diffusion-interactive-notebook?tab=readme-ov-file it turned out very well, thanks to @redromnon .

I share the final code with you. If there are any errors, I apologize for that:

Dependencies:

!pip install diffusers["torch"] transformers
!pip install accelerate
!pip install scipy
!pip install safetensors
!pip install xformers
!pip install mediapy
!pip install ipywidgets==7.7.1
!pip install git+https://github.com/huggingface/diffusers

Model and sampler selection:

from diffusers import StableDiffusionPipeline, EulerAncestralDiscreteScheduler, DDIMScheduler, EulerDiscreteScheduler, UniPCMultistepScheduler, DPMSolverMultistepScheduler
from diffusers.models import AutoencoderKL
import torch
import ipywidgets as widgets
import importlib

#Enable third party widget support
from google.colab import output
output.enable_custom_widget_manager()

#Pipe
pipe = None

#Models
select_model = widgets.Dropdown(
    options=[
        ("Stable Diffusion 2.1 Base" , "stabilityai/stable-diffusion-2-1-base"),
        ("Stable Diffusion 2.1" , "stabilityai/stable-diffusion-2-1"),
        ("Stable Diffusion 1.5", "runwayml/stable-diffusion-v1-5"),
        ("Dreamlike Photoreal 2.0" , "dreamlike-art/dreamlike-photoreal-2.0"),
        ("OpenJourney v4" , "prompthero/openjourney-v4"),
        ("epicRealism" ,"stablediffusionapi/epicrealism-natural-sin-r"),
    ],
    description="Select Model:"
)

#Schedulers
select_sampler = widgets.Dropdown(
    options=[
        "EulerAncestralDiscreteScheduler",
        "EulerDiscreteScheduler",
        "UniPCMultistepScheduler",
        "DDIMScheduler",
        "DPMSolverMultistepScheduler"
    ],
    description="Select Schedular:"
)
select_sampler.style.description_width = "auto"

#Safety Checker
safety_check = widgets.Checkbox(
    value=True,
    description="Enable Safety Check",
    layout=widgets.Layout(margin="0px 0px 0px -85px")
)

#Output
out = widgets.Output()

#Apply Settings
apply_btn = widgets.Button(
    description="Apply",
    button_style="info"
)


# Get scheduler
def get_scheduler(name):
    match name:
        case "EulerAncestralDiscreteScheduler":
            return EulerAncestralDiscreteScheduler.from_pretrained(select_model.value, subfolder="scheduler")

        case "DDIMScheduler":
            return DDIMScheduler.from_pretrained(select_model.value, subfolder="scheduler")

        case "EulerDiscreteScheduler":
            return EulerDiscreteScheduler.from_pretrained(select_model.value, subfolder="scheduler")

        case "UniPCMultistepScheduler":
            return UniPCMultistepScheduler.from_pretrained(select_model.value, subfolder="scheduler")

        case "DPMSolverMultistepScheduler":
            return DPMSolverMultistepScheduler.from_pretrained(select_model.value, subfolder="scheduler")



#Run pipeline
def pipeline(p):
    global pipe

    out.clear_output()
    apply_btn.disabled = True

    with out:
        print("Running, please wait...")

        # Crear la instancia de pipe
        pipe = StableDiffusionPipeline.from_pretrained(
            select_model.value,
            scheduler=get_scheduler(select_sampler.value),
            torch_dtype=torch.float16,
            vae=AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16).to("cuda")
        ).to("cuda")

        if not safety_check.value:
            pipe.safety_checker = None

        pipe.enable_xformers_memory_efficient_attention()

        # Configurar el scheduler específicamente para DPMSolverMultistepScheduler
        if select_sampler.value == "DPMSolverMultistepScheduler":
            pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True)

        print("Finished!")

    apply_btn.disabled = False

#Display
apply_btn.on_click(pipeline)

widgets.VBox(
    [
      widgets.HTML(value="<h2>Configure Pipeline</h2>"),
      select_model, select_sampler, safety_check, apply_btn, out
    ]
)

Image Generator:

import ipywidgets as widgets, mediapy, random
import IPython.display


#PARAMETER WIDGETS
width = "300px"

prompt = widgets.Textarea(
    value="",
    placeholder="Enter prompt",
    #description="Prompt:",
    rows=5,
    layout=widgets.Layout(width="600px")
)

neg_prompt = widgets.Textarea(
    value="",
    placeholder="Enter negative prompt",
    #description="Negative Prompt:",
    rows=5,
    layout=widgets.Layout(width="600px")
)

num_images = widgets.IntText(
    value=1,
    description="Images:",
    layout=widgets.Layout(width=width),
)

steps = widgets.IntText(
    value=30,
    description="Steps:",
    layout=widgets.Layout(width=width)
)

CFG = widgets.FloatText(
    value=7.5,
    description="CFG:",
    layout=widgets.Layout(width=width)
)

img_height = widgets.Dropdown(
    options=[('512px', 512), ('768px', 768)],
    value=512,
    description="Height:",
    layout=widgets.Layout(width=width)
)

img_width = widgets.Dropdown(
    options=[('512px', 512), ('768px', 768)],
    value=512,
    description="Width:",
    layout=widgets.Layout(width=width)
)

random_seed = widgets.IntText(
    value=-1,
    description="Seed:",
    layout=widgets.Layout(width=width),
    disabled=False
)

generate = widgets.Button(
    description="Generate",
    disabled=False,
    button_style="primary"
)

display_imgs = widgets.Output()


#RUN
def generate_img(i):

  #Clear output
  display_imgs.clear_output()
  generate.disabled = True

  #Calculate seed
  seed = random.randint(0, 2147483647) if random_seed.value == -1 else random_seed.value

  with display_imgs:

    print("Running...")

    images = pipe(
      prompt.value,
      height = img_height.value,
      width = img_width.value,
      num_inference_steps = steps.value,
      guidance_scale = CFG.value,
      num_images_per_prompt = num_images.value,
      negative_prompt = neg_prompt.value,
      generator = torch.Generator("cuda").manual_seed(seed),
    ).images
    mediapy.show_images(images)

    print(f"Seed:\n{seed}")

  generate.disabled = False

#Display
generate.on_click(generate_img)

widgets.VBox(
    [
      widgets.AppLayout(
        header=widgets.HTML(
            value="<h2>Stable Diffusion</h2>",
        ),
        left_sidebar=widgets.VBox(
            [num_images, steps, CFG, img_height, img_width, random_seed]
        ),
        center=widgets.VBox(
            [prompt, neg_prompt, generate]
        ),
        right_sidebar=None,
        footer=None
      ),
      display_imgs
    ]
)

My next challenge will be to incorporate the image to image function.

I was trying to adapt this code, but I didn't have good results, I can't get it to work with the model previously selected in the model and sampler selection cell, maybe I should do something independent:

import torch
from diffusers import AutoPipelineForImage2Image
from diffusers.utils import make_image_grid, load_image

pipeline = AutoPipelineForImage2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-refiner-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True
)
pipeline.enable_model_cpu_offload()
# remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed
pipeline.enable_xformers_memory_efficient_attention()

# prepare image
url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/img2img-init.png"
init_image = load_image(url)

prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
negative_prompt = "ugly, deformed, disfigured, poor details, bad anatomy"

# pass prompt and image to pipeline
image = pipeline(prompt, negative_prompt=negative_prompt, image=init_image).images[0]
make_image_grid([init_image, image], rows=1, cols=2)

[percinor]

RuntimeError Traceback (most recent call last)
in <cell line: 17>()
15 pipe = StableDiffusionPipeline.from_pretrained("Justin-Choo/epiCRealism-Natural_Sin_RC1_VAE", torch_dtype=torch.float16)
16 pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config,use_karras_sigmas=True)
---> 17 pipe = pipe.to("cuda")
18 pipe.safety_checker = None
19

[asomoza]

this code is very old (in AI time) and also you're not posting the error just some lines from the traceback.