Create a pipeline from pretrained config without downloading weights

[ljleb]

I would like to download only the configuration files from an entire model pipeline located on hugging face (for example runwayml/stable-diffusion-v1-5) and then instantiate an uninitialized model from that configuration. In this process, I want to download as little content as is necessary to create the model instance, because it takes a lot of time and bandwidth to do download models online.

Is there a simple way to do this using existing code? I tried using ConfigMixin.from_config but to no avail:

config = diffusers.StableDiffusionPipeline.load_config("runwayml/stable-diffusion-v1-5")
unet = diffusers.StableDiffusionPipeline.from_config(config)

This raises the following error:

  File "[redacted]\venv\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py", line 192, in __init__
    if hasattr(scheduler.config, "steps_offset") and scheduler.config.steps_offset != 1:
AttributeError: 'list' object has no attribute 'config'

As a side note, it appears to work perfectly fine to load each individual component of the pipeline one by one like this:

unet_config = diffusers.UNet2DConditionModel.load_config("runwayml/stable-diffusion-v1-5", subfolder="unet")
unet = diffusers.UNet2DConditionModel.from_config(unet_config)
...
pipeline = diffusers.StableDiffusionPipeline(unet=unet, ...)

However, it would be best to avoid having to load individual components of a pipeline and then having to put these together manually in this way. I intend to load a large number of models in this way, so it would be tedious to have to seek which components different pipelines have and then load them using the arbitrary class specified in their config every time.

All configuration necessary to create a model should be located in the corresponding hugging face repo. Surely I am missing something, there must be a simple and efficient way to create an empty pipeline using a pretrained model as a template.

[AbhinavJangra29]

did you find anything?