How can I change the image's size and aspect ratio?

[caol64]

I'm using a Swift library, but I cannot find a way to change the image's size and aspect ratio. The following code is not functioning properly:

        pipelineConfig.originalSize = 768
        pipelineConfig.targetSize = 768
        let images = try pipeline!.generateImages(configuration: pipelineConfig,
                                                 progressHandler: { progress in
            onProgress(progress)
            return !canceled
        })

The generated image size is fixed at 1024x1024 (XL) and 512x512, and the code does not seem to support changing the aspect ratio. Can anyone give me some suggestions?

Thanks!

[swiftuiux]

Hi If you are dealing with NSImage take a look on the code Might help

https://gist.github.com/The-Igor/5b41fa5ee881112de770665fc103a4d9

[ZachNagengast]

the originalSize and targetSize parameters are actually just another condition for the model, similar to text embeddings, where you can influence the composition of the scene by telling the model that you want a specific size or cropping. Here's a link from HF discussing the size conditioning, and here is a screenshot from the paper on crop conditioning.

If you want to change the actual pixel size value of the output image, you'll need to export a model using the --latent-w and --latent-h cli parameters. Or resize the image directly per @The-Igor's comment.

[caol64]

@The-Igor Thank you for your awesome example.
@ZachNagengast Thank you for your awesome response.

So far, it has not been possible to use the same model to generate images of different aspect ratios by varying parameters. To achieve this, one would have to use --latent-w and --latent-h to generate different models, and then use these models to generate images of various aspect ratios. Is my understanding correct?

[ZachNagengast]

Correct, and this is also theoretically achievable via two options alternative options:

1. Use the new adapters for the input layer to combine multiple models with a couple common aspect ratios into the same model
2. Use flexible/enumerated shapes for the input - but this will reduce performance quite a bit due to being limited to CPU

These will both need supporting code to achieve in this repo.