[Feature Request]: Upgrade to Gradio 4.0+

[abidlabs]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Hello folks! This is Abubakar, one of the maintainers of Gradio. We recently released Gradio 4.0 with significant more flexibility in building Gradio apps, such as the ability to create fully-custom Gradio components (see 4.0 highlights here).

I'd like to suggest that Automatic1111 WebUI upgrade to the latest version of Gradio, as 4.0 solves many annoying bugs when using Gradio within the WebUI. For example, this issue #13292 is fixed in Gradio 4.x

In addition, Gradio 4.x offers several improvements that many be beneficial to the users of the WebUI. Just a couple of examples:

• Gradio 4.x includes a redesigned Image and ImageEditor component with improved UI/UX, e.g. the Image component supports native clipboard-paste and the ImageEditor component provides a much nicer UX when sketching, cropping, or performing any sort of basic manipulation of images.
• Gradio 4.x includes official support for Custom Components. A Custom Component is just some Python + JS / Svelte that can be used as fully-featured component in any Gradio app. This offers a cleaner way to create extensions and new kinds of components that are not currently possible in Automatic1111 (e.g. a component that lets you interactively explore a panoramic 3D generated image), while getting all of the benefits of Gradio components.

Our plan with Gradio 4.x is to be backwards compatible not just with Gradio's core components, but also any Custom Components that are created on top of Gradio. This should make it much easier to upgrade the Gradio dependency within 4.x much easier if the WebUI would like to in the future.

Being a major version release, Gradio 4.x does include breaking changes from Gradio 3.x. These are listed here along with migration tips, and we're happy to offer specific suggestions on how to migrate any other code from Gradio 3.x to 4.x.

Proposed workflow

Bumping the gradio dependency to the latest 4.x version

Additional information

No response

[aonmkugs]

I wholeheartedly support this, the ImageEditor seems like a great improvement that many people were waiting for (especially an eraser for the Masking tool that got requested many times and users were always directed to ask Gradio to add it, well now they have :D ).

So the minimum version for the upgrade should be 4.5.x as it seems that's when it has been introduced.

Here's a demo video of the new ImageEditor from the changelog:

gradio-imgeditor.mp4

[w-e-w]

AUTO hasn't voiced his opinion on this, this is just my opinion

I have been trying to get greadio 4.x working with webui but haven't succeeded yet
theres just so many breaking changes
to name a few

• removes **kwarg from every component
• the restructure on how components are initialized
• removal of previously used components like gr.Box
• Listener _js is now js (possibly even more changes here)

at this point I feel either we need to write a compatibility layer or lots of things have to be rewritten
and this is not even considering the complication that extensions will bring to the table

I highly doubt we will switch to gradio 4.x in the next release

[abidlabs]

Yes indeed there are quite a few changes, if it helps, I'll explain some of the motivation for these changes:

removes **kwarg from every component

Previously, if a user passed an invalid argument to a component class, we would allow it and then raise a deprecation warning. Now, we've removed **kwargs since there's no advantage to allowing unsupported arguments and prevents accidental typos etc.

the restructure on how components are initialized

We've made some internal refactors to the classes to simplify file management and make it a bit easier for people to create custom components, but this shouldn't affect the public facing API for creating custom components. Let me know if you have questions / concerns here and we'll do what we can to mitigate.

removal of previously used components like gr.Box

In place of gr.Box(), please use gr.Group instead.

Listener _js is now js (possibly even more changes here)

The only difference is that now js is an officially supported parameter, but there are no changes in behavior. There is a new js parameter in Blocks that can be used to run scripts when the app initially loads.

[w-e-w]

yeah I kind of understand some of those changes
but this means that we will have to patch lots of stuff in order to make it backwards compatible with extensions
as extension updates tends to lag behind, it will cause complete chaos if it weren't backwards compatible

[nyukers]

Can I have to upgrade gradio to version 4.0 from version 3.41.2 which I see in A1111 v1.6.1?
Isn't my A1111 going to break down?

[thiagojramos]

Can I have to upgrade gradio to version 4.0 from version 3.41.2 which I see in A1111 v1.6.1? Isn't my A1111 going to break down?

It will probably break, yeah.

[ThereforeGames]

My two cents as an extension developer: I don't think it's reasonable to prioritize extension compatibility over keeping up with Gradio updates.

I have a few reasons for thinking this:

• The WebUI community generally understands that extensions will break when A1111 pushes an update to the main branch. Thus far, there has been little to no emphasis on backwards compatibility with extensions.
• If necessary, users can take advantage of GitHub's rollback features - it's especially straightforward now that Auto has versioning control.
• No matter how many compatibility layers are implemented, something's going to break. Extensions can interface with any and all WebUI modules. Such is the nature of the Python framework.

So as nice as it would be to go to sleep knowing that my code will automatically work with the next WebUI update, I don't think it's worth the effort. I'd rather see the WebUI team focus on improving the core experience than on writing countless abstraction layers that developers may or may not use.

[thiagojramos]

The responsibility for maintaining compatibility with WebUI should fall on the extension developers, not the other way around. Take, for example, a situation where WebUI receives a significant update that enhances the interface's stability and security. However, this update leads to incompatibility with several popular extensions. In this scenario, even though the update temporarily results in fewer functionalities due to the disabled extensions, the benefit of having a more stable and secure UI outweighs the drawbacks. This situation is akin to the bug in the 'masking' session of the regional prompt, which is related to gradio. In the long run, it's more beneficial for users to have a solid and reliable base platform, and it's up to the extension developers to adapt and maintain compatibility. Therefore, while we acknowledge the complexity involved in extension development, the priority should be maintaining the functionality and efficiency of the WebUI itself. But that's just the opinion of a regular user like me.

[dfl]

I hope this happens soon... there are some nice things that Gradio 4 can do that v3 can't.

[catboxanon]

Support in #14184 is expected to be merged when gradio-app/gradio#7591 is resolved.

[abidlabs]

Cool we're going to get in a bunch of fixes for the ImageEditor this week

[jhj0517]

I hope this happens!

1. You can "trim" the video input in gradio 4.36.1
   

2. You can access the original image name of the gr.Image() in the function, which is impossible in the gradio version of the current sd webui as far as I know.

img_input1 = gr.Image("input Image", type="filepath")
btn_generate = gr.Button("Generate")

def function(img_input):
      print(img_input.name)   # This returns the original name of the file, but does not do so in Gradio 3.x.

btn_generate.click(fn=function, inputs=[img_input1], outputs=None)

[thiagojramos]

I hope this happens!

1. You can "trim" the video input in gradio 4.36.1
   
2. You can access the original image name of the gr.Image() in the function, which is impossible in the gradio version of the current sd webui as far as I know.

img_input1 = gr.Image("input Image", type="filepath")
btn_generate = gr.Button("Generate")

def function(img_input):
      print(img_input.name)   # This returns the original name of the file, but does not do so in Gradio 3.x.

btn_generate.click(fn=function, inputs=[img_input1], outputs=None)

One day, maybe...

[w-e-w]

• Gradio 4 #14184 (comment) AUTOMATIC1111

I don't know when I will merge this in. With SD3, it definitely should be after that.
I also want to get some info about which big extensions this breaks, and possibly fix compatibility issues from our side.