Tutorial: Deploy stable-diffusion-webui on Google Cloud Computing #9131
Replies: 18 comments 23 replies
-
You just downloaded this file in the previous step. And |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for pointing that out. I knew it, and yes it makes better sense to start locally. Edited. But I do think it's easier to avoid file editing for beginners. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks. Maybe can add this when launching as well: stable-diffusion-webui/webui.sh --listen --xformers --share --api --disable-safe-unpickle --enable-insecure-extension-access --no-download-sd-model --no-half-vae --disable-console-progressbars |
Beta Was this translation helpful? Give feedback.
-
|
There are many arguments to add as needed. However, for this deployment on the Cloud, they are mostly not essential. Readers should determine their own needs when moving further from this article. |
Beta Was this translation helpful? Give feedback.
-
|
May i know how do you run the webui locally? |
Beta Was this translation helpful? Give feedback.
-
|
You can refer to the README of this repo. |
Beta Was this translation helpful? Give feedback.
-
|
apology. maybe i wasn't clear in my question. I have already deployed the web ui onto google cloud VM. I'm currently accesing via something like https://c62e9d7d6534549.gradio.live/ .. i saw you didn't use the "--share" flag. So i'm just wondering how u access the webui .. since i don't think we can access https://0.0.0.0:7860 from internet? |
Beta Was this translation helpful? Give feedback.
-
|
No worries. Please read section 1.5 and 4 of the tutorial. |
Beta Was this translation helpful? Give feedback.
-
|
I make it work on 2nd try, thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Is it still working for you? |
Beta Was this translation helpful? Give feedback.
-
|
Warning: Spot will change your IP often and will go offline whenever it wants. And you can't change it after without making a new instance and transferring the disk over. Just a heads up. |
Beta Was this translation helpful? Give feedback.
-
|
Is this working for you? |
Beta Was this translation helpful? Give feedback.
-
|
This is nice to mention, especially the point that you can mount or "transfer" the dish to another vm if you want. |
Beta Was this translation helpful? Give feedback.
-
|
For the IP, I guess you need to pay to have a pre-allocated IP for you to use. |
Beta Was this translation helpful? Give feedback.
-
|
Can these settings run multiple requests in parallel? if no, then please could you point out how do I do that? |
Beta Was this translation helpful? Give feedback.
-
|
Is this working for you? |
Beta Was this translation helpful? Give feedback.
-
|
Wehn trying to run stable-diffusion-webui/webui.sh --listen --xformers in Google Cloud in a Compute Engine instance. It gives me this error is that normal? Cannot locate TCMalloc (improves CPU memory usage) |
Beta Was this translation helpful? Give feedback.
-
|
you likely need to be on python version 3.10.XX according to this issue this article might help you |
Beta Was this translation helpful? Give feedback.
-
|
For me, |
Beta Was this translation helpful? Give feedback.
-
|
Had the same issue. So I manually reran the line in terminal ie Which works, but re-running it via the Comment out the following lines at about line 351, and it should work |
Beta Was this translation helpful? Give feedback.
-
|
How do you ensure the right packages are installed for GPU detection? I am getting the following: And I followed all the steps exactly. |
Beta Was this translation helpful? Give feedback.
-
|
I got this same error the first time I tried to install. But there were a lot of variables in play. I installed on 20.04 Ubuntu (but now am not sure it was "minimal", not even sure if that mattered). I had to upgrade python3 from 3.8 to 3.10, which caused some other issues I had to power through. I gave up when I got this Torch error at the end. SO, I deleted all that and started once again, but this time making sure I selected Ubuntu 22.04 MINIMAL. I went to 22.04 because it has Python3.10 by default so that helped. That Torch error went away. But ... using Ubuntu 22.04, I was not able to install the Nvidia drivers using the Google batch script. However, I was able to install it following the instructions here: https://www.thegeekdiary.com/ubuntu-drivers-command-not-found/ |
Beta Was this translation helpful? Give feedback.
-
|
For anyone who's having problems with RuntimeError, I found a workaround. The problem is that the Nvidia drivers don't install in this passage: You can confirm by running The solution is to install them manually, following this page: https://docs.nvidia.com/cuda/archive/12.1.0/cuda-installation-guide-linux/index.html#prepare-ubuntu After you're done, reboot with |
Beta Was this translation helpful? Give feedback.
-
|
when I followed the above instructions, the GPU did not install appropriately however when I followed the google compute engine instructions here and used the following commands, |
Beta Was this translation helpful? Give feedback.
-
|
@MinhNgyuen |
Beta Was this translation helpful? Give feedback.
-
|
Probably you just need to wait a few more minutes. You could try running the last command shown on screen, which is the installation of the driver. Try it without the arguements (delete the "-s --dkms --no-cc-version-check") so you can monitor the installation process. |
Beta Was this translation helpful? Give feedback.
-
|
My thanks to the OP (@walawaya) for going through this. I am sure I would have figured it out, but you gave me a very good roadmap that took a lot of guesswork out and reduced the many forks in the road I would have encountered. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the tutorial! About accessing the webUI (1.5, 4), I would share another possible option what I use. If you have gcloud installed on your local workstation than you can use the good old ssh tunneling feature. You do not need to create firewall rule or network tags in GCP, just start the webUI in your GCP VM instance without --listen or --share options and use this command on your local workstation to setup a secure tunnel: gcloud compute ssh VM_NAME \
--project PROJECT_ID \
--zone ZONE \
-- -4NL LOCAL_PORT:localhost:REMOTE_PORT For example: gcloud compute ssh vm-gpu \
--project something-somewhere-007 \
--zone us-yourzone-a \
-- -4NL 7880:localhost:7860 When the tunnel is established, open a browser on your local workstation and use the http://localhost:7880 URL to access the webUI on your GCP VM. All the traffic is sent over the secure channel. I used -4NL instead of just -NL, because I forced SSH to use IPv4 instead of IPv6. I am sure this can be done with "ssh" instead of "gcloud compute ssh", but I did not configure GCP/VM to use my own ssh key... |
Beta Was this translation helpful? Give feedback.
-
|
you can add this onto wiki/Guides-and-Tutorials if you wish |
Beta Was this translation helpful? Give feedback.
-
|
That would be great, thanks! I seem not able to edit the Wiki. You decide where and how to put it in the Wiki. Maybe just put a link to this discussion page because the comments here are nice. |
Beta Was this translation helpful? Give feedback.
-
|
Or here under Other https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Online-Services |
Beta Was this translation helpful? Give feedback.
-
|
If anyone is getting "Failed to create model quickly; will retry using slow method" on Nvidia T4, you may run this: "pip install open-clip-torch==2.20.0" |
Beta Was this translation helpful? Give feedback.
-
|
error: |
Beta Was this translation helpful? Give feedback.
-
|
Got this set up and running today, good tutorial! One thing I would note is that loading a big model (especially an XL) will run the VM out of memory and cause the whole thing to hang or even crash. I experienced 5+ minute load times even with 16GB. |
Beta Was this translation helpful? Give feedback.
-
|
anyone know how to install extensions when running stable diffusion on cloud? --listen and --share will disable all extensions |
Beta Was this translation helpful? Give feedback.
-
|
Establish an ssh tunnel between your cloud server and your local machine. If done correctly, you'll be able to access the servers localhost on your own machine without '--share' |
Beta Was this translation helpful? Give feedback.
-
|
if youwant to doit in the UI you have to enable you can also install extensions manuly by cloneing extensions to the extensions dir or as penguinbulletin suggested, establish a secure connection using methods such as SSH and access webui via a fake local connection |
Beta Was this translation helpful? Give feedback.
-
|
Cloning extensions to the extensions dir is fine, but you will need to
install the requirements manually. Typically they have a requirements.txt
so you can use `pip install -r requirements.txt`.
If you only open your firewall endpoint to the ip address you access the
server from, `--listen --enable-insecure-extension-access` is safe enough.
…On Sat, Mar 9, 2024 at 2:33 PM w-e-w ***@***.***> wrote:
if youwant to doit in the UI you have to enable
--enable-insecure-extension-access see
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Command-Line-Arguments-and-Settings
only enable this when you need it, as it is a security risk
you can also install extensions manuly by cloneing extensions to the
extensions dir
—
Reply to this email directly, view it on GitHub
<#9131 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE6QWDHCKX4BCTVSM7AQNSDYXN5ZPAVCNFSM6AAAAAAWLPOHBSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DOMZSGA3TE>
.
You are receiving this because you commented.Message ID:
<AUTOMATIC1111/stable-diffusion-webui/repo-discussions/9131/comments/8732072
@github.com>
|
Beta Was this translation helpful? Give feedback.
-
|
tldr: unless the extensions doc have extra instructions you should just clone and do nothing else but restart webui
no, requirements shoud be auto installed by webui by the extensions's |
Beta Was this translation helpful? Give feedback.
-
|
I did the SSH thingy and etc still stuck, i cant get any diffusion models or lora in this thingy |
Beta Was this translation helpful? Give feedback.
-
|
If you guys have same problem just use gcloud scp, so much easier. Just dont forget to set sudo chmod 777 permission to the home folder in the vm instance |
Beta Was this translation helpful? Give feedback.
-
Why Google Cloud
Hello there😉, I am a newbie in programming and everything I am about to say, but I think it's worth sharing the deployment solution I came up with ☁ Google Cloud Computing.
Many have shared their CoLab deployments which are awesome but since I was not sure about their paid "compute units", I thought I would go for a pay-as-you-go style from Google Cloud. Colab-free is only good if you want to get a feeling of the powerful professional GPUs, but is inconvenient for extended use #1814.
While CoLab does have an option to connect with Google Cloud, it requires a little more resource than what our webui needs. And a cherry on the top is the super low Spot (preemptible) price of Google Cloud you may take advantage of (not available for Colab). I feel one can get maybe 1000-2000 512x512 pictures for $1 of Spot machine over a few hours but I haven't particularly benchmarked.
Another thing of note is the use of firewall rule (Step 1.5) to avoid exposing your webui to the public. (Although it would also be difficult to find you with such a complex URL generated by gradio with the
--shareoption. And you can set usr/pw in the launch arguments.)Before you start, please make sure you know some Linux command line; You have a payment method for Google Cloud and don't care wasting a few dollars for the cloud machines you used.
1. Set up a Google Cloud billing account
Understandably, Google Cloud doesn't allow you free use of their GPU, so you will pay for each second of GPU+CPU usage. If you are new to Google Cloud, you may have free credit in a trial period which can pay for a lot of things, excluding GPU instances 😥.
We will follow this guidance with modifications.
https://cloud.google.com/compute/docs/create-linux-vm-instance
Please take some time to familiarize yourself with the UI of Google Cloud and the price sheet of Google Cloud. Keep in mind that data traffic from the cloud virtual machine to yourself (from Google to Internet to your browser) will incur a charge after the free 1 GB/month in North America. For people generating much more images, you will be smart enough to find a way.
If you are not sure, after finishing and moving all your images to your local device, just delete all the instances, delete all the disks in the console and you will be most probably free from unexpected charges.
1.5 (optional) Set up a firewall rule for direct private access to the webui
(Or you can use
--shareoption if you do want to generate a public link)2. Create a VM instance
Derived from the page above, Follow my setup selections:
Note: You may run into an error saying you don't have enough GPU quota to create the VM. As stated here, you will need GPU quota to start an instance with GPU. Follow the instructions in this page to check your GPU quota and request an increase of 1 GPU if needed.
The VM will auto start after creation. So don't forget about it because you are charged from now on.
To find your VM instance, use the left navigation bar and go to "Compute Engine" >> "VM instances".
3. Initial Configuration
3.1 SSH connect to the VM
Go to the "VM instances" page, start your instance if you haven't. After a while, the SSH button under the "Connect" column will light up.
Click the SSH button. You will have access to the command line in a new window.
3.2 Initialize
I am not an expert in this, so please point out if I am being unprofessional.😜
First get a few basic tools. (Some should be already included.)
The Ubuntu image doesn't come with GPU driver. So we need to install following the instruction from Google. https://cloud.google.com/compute/docs/gpus/install-drivers-gpu
Now you can test if GPU driver is properly installed using
sudo nvidia-smiInstall some dependencies. This worked for me but experts may find a subset of it would also work.
Now clone the stable-diffusion-webui.
If you will use its default sd-1.5, then move ahead. If you don't want webui to download sd-1.5, now you should download your desired ckpt to its model folder. Example:
wget [URL to .safetensors] -P stable-diffusion-webui/models/Stable-diffusion/If you use other extensions, additional networks, you should also get them into respective folders.
And now launch the webui as below. With the
--xformers, the program will install xformers along with pytorch and other dependencies.After installation, it will load the model and then show you can access by localhost:7860.
4. Access the webui
If you used
--shareat the end of the bash sentence above, then you can use the link given by the program.If you have set up the firewall rule:
Go to the "VM instances" page, copy the External IP of your instance. Paste it to your browser and add
:7860then GO.Now you should see the webui in your browser.😉
To get your generated imaged, right click on the image and "Save As". Don't click the "Save" button on the webpage.
If you want to directly manipulate/get the image files from the console, please refer to Linux guidebooks.
There are also inspiring discussions that add some python code to push the generated stuff to your destination.
5. Shut down
After you finished, you can directly click STOP in the "VM instances" page.
You will not be charged for the CPU/GPU when it is not running, and the 25 GB standard persistent disk is within 30 GB/month free tier, meaning at least if you only have this one disk at all times, you won't be charged.
6. Deal with Spot scheduling
I have too short experience with Google cloud to bring a clear pattern as for what time/machine would see less interruption. As Google suggested, try nights and weekends. Note that you can't change between regular and Spot, so I suggest the initial set up should be done in afterhours to avoid interruptions. (Other smart ways available) (That is, as @huotarih mentioned, to unmount the Boot disk by EDITing the Spot VM, then create another "non Spot" VM while selecting that disk as Boot disk.)
When Google needs to take over (stop, shut down, preempty, whatever) your Spot VM, you won't notice anything other than no response for ~ 1 minute. Then the SSH will disconnect and you will see your instance is stopped. If you are interrupted halfway, you can later restart the VM and look for the generated images in the "output" folder.
7. Clean up
As I said in the beginning, if you worry about charges, just delete everything when you feel you won't return for a while.
8. Afterwords
20230927
Thank you for viewing, trialing and commenting on this thread! Special thanks to everyone below who posted their solutions to problems!
Seeing some of the errors mentioned below, I gave it a clean test run and it is still working on 9/27/2023 as I wrote above. The only thing I feel different is it takes longer to install the GPU driver than before. Specifically, this last command
Executing: sh NVIDIA-Linux-x86_64-525.125.06.run -s --dkms --no-cc-version-checktook about 5 minutes to complete. It is a-silence installation so nothing is moving. But it is running: I can see from the OBSERVABILITY tab in the web console that the CPU is running high for several minutes.I once ran out of patience and aborted the installation, then installed manually by
sudo sh NVIDIA-Linux-x86_64-525.125.06.run- it works. Then I started over and followed my original plan, just waited for theinstall_gpu_driver.pyscript to finish: It also worked after 5 minutes of waiting. If you run into problems, I guess most likely it's because you didn't use the Ubuntu version that I used. (Every word counts: 22.04, Minimal, x86/64)Suggestions from @missionfloyd are integrated. (Thanks!)
This article is only for learning and testing purpose with no warranty at all. Reader takes all responsibility for following any of these steps.
Google Cloud documents were licensed under the Creative Commons Attribution 4.0 License.
Distribution or deriviation of this article should include a link to its sourse (this page).
Beta Was this translation helpful? Give feedback.
All reactions