How to get started

[digimbyte]

I'm an idiot with this stuff, I have a video file locally, I clone this repo, then what do I do?
does this package require other software? does it require any installation commands, etc?

[Filarius]

Are you on Windows, or Linux?
It needs FFmpeg and Python module sk-video
I wish i fixed bugs related to "there is no sk-video", so it will be installed at first (re)start.
For Win it will install FFmpeg automaticaly, for *Nix you need to install in manually

Place any video file into input of pluging, but i recommend to set path to file (I have UI lagging while uploading a video, and no lags if I set path to local file).
Choose FPS, plugin will change FPS of video as you set it, droping frames whats not fit to low FPS. Its good if you want to do a test run.
Keep input frames checkbox will make it do not drop any input frames (so you can do like of slow-mo, or speedup). Output video FPS will be set by slider.
Setting both checkboxes will make output FPS same as input (but i find there 1 frame can be dropped somewhy), ignoring FPS slider

Latent blending - makes it mix latent spacee between frames while SD doing each step.
Scale to 1.0 means frames will be mixed as 50%-to-50%,
Scale 0.5 means there 75% of current frame and 25% of previous frame
Scaling factor means the scale will be changed each step, at each step it will be multipled by this value. So its like setting this less then 1.0 will make scale be lower at final steps, so there will be less mixing of "small details" (if it can be said so) while harder mixing of "big details" at start.

[SethBonser]

do the files for this need to be copied into the automatic1111 repo and then it will just install and run automatically or do I need to do anything else to get it in the right place?

[Filarius]

do the files for this need to be copied into the automatic1111 repo and then it will just install and run automatically or do I need to do anything else to get it in the right place?

Files to run like ffmpeg and ffprobe,

• or must be available to run from any place (like open console and enter "ffmpeg"
• or must be placed in SD folder
• or for Windows now I made it to be downloaded automaticaly into SD_root/ffmpeg folder

Its also need a Python module sk-video, but i wish now it will be installed automaticaly for any platform (if available via pip for current Python version)

Video files - can be located anywhere on local PC, or even in internet to download by direct link (just use URL in thr box for file path, not actually tested, but ffmpeg must support this).

[digimbyte]

So I am aware of FFmpeg, and typically you download a dll file and host it on your main drive path.
I see comments that its auto installed with windows and a pip command for sk-video
do I run the pip command in the local directory or is that globally automatically?

As for this source, do I need to insert it into or overtop the stable diffusion webui in a particular folder?

[digimbyte]

to clarify, I ended cloning and using the extensions "install from url"
but there is no video2video tab

[Filarius]

do I run the pip command in the local directory or is that globally automatically?

I expect sk-video will be installed auto at any OS. Maybe at "install from URL" you need to restart Auto1111, i'm not sure.

As for this source, do I need to insert it into or overtop the stable diffusion webui in a particular folder?

There must be folder SD\extensions\video2video\ if you install from WEB UI
possibly you can place it manually, but update will not work.
At "old times" we put scitps into scripts folder.

to clarify, I ended cloning and using the extensions "install from url"
but there is no video2video tab

Not a "tab", but option in drop-down menu at bottom of img2img tab

[bfvogel]

This script is functioning but I am getting odd results. If I leave controlnet disabled then the output does not use any of the source video frames for reference outside of what appears to be Img2Img so there is too much drift. If I enable controlnet to get a bit more pose consistency frame to frame then the ControlNet detectmaps get injected into the video as well. The resulting video it one frame of output then a controlnet detectmap then the next frame / next detectmap and so in. I'm sure I am just missing something.