Add Custom Gradio Component to have a timeline metaphor for video creation as an extension

[AllreadyTaken]

Hi. I want to create a video extension to enable people to create videoinvideo / img2video videos. I already heavily modified the existing scripts but want to combine them into an extension to enable all kind of shenanigans like previews switching between video2video and img2video, reconfiguring waht is happening in terms of prompts, seeds, and so on.

For that i want to integrate an editable timeline like that:
https://visjs.github.io/vis-timeline/examples/timeline/

There is a gradio guide about how to write extensions which is great.
https://gradio.app/creating_a_new_component/

But I don't see how to possibly integrate that into this project in a sane way.

So i would be happy for sane and also insane ideas to get that going. We need a proper ai video editor :D

[AllreadyTaken]

The way I see right now:
Build a gradio extension and create a gradio build locally.
UI Layout would be Timeline with groups , a Selection of possible Group Entries (vid2vid effects, img2vid effects ) and a configuration for every selected entry.
2 Buttons One for Preview Generation (one image per timeline group) and one for actual rendering.

On "Backend" Site a modular system to work with those groups. Mixing vid2vid and img2vid in the same Group would be out of scope for now.
This could also be utilized using off the shelf gradio elements but would be like eating microwave food when there is also the option for a five star menu. So I'm not sure if leisure me would go trough the hazzle to create such a microwaved frontend.

Problem is that then our beloved webui would need to come with a custom gradio build if i go the timeline way. Since I don't wanna go trough the hazzle to make a proper pull request for Gradio, since this is my hobby and i mostly spend that stoned hacking away.

Since I need to restrain my chaotic adhd self for 4 Days a week working in critical infrastructure with every minor task coming with a massive overhead in peer reviews, documenation, specififcations, and cooperate politics . I really really need to let the hacker chaotic spirit roam free when i work on this in order to get anything done. Else it would become a chore and leisure time me avoids chores like hell.

There seems to be a way to load custom html with gradio though and if that could be utilized to integrate custom javascript I would love to hear how. In the end its serialized json to be handed over both ways (into the custom gradio html element and from it) Anyone has that experience?

Really wanna start next weekend.

[alvar036]

Great idea, but i would love to have more minimal functions. Since exporting to video from AI is not that handy since continuity is still hard to get you'll have to use tricks like halfing frame rates and then blend frames again in AE to get a smoother result.

To me some little options like Keyframing CFG over time would be really cool, so you can choose to increase or decrease a particular effect/setting. It would also help importing mp4 files but then again i feel like working with png/jpeg sequences is fine for now.

[Yelorix]

I've integrated the vis.js Timeline library into a custom gradio component.
https://github.com/Yelorix/gradio-vis-timeline

The demo app.py code shows a hacky way for how to make it behave like an audio track editor.