Animation Workspace

[Acly]

I've had this vague idea about a UI for animations for a while, but no time to tackle it. Well, I still have no time. But I added a first draft to the latest version (v1.15.0).

Krita's animation timeline on the left, with an animated layer, which is also the active layer. The new animation workspace on the right. Clicking the button will generate images from the active layer for all frames in the animation's start/end interval. It inserts them as a new animated layer.

You may add animated control layers too. No selections/masks yet.

The single frame mode is intended for fine-tuning afterwards. It allows to edit the input layer and regenerate individual frames. Generated content is inserted into the target layer automatically. This can be any layer, but you need to create keyframes yourself.

AnimateDiff / HotshotXL is not integrated. Yet? Currently you need low strength and good constraints to get somewhat stable/consistent output. A motion model should be able to improve that. From there it would be nice to be able to leave gaps in the original animation and fill them with generated in-between frames.

I haven't looked at motion models much yet, and not sure when I will have time to commit. If sombody has good workflows (in particular img2img / start+end image) or time to experiment / add features let me know.

[arvidtappert123]

Amazing, thank you, this will help a lot!

[wormpattern]

Any idea how to implement motion brush using ComfyUI? That would be great to have it in your plugin.
Great job by the way.

[Jim5865]

The free version of DaVinci Resolve has the option of exporting video clips to individual frames, that can then be imported into Krita. Keep the number of frames low (say 25-30 frames) to experiment. My results so far have not been anything wonderful to post, but the concept works. Acly, Thank you so much for this incredible plugin. I check in every week and the additions amaze me every time. The option to delete bad generations is a well-needed addition. Here is a helpful resource, Krita Animation is tough lol
https://www.youtube.com/watch?v=5kXlrBFznBs

[Ecleptomania]

I have been trying to understand how this works for days now and no matter how I try and how many posts I read nothing makes it work or makes sense to me so I turn here in hope to get some insights / answers.

In Krita, when I select the Animation alternative, I can choose "full animation" or "single frame" just like you show above in your screenshot. But if I select, say 10 frames, write "friendly green slime" and press full animation, it gives me 10 images of the same slime for 10 frames.

I have tried using reference photos, both at start and end, I have tried it from blank canvas, and even from "live mode" where it at least allows me to do changes to each frame. But the feature "Full Animation" does nothing but generate n-Frames that are exact replicas of each other. What am I doing wrong?

Or have I misunderstood the feature somehow? What I want to be able to do (and think it does based on interface and this post) is for me to select say 20 frames, tell Krita to create whatever (A friendly smile), then do slight variations on each frame to so it actually looks like it's creating an animation and not just a still picture (with multiple frames...).

Please help me understand where I have gone wrong. Can't it be used in the manner I think? Or am I just doing something obviously wrong?

[fretts88]

I think the idea is that you start from an existing animation, and generate a new one based on it, not so much interpolation between images. I could be wrong of course.

I assume the workflow is to load an animation video or from frames, use the animation workspace , add a prompt and generate, whereby it should create a new animation layer with the AI enhanced frames.

I did find that I had accidentally left the strength at 100%, and in doing so the same frame was output, creating a static animation. What's not clear to me though is that it doesn't appear to generate a new animation layer when the strength is < 100, instead it generates separate frames to disk, which you can then re-import.

I am also struggling to constrain the output with Control net animation layers; I have generated a line-art animation using Comfyui for an animation, and referenced that in the animation generation tab, but it seems affect the aesthetics of the output, kind of mixing in the line art and the base imagery rather than constraining the edges.

[Ecleptomania]

So, If I import a gif/animation already, I could use that to "draw on" ?

Will try, but if this is the case it also answers my other question; As in, I am using the tool wrong / expecting it to do something it can't.
Will continue to try with importing a gif and your tip about not leaving the strenght at 100% but I am thinking I need to find some other way to do what I wanted to try. :)

[bobobojan]

This plugin is amazing. I greatly thank you, Acly, for releasing it.

I have been playing around with ComfyUI for a bit, and being able, not only to draw live, but also to have access to a timeline is so much more artist friendly, and I believe a game changer, since I haven't seen anything like that elsewhere.

The perspective of being able to tweak and keyframe animate all parameters to fine-tune the look rather than dealing with a batch of files, is very exciting and also very promising. I am amazed by how simple and comprehensive it becomes.

I agree with above. Adding a selections/masks, and AnimateDiff would be great additions.
I also believe accessing the workflow in Comfyui (which I haven't found) would also be a great feature to the plugin.

[guzuligo]

This needs to accept clone layers. Also, it should be able to render the final composition instead checking whether there are keyframes or not.

Animating on one layer is hard. And if you merge layers, it is no longer possible to do any adjustment.

[Marksusu]

https://github.com/tin2tin/Pallaidium

这种能在时间线上预览图像的功能是 Krita 动画需要的，现有的“故事板面板”目前也代替不了这个，Blender 原本就有视频剪辑的功能，才有插件作者发挥的空间吧

[Marksusu]

https://github.com/MinusZoneAI/ComfyUI-CogVideoX-MZ

这个也许能用来生成动画