Speeding up generation on RTX cards via Tensor cores

[Kiyos]

I’m still a noob in ML and AI stuff, but I’ve heard that Nvidia’s Tensor cores were designed specifically for machine learning stuff and are currently used for DLSS. And that got me thinking about the subject. So, I have searched the interwebz extensively, and found this one article, which suggests that there, indeed, is some way:
Making stable diffusion 25% faster using TensorRT
What do you guys think?

[grexzen]

There are a couple threads about it already. It is true, the kernels for TRT can speed up the whole process by some margin.

There are a couple implementations out there, but not sure it has made it to autos yet.

[aliencaocao]

TensorRT has nothing to do with tensor cores. If you enable fp16, you will already be using tensor cores, and this repo allows for that by default.
See #4161