[question] llama support

[Tom-Neverwinter]

lets ask the real question here. can it run llama or more specifically oobabooga web ui for ai models?

AMD can have a lot of my money with their 300$ 16GB GPU'S if it can beat nvidia at their price point. [especially the m/p40 GPUs that are so easy to get on ebay atm. I'll gladly take the ram hit for speed]

<ZLUDA_DIRECTORY>\zluda.exe -- <APPLICATION_ARGUMENTS>

will it translate the entire app from the .bat in wsl or direct on windows for ease and convenience?

[I'll go a step further and bounty support if its even feasible to kickstart team red dominating nvidia, admittedly I'm not rich but I want to see this thing go!!! ZOOM ZOOM!!!]

[hchasens]

Depends on your loader. Zluda has confirmed that llama.cpp works, but that's it. I doubt you'd get anything other than llama.cpp to work, and even that will require some tinkering. It's mentioned that you need to build llama.cpp with a specific argument and it's performance is worse than llama.cpp with native AMD HIP.

I run Oobabooga fine on ROCm HIP though using Transformers. It's not as polished an experience and requires some fine tuning but I've been able to run any LLM under 13B. (In Linux anyways)