Unable to use GPU acceleration

[deadpipe]

@Maknee

I tried setting option(MINIGPT4_CUBLAS "minigpt4: use cuBLAS" ON) in the CMakeLists.txt.

But when i run cmake --build . --config Release,

i get this error below unfortunately : -

Any advice to deal with is highly appreciated

Originally posted by @deadpipe in #16 (comment)

[Maknee]

Sorry it hasn't been documented, but I have not tested cublas integration with minigpt4

[deadpipe]

@Maknee Do you have this on your roadmap to fix this CuBlas integration with minigpt4.cpp?

[Maknee]

Unfortunately I don't have that much time these days -- I believe that cublas could be integrated nicely just with the vicuna model from llama.cpp, which should provide a nice perf boost