bark.cpp is not using gpu

[sasatte]

inspite of turning on GGML_CUBLAS it is not using gpu. When running bark, it still says using CPU backend. How to force it to use gpu? there is no --cuda option.

[mfrederico]

I was wondering the same - but found this in the issues: #165

[soarek94]

have you checked the number of n_gpu_layers? for me, that's where the problem is coming from cause if the number of the gpu layers is not greater than 0, it will run on CPU

[axelatsiemens]

Same issue, in the bark_context n_gpu_layers is default initialized to 0, but then neither main nor server offer to possibility to set it. What should it be? does it need to be calculated form the model?

[newpolygons]

Im just getting here. So im assuming the Metal support check box in the readme just isnt true? Has anyone been able to implement the feature themselves or will I have to do more digging.

[vinovo]

@PABannier I see that n_gpu_layers is configurable only via examples, but if we want to build bark as a dynamic library, n_gpu_layers is not exposed via bark.h