You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The cpu mirror can run the 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' model normally, but using the gpu mirror to run this model will report the rpc error, can you help to point out?
#3169
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
报错信息如下:
3:12AM INF Trying to load the model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with the backend '[llama-cpp llama-ggml gpt4all llama-cpp-fallback piper rwkv stablediffusion whisper huggingface bert-embeddings /build/backend/python/openvoice/run.sh /build/backend/python/vall-e-x/run.sh /build/backend/python/mamba/run.sh /build/backend/python/bark/run.sh /build/backend/python/diffusers/run.sh /build/backend/python/autogptq/run.sh /build/backend/python/exllama/run.sh /build/backend/python/petals/run.sh /build/backend/python/transformers/run.sh /build/backend/python/sentencetransformers/run.sh /build/backend/python/vllm/run.sh /build/backend/python/sentencetransformers/run.sh /build/backend/python/parler-tts/run.sh /build/backend/python/coqui/run.sh /build/backend/python/rerankers/run.sh /build/backend/python/exllama2/run.sh /build/backend/python/transformers-musicgen/run.sh]'
3:12AM INF [llama-cpp] Attempting to load
3:12AM INF Loading model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with backend llama-cpp
3:12AM INF [llama-cpp] attempting to load with AVX2 variant
3:12AM INF [llama-cpp] Fails: could not load model: rpc error: code = Canceled desc =
3:12AM INF [llama-cpp] Autodetection failed, trying the fallback
3:12AM INF Loading model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with backend llama-cpp-avx2
3:12AM INF [llama-cpp] Fails: could not load model: rpc error: code = Canceled desc =
3:12AM INF [llama-ggml] Attempting to load
3:12AM INF Loading model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with backend llama-ggml
3:13AM INF [llama-ggml] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
3:13AM INF [gpt4all] Attempting to load
3:13AM INF Loading model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with backend gpt4all
3:13AM INF [gpt4all] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
3:13AM INF [llama-cpp-fallback] Attempting to load
3:13AM INF Loading model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with backend llama-cpp-fallback
3:13AM INF [llama-cpp-fallback] Fails: could not load model: rpc error: code = Canceled desc =
3:13AM INF [piper] Attempting to load
3:13AM INF Loading model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with backend piper
3:13AM INF [piper] Fails: could not load model: rpc error: code = Unknown desc = unsupported model type /build/models/Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf (should end with .onnx)
3:13AM INF [rwkv] Attempting to load
3:13AM INF Loading model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with backend rwkv
3:13AM INF [rwkv] Fails: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF
3:13AM INF [stablediffusion] Attempting to load
3:13AM INF Loading model 'Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf' with backend stablediffusion
3:13AM INF [stablediffusion] Loads OK
Beta Was this translation helpful? Give feedback.
All reactions