gfx90c not found in linux

[haol666]

What is the issue?

I‘m using ollama:rocm docker，gpu gfx90c，
I build ollama-amd docker，but gpu not found

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

0.46

[likelovewant]

It's not necessary to build a docker in linux for gfx90c. the upstream ollama official repo linux version would be fine . only need peroper env set eg,HSA_OVERRIDE_GFX_VERSION=9.0.0, However there are extra steps need to be done as explained https://github.com/ollama/ollama/blob/main/docs/docker.md#amd-gpu , also need extra rocm and Environment="HSA_OVERRIDE_GFX_VERSION=9.0.0 or gfx9.0.x ( some rocm version remove the gfx900 from supports lists)
refer to "ollama#6336 (comment)

I hope this info can help you . otherwise , there is a complex steps , build linux version for gfx90c rocmlibs and installed in your docker rocm and use it as normal rocm supported gpu
@haol666

[haol666]

used HSA_OVERRIDE_GFX_VERSION=9.0.0,
time=2024-12-03T15:24:29.430Z level=INFO source=images.go:753 msg="total blobs: 0" time=2024-12-03T15:24:29.430Z level=INFO source=images.go:760 msg="total unused blobs removed: 0" time=2024-12-03T15:24:29.430Z level=INFO source=routes.go:1248 msg="Listening on [::]:11434 (version 0.0.0)" time=2024-12-03T15:24:29.431Z level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu_avx cpu_avx2 cuda_v11 cuda_v12 cpu]" time=2024-12-03T15:24:29.431Z level=INFO source=gpu.go:221 msg="looking for compatible GPUs" time=2024-12-03T15:24:29.435Z level=WARN source=amd_linux.go:61 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2024-12-03T15:24:29.436Z level=WARN source=amd_linux.go:440 msg="amdgpu detected, but no compatible rocm library found. Either install rocm v6, or follow manual install instructions at https://github.com/ollama/ollama/blob/main/docs/linux.md#manual-install" time=2024-12-03T15:24:29.436Z level=WARN source=amd_linux.go:345 msg="unable to verify rocm library: no suitable rocm found, falling back to CPU" time=2024-12-03T15:24:29.436Z level=INFO source=gpu.go:386 msg="no compatible GPUs were discovered"

[likelovewant]

'verify rocm library: no suitable rocm found, falling back to CPU"'

Try set 'HSA_OVERRIDE_GFX_VERSION=9.0.0' to others like '9.0.6'or others you currently rocm supported arches.

Also there are some similar issue for your reference.

ollama#7723 (comment)

ollama#7817 (comment)

[Silverbullet069]

Please add AMD Ryzen 7 4800H with gfx90c-xnack- inside AMD-GPU-Arches-lists-Info

[likelovewant]

Please add AMD Ryzen 7 4800H with gfx90c-xnack- inside AMD-GPU-Arches-lists-Info

thansk for sharing the info . added .
@Silverbullet069

[Silverbullet069]

thansk for sharing the info . added . @Silverbullet069

@likelovewant First, I would like to express my gratitude to you for creating all the amazing ROCm libraries for lesser-known AMD GPUs architecture to provide GPU support for GPU-intensive application like Ollama. For the last few days I've dug quite a big rabbit hole to solve my problems: running Ollama Docker ROCm on APU Vega 8 using gfx90c.

I've managed to solve the problem by allocating 4GB of RAM for UMA Frame Buffer and add environment: - HSA_OVERRIDE_GFX_VERSION=9.0.0 to my docker-compose.yml file. APUs don't have dedicated VRAM like PC GPUs so it must be occupied from inside on-board RAM. How foolish of me to only allocate 1GB of UMA Frame Buffer, Docker Ollama ROCm didn't show warning or error in this situation, instead it just silently use 100% CPU when I check with ollama ps. Now after apply the workaround, this is my result:

NAME                                       ID              SIZE      PROCESSOR    UNTIL              
hf.co/arcee-ai/Arcee-VyLinh-GGUF:latest    a593a4782663    4.0 GB    45% CPU/55% GPU     3 minutes from now

This is my best result until now. Unfortunately, it's still not perfect. Sometimes my whole Plasma Shell crashed (I'm using Fedora KDE 41 Wayland btw) after I run a model, luckily my machine didn't reboot. I've tried to download a pre-built ROCmlib for Windows that supports gfx90c: rocm.gfx90c.for.hip.sdk.6.1.2.7z from your likelovewant/ROCmLibs-for-gfx1103-AMD780M-APU repository, extract its rocm.gfx90c.for.hip.sdk.6.1.2/rocblas/library and replace with /usr/lib/ollama/rocblas/library inside Docker Ollama ROCm. At last, the GPU can be detected! But when I test ran a model, it said:

Could not load /usr/lib/ollama/rocblas/library/TensileLibrary_lazy_gfx90c.dat
...
rocBLAS error: Could not initialize Tensile library

It must be because this ROCmlib gfx90c is built specifically for Windows and NOT for Linux. This is a potential solution, I didn't need to override HSA_OVERRIDE_GFX_VERSION=9.0.0 anymore.

there is a complex steps , build linux version for gfx90c rocmlibs and installed in your docker rocm and use it as normal rocm supported gpu

I would like to learn about "building Linux version for gfx 90c ROCmlibs and install in Docker Ollama ROCm", can you help me? There is a guide https://github.com/likelovewant/ROCmLibs-for-gfx1103-AMD780M-APU/wiki, but it is a little too advanced for me.

[likelovewant]

This is an issue for ROCm build and it's not a good choice to build it for an old arches unless you want to learn how to build it . anyway , like windows version , linux version also require rocblas.dll called librocblas.so to connnect library with rocm runtime .
options : using map technic as explained here ollama#2503 (comment)

or build it . For linux build ,please follow the official guide https://rocm.docs.amd.com/projects/rocBLAS/en/latest/install/Linux_Install_Guide.html , except when you make everything ready as explain and also follow the guide has rocm available .make sure build by run

python rmake.py -a "gfx90c:xnack-" --lazy-library-loading --no-merge-architectures -t "/home/file/Tensile-rocm-5.7.0"

something like this . otherwise ,it will build a bunch of arches you never needed .
The wiki guide for windows will give you some logic sense to build ,the same rule ,except no need patch things . simply edit info add your arches in those other arches show.

A bit trick for gfx90c , you may need get some file from here . grab the logic file to place in rocBLAS\library\src\blas3\Tensile\Logic\asm_full and delete some file as explained in the build guide for gfx90c only .

During the process you may across a lot of issue , try get some help from ai or chatgpt based on the terminal error log ,they will help you to get it done .

Note : rocblas and tensile version should compatible with your currently rocm version .otherwise , it's may not able to load the library also.

@Silverbullet069

[Silverbullet069]

This is an issue for ROCm build and it's not a good choice to build it for an old arches unless you want to learn how to build it . anyway , like windows version , linux version also require rocblas.dll called librocblas.so to connnect library with rocm runtime . options : using map technic as explained here ollama#2503 (comment)

or build it . For linux build ,please follow the official guide https://rocm.docs.amd.com/projects/rocBLAS/en/latest/install/Linux_Install_Guide.html , except when you make everything ready as explain and also follow the guide has rocm available .make sure build by run

python rmake.py -a "gfx90c:xnack-" --lazy-library-loading --no-merge-architectures -t "/home/file/Tensile-rocm-5.7.0"

something like this . otherwise ,it will build a bunch of arches you never needed . The wiki guide for windows will give you some logic sense to build ,the same rule ,except no need patch things . simply edit info add your arches in those other arches show.

A bit trick for gfx90c , you may need get some file from here . grab the logic file to place in rocBLAS\library\src\blas3\Tensile\Logic\asm_full and delete some file as explained in the build guide for gfx90c only .

During the process you may across a lot of issue , try get some help from ai or chatgpt based on the terminal error log ,they will help you to get it done .

Note : rocblas and tensile version should compatible with your currently rocm version .otherwise , it's may not able to load the library also.

@Silverbullet069

Okay, I will try my luck with the symlinking technique first. Building the libs will be my last solution now.

[Silverbullet069]

I've failed, miserably. I will stop digging my rabbit hole and leave this bug to someone else. Maybe it's better to buy a supported GPU with much more VRAM.

[likelovewant]

sure ,it is . it's easy but seems complex for a starter .Had to wipe linux system away due to recent Windows11 24h2 update broken the dual boot system ,sorry cannot help with it . HSA_OVERRIDE_GFX_VERSION still the best option so far for those old arches. A spported and more varm cards would be a plus.

[likelovewant]

Tips for HSA_OVERRIDE_GFX_VERSION in linux might fix this issue .