Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: [Windows] CI for AMD RoCM support #9

Open
10 tasks done
hiro-v opened this issue May 1, 2024 · 0 comments
Open
10 tasks done

feat: [Windows] CI for AMD RoCM support #9

hiro-v opened this issue May 1, 2024 · 0 comments
Labels

Comments

@hiro-v
Copy link

hiro-v commented May 1, 2024

Problem

  • Lack support for AMD RoCM on Windows

Success Criteria

  • CI pipeline on hosted Github action runner
  • Cortex llama.cpp support for AMD RoCM

Steps
BUILD

  • Register github action runner for Windows AMD

  • Install Visual Studio with C++

  • Install HIP SDK for Windows

  • choco install cmake llvm ninja strawberryperl

  • PATH=C:\Program Files\AMD\ROCm\5.7\bin

  • Build command

cmake -S ./nitro_deps -B ./build_deps/nitro_deps
cmake --build ./build_deps/nitro_deps --config Release
mkdir -p build
cd build
cmake -G Ninja -DAMDGPU_TARGETS=gfx1100 -DLLAMA_HIPBLAS=ON -DCMAKE_C_COMPILER="C:\Program Files\AMD\ROCm\5.7\bin\clang.exe" -DCMAKE_CXX_COMPILER="C:\Program Files\AMD\ROCm\5.7\bin\clang++.exe" -DCMAKE_BUILD_TYPE=Release -DLLAMA_NATIVE=OFF -DLLAMA_BUILD_SERVER=ON -DLLAMA_BLAS=ON -DBUILD_SHARED_LIBS=OFF -DWHISPER_SDL2=ON -DLLAMA_HIPBLAS=ON -DAMDGPU_TARGETS="gfx803;gfx900;gfx906:xnack-;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-;gfx940;gfx941;gfx942;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102" ..
  • Replace -lm.lib -> -lmsvcrt.lib in build/build.ninja
  • Run build command after the fix: cmake --build . --config Release

INFERENCE

  • Install HIP SDK so that it has rocblas and rocm in C:\Program Files\AMD\ROCm\5.7\bin (We can discover the ability to package this and put into the same directory similar to what we did with CUDA)
  • Usage
cp build_deps/bin/zlib.dll build/
$env:HIP_VISIBLE_DEVICES = "1" # Choose external card instead of integrated card
$env:HSA_OVERRIDE_GFX_VERSION = "11.0.0" # AMD Radeon 7600 (however as the target is for multiple targets, it should be fine. However in order to make it work with 
.\nitro.exe
@hiro-v hiro-v added the type: feature request A new feature label May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant