Vulkan backend & ggml-vulkan-shaders.hpp

[mjgarton]

I am just wondering how the contents of ggml-vulkan-shaders.hpp was generated? I would like to learn about how the vulkan backend works.

If I understand correctly, this is compiled spirv shader code. Why store the compiled code as bytes here, rather than storing source and generating the binary at build time? Would this improve maintainability & transparency?

PS. I am fairly new to llama.cpp, GPUs, vulkan, shaders and ML in general, so apologies if my questions are somewhat confused.

[teleprint-me]

The shaders are generated using the ggml_vk_generate_shaders.py script. You shouldn't need to use it unless the source is modified. The shaders are included to smooth out compilation and runtime.

You can see the rationale behind most of the design decisions in the initial PR #2059.