Pointers for the minimum needed to embed inference in a c++ app ?

[gitced]

Hello,

Sorry for the absolute newbie question, just discovering llama.cpp.
I need to embed llama.cpp in a c++ program.
The goal is only to embark a small lm, say llama3 8B quantized, for inference only, in a self sufficient c++ app to be distributed.
So in an ideal world, i'd have my app.exe (i'm on Windows btw) file along with a gguf file and nothing more.

I took a llama.cpp release and built both llama and ggml_static projet.
I got two seemingly static libs (llama.lib and ggml_static.lib with large sizes, 1.5 and 5 Mb) in the release dir.
Are they both needed, or is llama.lib enough ?

Then for headers, llama.h is obvious, but do i also need unicode.h to use llama.lib ?
And for ggml_static (if needed), do i need all four ggml.h, ggml-alloc.h, ggml-backend.h and ggml-quants.h for inference only ?

I didn't see any gguf related static lib, so i'm wondering, what's the bare minimum i need to run gguf file ?
Is it somehow part of the llama.lib, or do in need to build the gguf projet (which doesn't seem to create any static lib) ?

Again, sorry for the very basic questions and thanks for any pointers: i'm currently just exploring the possiblities and i'd like to avoid spending hours trying to understand the example codes only to discover i can't do what i want.
If i can have a quick and dirty working prototype, i'll be reassured and will then invest the time to properly learn to use llama.cpp.

Thanks
Cedric

[FSSRepo]

Hello, first of all, llama.lib depends on ggml_static.lib. llama.lib is the library for model inference and some utilities, while ggml_static.lib is the machine learning engine and also includes the gguf file loader. The weight of these is negligible compared to having an inference library for PyTorch CPU-only, which weighs 2GB.

If you need all the headers llama.h, common.h, you can refer to the simple example as a reference for minimal inference.

[gitced]

Thank you very much.

[wilderfield]

Whatever main.exe links is what your application would need correct?

common.lib llama.lib threading library

https://gitenterprise.xilinx.com/VitisAI/llama.cpp/blob/319c6d354c1ae6e59eb1ae012fbbe37df35ea370/examples/main/CMakeLists.txt#L4

[gitced]

Oh great ! Silly me, i didn't think about looking into the CMAkeLists.txt files.
Thanks a lot for the answer and the method.

[gitced]

Finally found the time to test everything, it was quite straight forward and i have to say it's kind of amazing to be able to use an lm with a exe that is 1Mb.

It's really hard to believe !

So just in case any absolute newbie like me wonders what's the bare minimum needed for an inference, here's what i did to compile and use simple.cpp in VS.
1/ create an empty project
2/ add a cpp file and copy paste the content of simple.cpp
3/ add an include dir in your project and copy those headers:

4/ add a lib dir in your project and copy those libs:

5/ declare everything in your project properties (compiler/linker additional directories), compile, done.

I'm was quite surprised to discover that ggml_static.lib was actually not needed !
Maybe Simple is too simple to need ggml (don't know if that makes sense, will probably sound ridiculous a few weeks from now when i start getting to know LLCPP better^^).

Anyway, thanks everyone for your answers, and hope this little noob thread will help some future beginners :-)
Cheers
Cedric