feat: nix package management

[collinarnett]

This PR adds Nix as a method of installing and running LLM-VM. With this PR, as long as the flake experimental feature is enabled with Nix, users can easily run the LLM-VM server with:

$ nix run github:collinarnett/LLM-VM/nix

Users can also run the server with cuda support baked in:

$ nix run --impure github:collinarnett/LLM-VM/nix#llm_vm_server-cuda

(the impure is needed here in order for nixGL to find the host nvidia drivers)

Developers also benefit from this PR with the ability to start a shell with a python environment including LLM-VM.

$ nix develop github:collinarnett/LLM-VM/nix                    

[collin@vampire:~]$ python
Python 3.11.6 (main, Oct  2 2023, 13:45:54) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from llm_vm.client import Client

From here tests can be run by starting the server with the first command then running the tests in the developer shell.

In total, the following outputs are provided.

$ nix flake show github:collinarnett/LLM-VM/nix
github:collinarnett/LLM-VM/2cb0b872c4cbec7ea655c6bb4ec178d6e69f9bd2
├───apps
│   └───x86_64-linux
│       ├───default: app
│       ├───llm_vm_run_agent: app
│       ├───llm_vm_run_agent-cuda: app
│       ├───llm_vm_run_agent-rocm: app
│       ├───llm_vm_run_agent_backwards_chaining: app
│       ├───llm_vm_run_agent_backwards_chaining-cuda: app
│       ├───llm_vm_run_agent_backwards_chaining-rocm: app
│       ├───llm_vm_run_agent_flat: app
│       ├───llm_vm_run_agent_flat-cuda: app
│       ├───llm_vm_run_agent_flat-rocm: app
│       ├───llm_vm_run_agent_rebel: app
│       ├───llm_vm_run_agent_rebel-cuda: app
│       ├───llm_vm_run_agent_rebel-rocm: app
│       ├───llm_vm_server: app
│       ├───llm_vm_server-cuda: app
│       └───llm_vm_server-rocm: app
├───devShells
│   └───x86_64-linux
│       ├───cuda: development environment 'nix-shell'
│       ├───default: development environment 'interactive-python3-3.11.6-environment'
│       └───rocm: development environment 'nix-shell'
└───packages
    └───x86_64-linux
        ├───cuda: package 'python3.11-llm_vm-0.1.55'
        ├───default: package 'python3.11-llm_vm-0.1.55'
        └───rocm: package 'python3.11-llm_vm-0.1.55'

I would be grateful if others could review this PR by installing nix via the The Determinate Nix Installer and running the commands I included above. I do not have a rocm enabled video card so I would appreciate for those that do to leave their feedback.

Also let me know your general experience. There may be things I can do to make things better.

[collinarnett]

One more thing. Some of the executables installed from LLM-VM don't work due to import errors.

  File "/nix/store/02w0r9mqq5z19gmzxf7qgcp8m4cnlypc-python3.11-llm_vm-0.1.55/lib/python3.11/site-packages/llm_vm/agents/FLAT/models/utils/tool_picker_model/tool_picker_model_data.py", line 25, in __get_toolpicker_model
    json_data: List[QuestionSplitModelJSONData] = json.load(open(file_path, "r"))["data"]
                                                            ^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/nix/store/02w0r9mqq5z19gmzxf7qgcp8m4cnlypc-python3.11-llm_vm-0.1.55/lib/python3.11/site-packages/llm_vm/agents/FLAT/models/raw_data/toolpicker_data.json'

I imagine this issue is limited to the agents since they are run relative to the project root in the README.

[SomeoneSerge]

https://github.com/numtide/nix-gl-host/ is often a much simpler solution for FHS distributions (it just uses driver libraries from the host). Which is only to say that hardcoding nixGL is not necessarily a good idea. Note that NixOS users would want to skip the wrappers altogether

Sidenote: maybe we should make it possible to use both nixGL and nixglhost as "bundlers"

[collinarnett]

My assumption was that nixglDefault would try to find the drivers and use them if available. From what I can see on both READMEs they're trying to accomplish the same thing, although I don't know too much about nix-gl-host.

[SomeoneSerge]

• nixglhost uses whatever drivers deployed by the host system (e.g. if you run it on ubuntu, it'll find and use the drivers deployed by ubuntu); sometimes using the host system's libraries is the only choice (e.g. nvidia jetsons).
• nixGL tries to choose a nixpkgs attribute which would provide the equivalent drivers but packaged by nix (and patchelfed to use the same libc &c as your project).

There are pros and cons for both, they strike different tradeoffs.

[SomeoneSerge]

Also,

(the impure is needed here in order for nixGL to find the host nvidia drivers)

I think you could avoid --impure. I think the only reason it might need impure is for <nixpkgs>. The nixpkgs revision I think you want to use for the drivers is the revision compatible with your project, i.e. the same nixpkgs you pin in the flake.lock. So it should be ok to just pass it