Simple Docker script to create an OpenAI compatible server using llama-cpp-python on port 8000.
Build the Docker Container
docker build -t llm-server .
Run the Docker container
docker run -d --name llmserver -p 8000:8000 -v $PWD/config:/home/config -v $PWD/models:/home/models llm-server
You can change config.json to change port or model information. Supports multiple models, check llama-cpp-python documentation.