API Access

[j-taptok]

Is there a way to use OpenWebUI as an API endpoint?

For example do similar request via API to the ones we do on the UI?

Including on the call references to uploaded documents. I can't find on the documentation anything for the API.

[TheMasterFX]

I have that question too.
So each user can generate a private API key. When I try to access the documentend endpoint e.g. curl http://myopenwebui.server:3000/ollama/api/tags I get {"detail":"Not authenticated"}
When I use the API key as bearer I get the models listed curl -H "Authorization: Bearer sk-5325fa5974804956a854d8092681bd50" http://alma-docker-vm-01.fritz.box:3000/ollama/api/tags I get the models listed:

{"models":[{"name":"codellama:code","model":"codellama:code","modified_at":"2024-03-16T11:42:54.2039394+01:00","size":3825910440,"digest":"fc84f39375bcfe7612f7636a681ebb13d54eb4466e6ea6da07b5d1c37b49994d","details":{"parent_model":"","format":"gguf","family":"llama","families":null,"parameter_size":"7B","quantization_level":"Q4_0"},"urls":[0]},{"name":"dolphin-mistral:latest","model":"dolphin-mistral:latest","modified_at":"2024-04-11T23:08:36.4088403+02:00","size":4108940323,"digest":"5dc8c5a2be6510dcb2afbcffdedc73acbd5868d2c25d9402f6044beade3d5f70","details":{"parent_model":"","format":"gguf","family":"llama","families":["llama"],"parameter_size":"7B","quantization_level":"Q4_0"},"urls":[0]},
...
}

Great. But actually the generate doesn't work

curl -X POST -H "Authorization: Bearer sk-5325fa5974804956a854d8092681bd50" http://myopenwebui.server:3000/ollama/api/generate -d '{ "model": "dolphin-mistral:latest", "prompt": "Why is the sky blue?", "stream": false }'

I get

{"detail":[{"type":"model_attributes_type","loc":["body"],"msg":"Input should be a valid dictionary or object to extract fields from","input":"{ \"model\": \"dolphin-mistral:latest\", \"prompt\": \"Why is the sky blue?\", \"stream\": false }","url":"https://errors.pydantic.dev/2.7/v/model_attributes_type"}]} 

According to the documentation it should behave like the original endpoint.

If I do reverse-proxy as mentioned here:
#261
and here
#256

The Whitelist filter of openwebui is basically useless.

[dbenoot33]

curl -X POST -H "Authorization: Bearer sk-5325fa5974804956a854d8092681bd50" http://myopenwebui.server:3000/ollama/api/generate -d '{ "model": "dolphin-mistral:latest", "prompt": "Why is the sky blue?", "stream": false }'

add : -H 'Content-Type: application/json'

Here is the complete curl command with the headers and JSON request body:

curl -X POST -H "Authorization: Bearer sk-5325fa5974804956a854d8092681bd50" -H 'Content-Type: application/json' http://myopenwebui.server:3000/ollama/api/generate -d '{ "model": "dolphin-mistral:latest", "prompt": "Why is the sky blue?", "stream": false }'

And here is the expected response from LLaMA (I used llama3 for the example):

curl -X POST -H "Authorization: Bearer sk-5325fa5974804956a854d8092681bd50" -H 'Content-Type: application/json'  http://myopenwebui.server:3000)/ollama/api/generate -d '{ "model": "llama3", "prompt": "Why is the sky blue?", "stream": false }'
{"model":"llama3","created_at":"2024-04-30T00:37:17.390541379Z","response":"The sky appears blue to our eyes because of a phenomenon called Rayleigh scattering. This scattering occurs when sunlight interacts with tiny molecules of gases in the Earth's atmosphere, such as nitrogen (N2) and oxygen (O2).\n\nHere's what happens:\n\n1. **Sunlight**: The sun emits white light, which is made up of all the colors of the visible spectrum (red, orange, yellow, green, blue, indigo, violet).\n2. **Atmospheric molecules**: When this sunlight enters the Earth's atmosphere, it encounters tiny molecules of gases like nitrogen and oxygen.\n3. **Scattering**: The shorter wavelengths of light (like blue and violet) are scattered more than the longer wavelengths (like red and orange). This is because the smaller molecules are better at scattering shorter wavelengths.\n\nAs a result:\n\n* Blue light is scattered in all directions by the atmospheric molecules, making it reach our eyes from all parts of the sky.\n* The other colors of sunlight (red, orange, yellow, green) continue to travel in more or less straight lines, reaching our eyes primarily through a smaller range of angles.\n\nThis scattering effect is stronger at shorter wavelengths (like blue), so we see more blue light being scattered towards us. That's why the sky appears blue during the daytime!","done":true,"context":[128006,882,128007,198,198,10445,374,279,13180,6437,30,128009,128006,78191,128007,198,198,791,13180,8111,6437,311,1057,6548,1606,315,264,25885,2663,13558,64069,72916,13,1115,72916,13980,994,40120,84261,449,13987,35715,315,45612,304,279,9420,596,16975,11,1778,439,47503,320,45,17,8,323,24463,320,46,17,570,198,198,8586,596,1148,8741,512,198,16,13,3146,31192,4238,96618,578,7160,73880,4251,3177,11,902,374,1903,709,315,682,279,8146,315,279,9621,20326,320,1171,11,19087,11,14071,11,6307,11,6437,11,1280,7992,11,80836,570,198,17,13,3146,1688,8801,33349,35715,96618,3277,420,40120,29933,279,9420,596,16975,11,433,35006,13987,35715,315,45612,1093,47503,323,24463,13,198,18,13,3146,3407,31436,96618,578,24210,93959,315,3177,320,4908,6437,323,80836,8,527,38067,810,1109,279,5129,93959,320,4908,2579,323,19087,570,1115,374,1606,279,9333,35715,527,2731,520,72916,24210,93959,627,198,2170,264,1121,1473,9,8868,3177,374,38067,304,682,18445,555,279,45475,35715,11,3339,433,5662,1057,6548,505,682,5596,315,279,13180,13,198,9,578,1023,8146,315,40120,320,1171,11,19087,11,14071,11,6307,8,3136,311,5944,304,810,477,2753,7833,5238,11,19261,1057,6548,15871,1555,264,9333,2134,315,27030,627,198,2028,72916,2515,374,16643,520,24210,93959,320,4908,6437,705,779,584,1518,810,6437,3177,1694,38067,7119,603,13,3011,596,3249,279,13180,8111,6437,2391,279,62182,0,128009],"total_duration":6648274911,"load_duration":2705329,"prompt_eval_duration":66941000,"eval_count":262,"eval_duration":6532971000}(

[j-taptok]

@dbenoot33 how can I add a RAG file that is already processed and part of Open Web UI to the request? I can't find the documentation of the API.

Also is this safe for a production environment? Around 10k requests per day.

[JJT30]

Is there a way to log the messages/prompts requested via user API in Open Web UI Database? I see in the chat history only the chat logs of users made in GUI and not via the API. I am wondering if it is already implemented?

[Halbstein]

Thank you! Saved me a lot of time!

[kroonen]

You can use Ollama's API Server to make request as this is the llama.cpp wrapper used.

[TheMasterFX]

No doubt, but how to use the OpenwebUI API Server with the individual API key, including the usage of the whitelist of OpenwebUI?

[Halbstein]

... and maybe it is possible to use the Open AI models as well (if the OpenAi key is entered in OpenWeb UI?