FEEDBACK WANTED: making llamafile a better drop-in replacement for OpenAI

[stlhood]

llamafile's main goal is to make it easier for developers to use open models. The project already greatly simplifies running open models. And since it's built on top of llama.cpp, llamafile also comes with OpenAI-compatible API endpoints.

But we want to do more. We want llamafile to become a viable drop-in replacement for commercial inference APIs, so that developers can easily switch from services like GPT4 to using open models and the open source ecosystem.

We want your feedback and ideas. What's holding you back today from using open models in your applications, instead of services like GPT4? What features or capabilities are missing or lacking in tools like llamafile?

(There are likely a number of issues that this project can't directly address, like the quality/performance gap between OpenAI and today's open models. But there are also probably plenty of other ways that we can make a difference!)