Our mission is to provide open-source tools that enable the effective deployment and use of LLMs and ML models in production settings for developers and MLOps/LLMOps. Our key projects include:
- Resonance: A PHP framework for building IO-intensive web applications and services featuring AI capabilities, a built-in web server, and integration with llama.cpp. It leverages asynchronous PHP and Swoole.
- Paddler: An open-source load balancer and reverse proxy optimized for servers running llama.cpp. Paddler ensures efficient request distribution with a stateful load balancer that monitors server slots and health, supporting dynamic scaling.
- Structured: A tool for extracting structured data from unstructured input, using llama.cpp. It is programming language agnostic and maps data from arbitrary JSON schemas to Go structs or plain JSON.
Learn more about Resonance Framework on the offical website.
Join our community on Discord and Telegram to collaborate and stay updated!
