What
A local LLM runtime (Ollama) exposed through a FastAPI service, packaged with Docker. Send a prompt over HTTP, get a completion back, with experimental streaming.
Why
Hosted LLM APIs are fast and easy, but they cost money, add an external dependency, and take away control. This explores the other path: keep the model on your own box.
Usage
git clone https://github.com/ak3ph-labs/llm-in-a-box
cd llm-in-a-box
docker compose up