llmhut is building the missing infrastructure layer for LLM-native systems — from discovery and access control to runtime and beyond. Modular. Open-source. Built to last.
Every major technology wave produced a foundational infrastructure layer. The AI wave is no different — and that layer doesn't fully exist yet. llmhut is building it, piece by piece, in the open.
Audit any codebase for LLM SDK usage, AI frameworks, and exposed secrets. Continuous visibility from day one.
Per-developer, per-team, vendor-level keys. Instant revocation. Budget controls. Full audit trail. No shared credentials.
Sub-millisecond LLM gateway. Swap providers, load-balance, retry — transparently. No SDK changes required.
Observability, eval pipelines, cost attribution, deployment primitives. The stack keeps growing with the community.
Scan any codebase for LLM SDK usage, AI framework integrations, and exposed API tokens. 145 detection patterns, 23 SDKs, 24 frameworks. Zero dependencies. MCP server for Claude Code, Cursor, and Windsurf.
Blazing-fast LLM routing written in Go. Sub-millisecond overhead, zero external dependencies. Drop-in for any provider — OpenAI, Anthropic, Gemini, Groq, Bedrock. Route, retry, and load-balance without changing your SDK.
Per-developer vendor API keys — real keys, not proxy tokens. Provision, revoke, rotate with vendor-level enforcement. Budget caps, rate limits, and immutable audit log. OpenAI, Anthropic, Azure, GCP.
Building with LLMs is fragmented. Shared keys across teams. No visibility into what's running. No control over cost or audit trail. No standard way to swap providers. llmhut replaces duct tape with real infrastructure.
One cohesive stack covering the full AI lifecycle from dev to production.
Minimal setup. Works with existing SDKs. No vendor lock-in. Drop it in and go.
OpenAI, Anthropic, Gemini, Groq, Bedrock — one unified layer across all of them.
Fully open-source, extensible, community-driven. Fork it. Build on it.
No shared secrets. No proxy risk. Vendor-level enforcement from day one.
Use one layer or all. Each project works standalone or as part of the full stack.
Every mature technology wave has a defacto open-source foundation. Web got nginx, Redis, Kafka. AI deserves the same — llmhut is building it from first principles.
Join us on GitHub ★llmhut is community-driven. Every issue filed, PR merged, and pattern contributed makes the stack better for everyone building AI systems.
Show support and help others discover the projects. Every star helps the community grow.
github.com/llmhut → 🐛Found a bug or missing detection pattern? Open an issue — every report makes the tools sharper.
Open an issue → 🔧New vendor integrations, detection patterns, routing strategies — PRs are welcome across all projects.
Submit a PR →Open-source. MIT & Apache 2.0. Built with ❤️ for the AI developer community.