I built Inference Gateway to simplify how AI requests are routed across different LLM providers.
It speaks the OpenAI API but works with Anthropic Claude, Google Gemini, Groq, Ollama, and more. It supports streaming, function calling, tool execution, and context injection. You can deploy it anywhere, including Kubernetes.
It’s designed to be vendor-neutral, easy to extend, and production-ready.
Happy to get feedback or suggestions. Source code is here:
I built Inference Gateway to simplify how AI requests are routed across different LLM providers.
It speaks the OpenAI API but works with Anthropic Claude, Google Gemini, Groq, Ollama, and more. It supports streaming, function calling, tool execution, and context injection. You can deploy it anywhere, including Kubernetes.
It’s designed to be vendor-neutral, easy to extend, and production-ready.
Happy to get feedback or suggestions. Source code is here:
https://github.com/inference-gateway/inference-gateway