A drop-in OpenAI-compatible AI gateway with a semantic cache in front
Point your existing OpenAI client at Crowkis and change nothing else. Repeated questions are served from cache with no upstream call, and you get retries and routing for free.
The lowest-friction way to put a cache in front of your model is to not change your code at all. Crowkis exposes an OpenAI-compatible POST /v1/chat/completions, so you point your client's base URL at Crowkis and every request flows through a semantic cache on its way to the provider.
On a hit, the gateway answers from cache with no upstream call and no token cost, and marks the response with an x-crowkis-cache: hit header so you can measure savings. On a miss, it forwards upstream, caches the result, and returns it. It's off by default, there's zero egress until you set CROWKIS_GATEWAY=1 and an upstream.
- 1your OpenAI client
- 2Crowkis gateway
- 3cache hit, served free
- 4miss, forward upstream + cache
Change one base URL; the cache, retries, and routing come along.
Around the cache, the gateway adds the operational layer a raw provider call lacks: weighted multi-provider routing with fallback on error class, and automatic retries with exponential backoff and jitter that honour Retry-After. It's the adoption path for teams that don't want to learn a cache API, keep your client, change one URL.
The best integration is the one that doesn't ask you to rewrite anything.