One signed binary. Every feature compiled in. Free to run. Install Crowkis →
← back to the Roost
benchmarksJuly 18, 2026· 7 min read

We put our tiny embedding model up against OpenAI and NVIDIA. It didn't blink.

crowsight is a small, offline embedding model that ships inside Crowkis. We didn't trust it on faith, so we made it compete with the biggest embedding APIs on the one job a semantic cache actually needs. Here's what happened.

There's a quiet insecurity that comes with shipping your own model. Everyone else is calling OpenAI's text-embedding-3 or a hosted NVIDIA endpoint, and here we are with a 22-megabyte file called crowsight baked into the binary. So we did the uncomfortable thing: we set up a head-to-head and let the giants swing first.

In plain words: An embedding model turns a sentence into a list of numbers that captures its meaning. It's the part that lets a cache know 'reset my password' and 'how do I change my password' are the same question. crowsight is the one Crowkis ships with, no API call, no per-query fee.
what crowsight actually does
“how do I reset my password?” “forgot my login, help” “change account password” CROWSIGHT one meaning · one cache hit

Three phrasings, one meaning, one cache hit. That's the whole game.

The test was deliberately real. We took genuine rephrasings, the same question worded differently, the way a real user or a real agent would, and measured how reliably each model recognizes them while safely ignoring genuinely different questions. In semantic caching that trade-off is everything: catch the paraphrase (you save money) without matching a different question (you'd serve a wrong answer).

catching real rephrasings, recall at a 1% false-match rate%
crowsight (ours · offline · free)100% · 0 API cost
OpenAI text-embedding-3 (API)100%
NVIDIA nv-embedqa (API)100%
a popular open MiniLM78%

On genuine rephrasings, crowsight matched the big hosted models, and beat the usual open-source picks.

The headline isn't 'we beat OpenAI', the top models all nail this task. The headline is that a small model you run yourself, with zero per-query cost and nothing leaving your box, sits in the same tier as embedding APIs that bill you forever. crowsight also had the cleanest separation of the whole field: rephrasings scored high, different questions scored near zero. Clean separation is what keeps a cache from confidently serving the wrong answer.

The bottom line

If your semantic cache phones an embedding API for every lookup, you're paying a tax to understand questions you've already answered, and shipping every prompt off-box to do it. crowsight makes that unnecessary. It's bundled, it's offline, it's fast, and on the job that matters it keeps pace with models a hundred times its size.