Topic · 8 posts
Measured latency, throughput, agent-memory recall, and cost, honest numbers from independent harnesses, not marketing.
crowsight is a small, offline embedding model that ships inside Crowkis. We didn't trust it on faith, so we made it compete with the biggest embedding APIs on the one job a semantic cache actually needs. Here's what happened.
We ran Crowkis memory against two public, hostile retrieval benchmarks, SNAP's LoCoMo and LongMemEval, on a laptop with no cloud calls. Here are the recall numbers, by question type, with the reranker on and off.
A semantic cache hit isn't free, it has to embed your query first. We measured every operation's percentiles so you know exactly what you're paying for, and where the cache engine itself is microsecond-fast.
On v0.2.2, throwing 16 threads at Crowkis got the same throughput as one. That's a real ceiling, we found it in our own harness, and here's both why it happened and how the embedding-deferral work lifts it.
Empty strings, 100 KB values, null bytes, emoji, 16 threads hammering across tenants. The stress harness throws 84 nasty checks at Crowkis and counts the cross-tenant leaks. The leak count is zero.
CDEDUP works, and at 1,340 vectors it froze the whole server for 150 seconds in our harness. Here's the honest finding, why it happens, and what it means for how you should run dedup.
A v0.2.1 bug let the HNSW index go cold under heavy write-and-flush churn, semantic search silently stopped finding neighbours. Here's the soak test that reproduces it and proves v0.2.2 fixed it.
Crowkis is a cache with a vector index, not a vector database, but it should still hold up at scale. We indexed 100K and 1M vectors and measured build time, search latency, and recall. Including where dedicated vector DBs still win.