engineeringJune 22, 2026· 6 min read

HNSW explained: finding the needle in a million haystacks

Once meaning is a point in space, the hard part is finding the nearest point out of millions, fast. HNSW is the elegant trick that makes it feel instant, here's the intuition.

Say you've turned a million questions into a million points, and a new question arrives. To reuse an answer, you need its nearest neighbour, the closest existing point. The naive way is to measure the distance to all million and take the smallest. That's correct, and it's far too slow to sit in front of a live request. You need the answer in under a millisecond.

a few hops, not a million comparisons

Start anywhere, walk toward the query, arrive in a handful of steps.

In plain words: HNSW builds a graph where each point is linked to its neighbours, with a few long-range shortcuts layered on top. To search, you start somewhere and keep hopping to whichever neighbour is closer to your query, arriving in a few steps instead of a million comparisons.

The magic is in those layers. The top layer has sparse, long-range links that get you into the right neighbourhood in a couple of jumps; lower layers have dense, local links that home in on the exact nearest point. It's like using highways to reach the right city, then streets to find the address. That structure is why vector search can be both accurate and effectively instant, and why we built our own to keep it that fast under real load.

The bottom line

HNSW turns 'compare against everything' into 'take a few smart hops.' It's the reason a cache can find the nearest meaning among millions before your model would have even started typing.