embedding

An embedding is a vector, and “close” is an arithmetic operation on that vector. Nothing more: no understanding is involved, which explains both what the technique succeeds at and what it misses.

1. 01

   A text becomes a point

   The text is replaced by a list of numbers of fixed length, the same for a word and for a paragraph. Closeness is then measured by the angle between two lists, which allows two texts that share no word to be brought together.

   jsCopy

   embed("how do I cancel my subscription")
   // → [0.021, -0.184, 0.077, 0.003, ...]   1,024 numbers, always 1,024
   
   // closeness is an angle, not an overlap of vocabulary
   const cos = (a, b) => dot(a, b) / (norm(a) * norm(b));
   
   cos(embed("how do I cancel my subscription"),
       embed("end of contract procedure"));          // 0.83  not one word in common
   
   cos(embed("how do I cancel my subscription"),
       embed("the cat is asleep on the sofa"));      // 0.04
   

   1,024

   The dimension, fixed by the embedding model. A word and a thirty-page report give a list of the same length: the longer the text, the more the point is an averaged summary, and therefore a blurred one.

   cos

   A cosine similarity: dot product divided by the norms. That is the whole computation, and it consults nothing.

   The trapThe values above mean something only inside a single space. That is the first of the two practical rules from layer 2, and it can be seen here: a score of 0.83 compares to nothing coming from another model.

2. 02

   Why the closest is not the right one

   Here is the counter-example that governs the whole design of a serious search. Two opposite clauses speak about the same object, with almost the same words: they are therefore very close, and neighbourhood alone will choose badly.

   jsCopy

   cos(embed("remote working is allowed two days a week"),
       embed("remote working is not allowed"));          // 0.91  <- opposites
   
   // hence the two-pass ranking: recall wide, then judge expensively
   const candidates = dedupe([
     ...index.byMeaning(question, { k: 50 }),  // neighbourhood: catches rephrasings
     ...index.byWords(question, { k: 50 }),    // lexical: catches acronyms and references
   ]);
   
   // reranking reads the question AND the passage together, as a pair:
   // far more expensive, so reserved for the 100 candidates, not the corpus.
   const kept = (await rerank(question, candidates)).slice(0, 5);
   

   byWords

   The lexical search, which neighbourhood does not replace: a reference number, an in-house acronym or a rare proper noun are found through their characters, not through meaning.

   rerank

   A second model that scores the question/passage pair instead of comparing two points computed separately. That is what catches the case above, because it reads the negation.

   The trapThe upstream chunking weighs as much as the model, and the `chunking` entry deals with it. What matters here: the vector never rescues a badly placed cut, it inherits it.

What variesThe dimension, the thresholds and the interface names vary from one embedding model to the next, and the similarity values shown here are illustrative: a score only means something inside one and the same space, never compared with that of another model. What does not vary: a vector of fixed length, a similarity that measures closeness of meaning without judging relevance, and an index that has to be recomputed in full as soon as you change model.