Skip to content
POLARISAI ACADEMY
R07-201 · Navigator

Production RAG and Hybrid Search

You can cut four handbook sentences at a cosine of 0.80, rank four stored chunks by cosine and by a rare-word weight, fuse the two ranks, and return an empty citation when a question names a thing none of the chunks contain.

Level
Navigator
Time
26 h (estimate)
Prerequisites
R07-101
Course 2 of 4
Jump to

After this course you can

The student can build a hybrid BM25 + vector + RRF retrieval pipeline over a frozen 1,000-document corpus that achieves ≥90% Recall@5 on keyed queries and abstains on 100% of unanswerable queries.

Consecutive sentences and the cut

A chunk is one stored piece of a handbook: the words a later search is allowed to return, and no others. Semantic chunking picks the cuts from the sentences. This lesson stores each sentence as three numbers, in the same dimension order every time. An embedding is that short list of numbers. You take the cosine of sentence n and sentence n+1. Cosine is the dot product divided by the product of the two lengths. At this lesson's threshold of 0.80 the next sentence stays in the current chunk when the cosine is 0.80 or higher, and it opens a new chunk when the cosine is lower. The comparison is only between neighbors, so a sentence never joins a chunk that ended two cuts earlier.

The four sentences are these. S1: A phone repair takes 30 days. Its vector is [3, 4, 0]. S2: The phone must arrive with its charger. Its vector is [3, 4, 0]. S3: A tablet repair takes 14 days. Its vector is [0, 4, 3]. S4: The counter closes on Sunday. Its vector is [0, 0, 5]. Each vector has length 5, because 3²+4²+0² = 25, 0²+4²+3² = 25, and 0²+0²+5² = 25, and the square root of 25 is 5. The cosine of S1 and S2 is 25/25 = 1, so S2 stays with S1. The dot product of S2 and S3 is 16, so the cosine is 16/25 = 0.64, and S3 opens a new chunk. The dot product of S3 and S4 is 15, so the cosine is 15/25 = 0.60, and S4 opens a new chunk. A fixed window of 9 words, counted from the first word of S1, ends on the word must. The words arrive, with, its, and charger fall outside that window. The cosine cut keeps those four words inside the first chunk.

The stored row and the dense rank

After the cuts, the search table holds four rows. C1 stores the text of S1 and S2 and the vector [3, 4, 0]. C2 stores S3 and [0, 4, 3]. C3 stores one further sentence, A phone battery ships in 5 days, and the vector [3, 0, 4]. C4 stores S4 and [0, 0, 5]. The length of C3 is also 5, because 3²+0²+4² = 25. A vector store keeps rows of this shape and returns the ones closest to a query vector. SQLite-vec, pgvector, and Qdrant are three stores that can hold the rows. You keep the original words in a text column beside the vector, because a citation has to name a chunk id and a sentence from that row. The store compares vectors. It does not count how often a word occurs. The order it returns is the dense rank.

The query words phone sunday use the vector [3, 4, 0]. Its length is 5. The dot product with C1 is 25, so the cosine is 1. The dot product with C2 is 16, so the cosine is 0.64. The dot product with C3 is 9, so the cosine is 0.36. The dot product with C4 is 0, so the cosine is 0. The dense rank, highest cosine first, is C1, C2, C3, C4. C4 is the row that states the Sunday closing, and it still sits last, because [0, 0, 5] shares no direction with [3, 4, 0]. The word list in the next section is what can lift C4. Paste the four rows into a chat box and ask only for this dense order. Keep a returned order when it matches these four cosines, and recompute the dot products when it does not.

Rare words, then fused ranks

The lexical score is a paper rule with the same job as BM25: a rare query word is worth more than a common one. Lowercase the chunk and the query, then split on spaces. Drop these stop words if they appear: a, an, the, on, which, does, do, is, of, for, how, many, what, when, where. Count a query word once in a chunk, even when the text repeats it. Let df be how many of the four chunks contain that word. When df is 1, 2, or 3, the weight is 3 minus df, so the weight is 2, 1, or 0. When df is 0, no row contains the word, the weight is unused, and a later gate refuses the question. Sum the weights of the query words the chunk does contain. Ties break toward the smaller chunk id. In the Lucene form of BM25 the usual starting values are k1 = 1.2 and b = 0.75. That form also raises rare words, then saturates repeats, so a second copy of a word adds less than the first, and it scales a row by its length. The paper rule counts each query word once, so the sum stays an integer.

Reciprocal rank fusion, RRF, reads ranks and ignores the raw scores. With a rank that starts at 1, a chunk's fused score is the sum of 1/(k + rank) across the lists you have. The usual constant is k = 60, so a first place on one list cannot wipe out a last place on the other. Sort by that sum, high to low, and break a tie by the smaller chunk id. Adding a cosine of 1 to a lexical score of 2 would mix a ratio with a point total. RRF meets the lists only after each one has been sorted. For phone sunday, phone is in C1 and C3, so df = 2 and the weight is 1. sunday is in C4 only, so df = 1 and the weight is 2. A person marked C1 and C4 as the relevant rows, C1 because it contains phone and C4 because it contains sunday. Recall@2 is how many of those marked rows sit in the top 2, divided by 2. Recall@5 is the same ratio with a top 5. On four rows the top 2 is the cut you can check by hand. The dense top 2 is C1 and C2, which holds 1 of the 2 marked rows, so dense Recall@2 is 1/2.

The pair score and the empty citation

A bi-encoder builds one vector per text, which is the path you already used for cosine. A cross-encoder reads the query and one chunk in a single pass and emits one relevance score for that pair, while the words of both sides are still on the page. The cost grows with the number of pairs, so this lesson sends only the fused top 2. A chunk outside those 2 receives no pair score and cannot climb. A 9 written next to a row outside the shortlist stays unused. This lesson's pair scores for the phone sunday shortlist, standing in for that one pass, are 3 for C1 and 8 for C4. Sorting those two puts C4 first. The score gate cites a chunk when its pair score is 5 or higher. C4 passes and C1 stays uncited. The printed citation is [C4]. The sentence that citation can support is that the counter closes on Sunday. The 30-day phone repair stays in C1, and you do not attach it to [C4].

Abstention means you print an empty citation and add no repaired number from a neighbor row. The word gate: if any content word of the question sits in zero chunks, you cite nobody. Content words are what remains after the stop list. The question What is the watch repair price? leaves watch, repair, and price. watch is in zero chunks, and price is in zero chunks, so the gate fires. A query vector equal to [3, 4, 0] still has cosine 1 with C1, and the citation is still empty. The 30 in C1 is about a phone. Any question you form by adding one word that none of the four chunks contain gets the same empty list. Ten such questions give ten empty lists. When this search sits behind a tool, the tool returns the cited chunk ids, or that empty list when the word gate fires. The caller then has no row to quote.

An example

Phone and Sunday, both lists

Use the four stored rows and the query words phone and sunday, with query vector [3, 4, 0]. Every side has length 5. Dot products: C1 is 3·3 + 4·4 + 0·0 = 25, C2 is 0·3 + 4·4 + 3·0 = 16, C3 is 3·3 + 0·4 + 4·0 = 9, and C4 is 0. Cosines: 25/25 = 1, 16/25 = 0.64, 9/25 = 0.36, and 0/25 = 0. Dense order: C1, C2, C3, C4. Dense ranks: C1 is 1, C2 is 2, C3 is 3, C4 is 4.

Lexical weights: phone has df 2, so the weight is 3−2 = 1. sunday has df 1, so the weight is 3−1 = 2. C1 contains phone and scores 1. C2 contains neither word and scores 0. C3 contains phone and scores 1. C4 contains sunday and scores 2. The tie at 1 puts the smaller id first, so the lexical order is C4, C1, C3, C2. Lexical ranks: C4 is 1, C1 is 2, C3 is 3, C2 is 4.

RRF uses k = 60. C1 stands at dense rank 1 and lexical rank 2, so the terms are 1/61 + 1/62. 1/61 is 0.0163934426 and 1/62 is 0.0161290323. The sum is 0.0325224749, which rounds to 0.0325. C4 stands at dense rank 4 and lexical rank 1, so 1/64 + 1/61 = 0.015625 + 0.0163934426 = 0.0320184426, which rounds to 0.0320. C2 stands at dense rank 2 and lexical rank 4, so 1/62 + 1/64 = 0.0161290323 + 0.015625 = 0.0317540323, which rounds to 0.0318. C3 stands at dense rank 3 and lexical rank 3, so 1/63 + 1/63 = 0.0317460317, which rounds to 0.0317. Fused order: C1, C4, C2, C3. Both marked rows, C1 and C4, sit in the top 2, so fused Recall@2 is 1. The dense top 2 held only C1, so dense Recall@2 stayed 1/2.

The cross-encoder receives only that fused top 2, C1 and C4. The lesson pair scores are 3 and 8. The score gate is 5, so the citation printed for the query is [C4], and the supported sentence is that the counter closes on Sunday. C2 and C3 were outside the top 2, so a pair score was never produced for them. A 9 placed beside C3 would stay unused while the shortlist is C1 and C4.

Exercise

Tablet and Sunday on paper

Use the same four rows. The query words are tablet and sunday, and the query vector is [0, 4, 3]. Write the four cosines in order C1, C2, C3, C4, then the dense rank. Write the four lexical scores in that same order, then the lexical rank. Fuse with k = 1, so each term is 1/(1 + rank), and break a tie by the smaller chunk id. The marked relevant rows are C2 and C4. Write the fused order and Recall@2 for the fused top 2. Pair scores sitting on the desk are C1 = 4, C2 = 8, C3 = 1, and C4 = 9. Send only the fused top 2 through the cross-encoder, then apply the score gate of 5. Last, write the citation for What is the watch repair price?

The cosines are 0.64, 1, 0.48, and 0.60, so the dense rank is C2, C1, C4, C3. The lexical scores are 0, 2, 0, and 2, so the lexical rank is C2, C4, C1, C3. The RRF scores are 1 for C2, 7/12 for C1, 7/12 for C4, and 2/5 for C3. The tie on 7/12 puts C1 before C4. The fused order is C2, C1, C4, C3. The fused top 2 contains C2 and misses C4, so Recall@2 is 1/2. The cross-encoder scores C2 and C1 only. C4's 9 is unused. The score gate cites [C2], because 8 is at least 5, and it leaves C1 uncited because 4 is under 5. The watch question prints an empty citation, because watch is in zero chunks.

Check questions

Answer on your own first. Then open a question to see the answer.

A 9-word window starts at the first word of S1. On which word does it end, and what is the cosine of S2 with S3?

It ends on must. The cosine of S2 with S3 is 0.64, from the dot product 16 divided by 25.

For the query vector [3, 4, 0], what is the cosine of C1, and what is the cosine of C4?

C1 has cosine 1, from 25/25. C4 has cosine 0, from the dot product 0.

With k = 60 on phone sunday, what do the RRF scores of C1 and C4 round to, and what is dense Recall@2 on the marked set C1 and C4?

C1 rounds to 0.0325 and C4 rounds to 0.0320. Dense Recall@2 is 1/2, because the dense top 2 is C1 and C2.

What citation does the worked phone sunday query print, and what citation does What is the watch repair price? print?

The worked query prints [C4], because the pair score 8 passes the gate of 5. The watch question prints an empty citation, because watch is in zero chunks.

Take with you

Semantic chunking keeps a neighbor sentence when its cosine clears 0.80. The dense list ranks the stored rows by cosine, the lexical list ranks them by 3 minus df, and RRF adds 1/(60 + rank) from each list. The cross-encoder scores only that short top, and a content word found in zero chunks leaves the citation empty.

Final task

Submit a hybrid retrieval service evaluated by the automated runner against 100 frozen test queries (80 answerable with keyed chunk IDs, 20 unanswerable); must achieve ≥ 90% Recall@5, exact citation formatting, and 100% abstention on unanswerable items.

The final task is marked against fixed answer keys and automated tests once the tests open.

Terms in this course

This course has no terms of its own in the glossary yet. Terms from earlier courses are used here too.

More lessons coming

The first lesson is open to read. The rest of the course is being written, and tests and diplomas come later. Join the waitlist