build-a-search-engine / lesson-15.md
Lesson 15 · The inverted index

Unioning postings

Today you merge two sorted postings lists into one that holds every document from either - the union. It is what an OR query needs, and it must not repeat ids.

The goal

Merge two sorted postings lists into their sorted, duplicate-free union.

Start here - the target
TO DO
Scenario: Documents in either of two terms
Giventhe postings ["d1", "d3"] and ["d2", "d3"]
Whenthey are unioned
Thenthe result is ["d1", "d2", "d3"]
Andthe shared "d3" appears only once
Background

A query for cat OR dog wants every document containing either term - the union of the two postings lists. The merge is a cousin of yesterday’s intersection: walk both pointers, but this time emit the smaller id each step rather than only the matches, and when one list runs out, append the rest of the other.

The trap is duplicates. When both lists point at the same id, emit it once and advance both pointers, so a document in both lists still appears a single time. Keep the output sorted and unique, and it is itself a valid postings list you can feed into the next operation.

Make it work
def union(a, b):
i = j = 0
out = []
while i < len(a) and j < len(b):
# take the smaller; on a tie, take one and advance BOTH
...
# append whatever remains in a or b
return out
Further Reading

Manning, Introduction to Information Retrieval - ch. 1.3.

CheckpointDONE
You can combine the documents of two terms without duplicates. Commit and stop here.