build-an-lsm-storage-engine / lesson-33.md
Lesson 33 · Compaction & bloom filters

Skipping tables with the filter

Now the bloom filter earns its space. Today you consult each SSTable's filter before searching it, so a point lookup skips every table that definitely lacks the key - without changing a single answer.

The goal

Attach the bloom filter to each SSTable and use it to skip tables on a point lookup, keeping results identical.

Start here - the target
TO DO
Scenario: A lookup skips tables that cannot hold the key
Givena store whose SSTable A holds "apple" (its filter excludes "banana") and SSTable B holds "banana"
WhenGet("banana") is called
ThenA's bloom filter returns false for "banana" so A's data is never searched, and the value is read from B
AndGet returns the same values it did before the filter existed - every present key is still found
Background

The bloom filter turns “search every table” into “search only the tables that might have it.” Before probing an SSTable’s index, ask its filter MayContain - if it says no, that table provably lacks the key, so skip it entirely. Because the filter has no false negatives, skipping is always safe: you can never skip a table that actually holds the key.

Correctness is unchanged - every present key is still found, every value is the same as before - which is the tell of a good optimization: it only removes wasted work. This completes the storage engine’s internals. The engine now writes durably, reads across memory and many files with recency and deletes honored, compacts itself, and skips irrelevant files on lookup. The final chapter wraps all of it in the clean, crash-safe public API a user actually touches.

Make it work
// First wire the filter (from lesson 32) onto the table: give SSTable a `bloom`
// field and build it in OpenSSTable from the keys already in the index - no
// file-format change. Then consult it on lookup, newest table first:
func (d *DB) Get(key string) ([]byte, bool) {
// memtable first, then for each SSTable:
// if !sst.bloom.MayContain(key) { continue } // provably absent -> skip
// else search it as before
}
CheckpointDONE
Point lookups skip tables the filter rules out, with identical results. Commit and stop here.