build-an-lsm-storage-engine / lesson-11.md
Lesson 11 · The write-ahead log

Surviving a torn write

A crash rarely happens between records - it happens in the middle of one. Today you make replay tolerate a half-written final record, keeping every complete record before it, so a crash mid-append costs at most the one unfinished write.

The goal

Replay all complete records from a log whose final record is truncated, and stop cleanly at the torn tail.

Start here - the target
TO DO
Scenario: Replaying a log with a truncated last record
Givena log holding complete records (Put,"apple","red") and (Put,"banana","yellow"), followed by a third record that is cut off partway through (a partial append)
Whenthe log is replayed
Thenthe memtable contains "apple" and "banana" and no third key
Andreplay completes without error - the torn tail is discarded, not treated as failure
Background

An append is not atomic: a crash can strike after some of a record’s bytes reach the disk but before the rest do, leaving a torn write at the end of the log. This is the normal, expected shape of a crash - not an exotic failure - so replay must handle it gracefully. The records before the tear are complete and fsynced, so they are safe; only the interrupted final write is lost, and it was never acknowledged.

Two things flag the torn tail: the remaining bytes are too few to hold even a record header, or the checksum from the previous lesson fails on the partial data. Either way replay’s job is to stop at the tear and keep everything before it - finishing successfully, not erroring out. This is the exact boundary that makes the crash-safety claim honest: a crash mid-write loses only the unfinished write, and the store still opens.

Make it work
// walk records until fewer bytes remain than a header needs, OR
// decodeRecord returns a corruption/short error on the tail.
// A torn TAIL is expected after a crash: stop and keep everything
// decoded so far. (Corruption in the MIDDLE is a real fault - your
// choice to surface, but the tail case must be graceful.)
CheckpointDONE
Replay recovers every complete record and gracefully drops a torn final one. Commit and stop here.