Projects/Build a SQL Database

Build a SQL Database

Start with a typed value in memory and end with an interactive SQL prompt that runs SELECT, JOIN, GROUP BY, and more. Every lesson gives you a concrete spec to hit, and the engine grows one operator at a time.

45 lessonsMedium~20 min / lessonRecursive descentRelational operatorsQuery execution
The project

What you'll build over the next 45 lessons

Over 45 lessons you build a working SQL database engine from scratch: the in-memory tables that hold typed rows, a tokenizer and recursive-descent parser that turn SQL text into a syntax tree, an execution engine that runs that tree as a pipeline of relational operators, and a durability layer that keeps your data safe across a crash.

By the end you have an interactive SQL prompt that runs a real subset of SQL - CREATE TABLE, INSERT, SELECT with WHERE / ORDER BY / LIMIT, the aggregate functions with GROUP BY and HAVING, INNER JOIN across tables, UPDATE and DELETE - and persists your data so it survives not just a clean restart but a crash mid-write, using atomic snapshots, a write-ahead log that flushes every committed mutation, and replay on open that loses nothing acknowledged.

This is a teaching-grade engine: it executes each query as written, directly and correctly, but stops short of the machinery a production database adds on top - indexes, a cost-based query optimizer, transactions and concurrency, and subqueries. What you finish with is the honest core those systems are built around.

build-a-sql-database / lesson-01.md
Lesson 01 · In-memory tables

A typed value

A database stores more than numbers - it stores integers, text, and the absence of a value. Today you build the single tagged value type that every row, expression, and result in the engine is made of.

The goal

Represent an integer, a text, and a null value in one type, and compare two values for equality.

Start here - the target
TO DO
Scenario: Comparing typed values
Givenan integer value 42 and a text value "42"
Whenthe two values are compared for equality
Thenthey are not equal (different types)
Andtwo integer values 42 are equal
Anda null value equals another null value, and equals no integer or text
Background

Every cell in a table holds a value, and a value carries a type - an integer, a string of text, or NULL, the explicit absence of data. The trick that makes the rest of the engine simple is representing all three with one type that tags which shape it currently holds, so a row is just a list of these and nothing downstream has to special-case “what kind of thing is this.”

Equality is where the type tag earns its keep: the integer 42 and the text "42" print the same but are not the same value, because comparing them must check the kind before the contents. Get this one comparison right and filtering, joining, and grouping all inherit it for free later.

Make it work
// one value, three shapes: tag says which
type Kind int
const ( KindInt Kind = iota; KindText; KindNull )
type Value struct { Kind Kind; Int int64; Text string }
// Equal compares Kind first, then the matching payload
CheckpointDONE
You have a Value that holds an integer, text, or null and knows when two values are equal. Commit and stop here.
Scope & extensions

Where this project stops - and where to go next

The engine correctly executes the standard single-table and two-table-join SQL surface (filtering, sorting, limiting, grouping with aggregates, and inner joins, all composable) with graceful error handling, and it persists crash-safely - atomic snapshots, a fsync'd write-ahead log, and replay on open recover every committed mutation after a crash - but it stops well short of real SQL: no NULL/three-valued logic, no arithmetic expressions, only single-column GROUP BY / ORDER BY / UPDATE SET, and INNER-only two-table joins via a naive nested-loop.

Extend it next
  • Add a NULL literal to the grammar and give WHERE/HAVING proper three-valued (UNKNOWN-propagating) logic plus IS NULL / IS NOT NULL
  • Add arithmetic expressions (+ - * / and parentheses) so WHERE and the SELECT list can compute values, not only compare bare columns
  • Generalize GROUP BY, ORDER BY, and UPDATE SET from a single column/assignment to comma-separated lists
  • Add LEFT/RIGHT OUTER JOIN, joins across more than two tables, and table aliases
  • Add a real fractional numeric type (float or decimal) so AVG stops truncating with integer division
  • Add LIMIT ... OFFSET and DISTINCT
Recommended reading

Books & references that go deeper

  • Crafting Interpreters · Robert Nystrom

    The clearest guide to tokenizers and recursive-descent parsers - the exact techniques the SQL front-end in this project uses.

  • Database System Concepts · Silberschatz, Korth, Sudarshan

    The standard textbook on the relational model, relational algebra, and query processing that the execution engine implements.

  • Architecture of a Database System · Joseph M. Hellerstein, Michael Stonebraker, James Hamilton

    A survey tying the query parser, planner, and execution operators into one architecture - the map for how the pieces you build fit together.

  • A short, practical walkthrough of building a query engine as a pipeline of operators, mirroring this project's execution model.

  • Readings in Database Systems (The Red Book) · Peter Bailis, Joseph M. Hellerstein, Michael Stonebraker

    Curated classic papers for going deeper into optimization, transactions, and the systems ideas this teaching engine leaves out.