Projects/Build a TOML Parser

Build a TOML Parser

Start with a parser that reads one key = value line into a flat table and end with a library that turns a whole TOML document into a nested tree of typed values: strings, integers, floats, booleans, datetimes, arrays, and tables. Every lesson is one concrete spec with exact parsed values, structure, and error positions: a \uXXXX escape decodes to its character, a literal string keeps its backslashes, a trailing backslash trims the newline in a multiline string, 1_000 becomes 1000 and 0xDEAD becomes 57005, an offset datetime is told apart from a local date, [a.b] builds nested tables, a dotted key builds its intermediates, [[products]] appended twice gives a two-element array, and a duplicate key fails at its exact position.

32 lessonsSmall~20 min / lessonTOMLRecursive descentConfiguration formats
The project

What you'll build over the next 32 lessons

Over 32 lessons you build a working TOML library from scratch: a line reader that turns key = value pairs and # comments into a flat table; every TOML string flavor (basic strings with all their escapes including \uXXXX and \UXXXXXXXX, literal strings that keep their backslashes, and both multiline forms with the leading-newline trim and the line-ending backslash trim); the full scalar grammar (signed integers with _ separators and 0x, 0o, 0b bases; floats with fractions, exponents, and inf and nan; booleans; and RFC 3339 offset date-times, local date-times, local dates, and local times); table headers [a.b.c] and dotted keys that build a nested document tree; inline arrays, inline tables, and arrays of tables that append an element each occurrence; and error reporting that names the exact line and column of a duplicate key, a redefined table, an unterminated string, or a bad escape.

By the end you have an importable library whose public API parses TOML text into a nested tree of typed values, told apart by kind, and reports malformed input with a precise position. The finalize pass wraps it in a small runnable demo that parses a built-in configuration string or a .toml file and pretty-prints the resulting structure, printing a clear line-and-column error when the input is invalid.

This is a teaching-grade TOML 1.0 parser built around the real toml.io grammar and a recursive-descent design: it parses the value types, tables, dotted keys, arrays, inline tables, and arrays of tables that make up ordinary configuration files, and it is honest about its limits - it targets valid UTF-8 input, keeps insertion order, and stops short of a serializer and of the rarer conformance corners the official test suite probes. What you finish with is the honest core that production TOML libraries extend with encoding, richer datetime handling, and exhaustive spec coverage.

build-a-toml-parser / lesson-01.md
Lesson 01 · Keys, values, and a flat table

A document is a table

Every TOML file is, at the top, a table: a set of key/value pairs. Today you build the smallest possible parser, one that reads an input with no pairs in it and returns an empty table, so the library has a public surface from day one that every later lesson thickens.

The goal

Parse an empty or blank-only input into an empty table.

Start here - the target
TO DO
Scenario: Parsing an empty document
Giventhe empty input string, and separately an input of only blank lines and spaces
WhenParse is called on each
Theneach returns a table with zero entries and a nil error
Andthe table has an ordered list of entries and a Get(key) that reports the value is absent
Background

Every TOML document describes one table - an ordered collection of key/value pairs - and the whole file is the contents of that top-level table. So the natural public shape of the library is Parse(input) -> (*Table, error): hand it text, get back the root table. The Table keeps its entries in a slice so insertion order is preserved, which matters later when a document defines keys and subtables in a particular sequence.

Start at the degenerate case. An empty file has no pairs, and a file of only blank lines and spaces carries no meaning either, so both parse to an empty table. Define the Value union now with every Kind it will ever hold - strings, numbers, dates, arrays, and nested tables - even though today only the empty table exists; naming the whole family up front means later lessons add a case, never reshape the type. Splitting the input into lines and skipping the blank ones is the entire parser today; every value form grows on top of this loop.

Make it work
// the value kinds this library will ever hold - name them all now
type Kind int
const ( KindString Kind = iota; KindInteger; KindFloat; KindBool
KindOffsetDateTime; KindLocalDateTime; KindLocalDate
KindLocalTime; KindArray; KindTable )
type Value struct {
Kind Kind
Str string; Int int64; Float float64; Bool bool
Arr []Value; Tbl *Table // datetime fields arrive later
}
type Entry struct { Key string; Val Value }
type Table struct { Entries []Entry }
func (t *Table) Get(k string) (Value, bool) { /* linear scan */ }
func Parse(input string) (*Table, error) { /* split lines, skip blanks */ }
CheckpointDONE
You have an importable parser whose empty document is an empty table. Commit and stop here.
Scope & extensions

Where this project stops - and where to go next

A teaching-grade TOML 1.0 parser: a recursive-descent library that turns TOML text into a nested tree of typed values (all four string flavors, numbers with 0x/0o/0b bases and inf/nan, RFC 3339 datetimes, tables, dotted keys, arrays, inline tables, and arrays of tables) with positioned line-and-column errors, plus a runnable tomldump command that pretty-prints a config or fails gracefully on invalid input. It is honest about its limits: it only parses (there is no encoder), stores datetimes as unvalidated component fields rather than real time values, caps integers at 64 bits, and does not chase every toml-test conformance corner.

Extend it next
  • Add an encoder that emits canonical TOML text, so a parsed document can round-trip back to a file
  • Expose datetimes as real time values with calendar range validation, alongside the raw component fields the parser records now
  • Run the official toml-test compliance suite and close the gaps it reveals, both valid documents that should parse and invalid ones that should be rejected
  • Support arbitrary-precision and big integers so very large IDs do not overflow the 64-bit range
  • Tighten string validation to reject disallowed control characters and validate the surrogate ranges of \uXXXX escapes
  • Add a decode-into-struct convenience that maps a document onto Go-style struct fields, the way production TOML libraries expose their data
Recommended reading

Books & references that go deeper

  • TOML v1.0.0 specification · Tom Preston-Werner et al.

    The canonical grammar this project parses: keys, the string flavors, the number and datetime forms, tables, dotted keys, arrays, inline tables, and arrays of tables. Keep it open beside every lesson.

  • A TOML-compatible ABNF grammar · TOML maintainers

    The formal grammar that pins the exact character classes for bare keys, the boundaries of each value, and the multiline trim rules the string lessons implement.

  • toml-test: a language-agnostic TOML compliance suite · Caleb Meredith, Martin Tournoij

    Hundreds of valid and invalid documents with their expected parse results. The edge cases the error and semantics chapters pin are drawn from the same corners this suite probes.

  • The timestamp grammar behind TOML datetimes: the offset date-time, and the local date, time, and date-time forms the datetime lessons tell apart.

  • Crafting Interpreters · Robert Nystrom

    The scanning and recursive-descent chapters explain the reader-and-parser structure this project mirrors: consume one meaningful piece at a time, then descend through the grammar to build a tree.