Projects/Build a JSON Parser

Build a JSON Parser

Start with a scanner that reads one token and end with a library that parses text into a value tree, reports errors with exact line and column, serializes back to compact or pretty text, and resolves JSON Pointer paths. Every lesson is one concrete spec with exact tokens, values, output, and error positions: surrogate pairs collapse to one rune, a lone surrogate is rejected, 01 is rejected as a leading zero, a trailing comma fails at its exact position, and deep nesting hits a depth cap.

33 lessonsSmall~20 min / lessonTokenizationRecursive descentSerialization
The project

What you'll build over the next 33 lessons

Over 33 lessons you build a working JSON library from scratch: a tokenizer that reads the structural punctuators, literals, whitespace, and positions; string scanning with every escape (including surrogate pairs that combine into one rune, and lone surrogates rejected); number scanning across the full grammar (sign, no leading zeros, negative zero, fractions, exponents); a recursive-descent parser that builds a value tree of nulls, booleans, numbers, strings, arrays, and objects to any depth; error reporting that names the exact line and column of a trailing comma, a missing colon, an unterminated string, or trailing garbage; a depth guard; a compact serializer and a configurable pretty-printer proven by round-trip; and a JSON Pointer query layer that resolves paths like /a/b/0.

By the end you have an importable library whose public API parses text into values, serializes values back to text, and queries them by pointer, all demonstrated by a runnable jq-lite command that reads JSON from a built-in demo string, stdin, or a file, validates it, and pretty-prints it - printing a clear error with its position when the input is invalid.

This is a teaching-grade JSON library built around the real json.org grammar and the recursive-descent design: it parses and serializes correct JSON exactly and rejects malformed input with precise positions, and it is honest about its limits - numbers are IEEE-754 doubles (so very large integers lose precision), duplicate object keys resolve last-wins, and it targets UTF-8 text rather than every encoding a spec permits. What you finish with is the honest core that production JSON libraries extend with streaming, arbitrary-precision numbers, and richer configuration.

build-a-json-parser / lesson-01.md
Lesson 01 · A scanner for structure

A token stream that ends in EOF

Every parser starts by turning a flat string into a stream of meaningful pieces called tokens. Today you build the smallest possible scanner, one that reads nothing and reports a single end-of-input token, so the public surface exists from day one and every later lesson thickens it.

The goal

Scan an empty or all-whitespace input into a one-element token stream holding a single EOF token.

Start here - the target
TO DO
Scenario: Scanning empty input
Giventhe empty input string ""
Whenit is scanned into a list of tokens
Thenthe result is exactly one token whose kind is EOF
Andscanning the all-whitespace input " " also yields exactly one EOF token
Background

A JSON parser never works on raw characters directly. It works on tokens: the meaningful atoms of the grammar, like a left brace, a string, or a number. The first job of any parser is a scanner (also called a lexer or tokenizer) that walks the input once and produces that stream. Everything downstream reads tokens, never bytes, so the shape of a token has to exist before anything else.

Start with the simplest possible stream. An empty input has no atoms in it, but it still needs a way to say “there is nothing more here” so the parser knows when to stop. That sentinel is the EOF token, and it always sits at the end of the stream. Today the whole input is nothing (or only whitespace, which carries no meaning in JSON), so the stream is just that single EOF. Keep Scan returning a plain slice of tokens - the storage, the punctuation, the positions all arrive one lesson at a time on top of this.

Make it work
// one Kind value per token type; EOF marks the end of input
type Kind int
const ( EOF Kind = iota )
type Token struct { Kind Kind }
// the whole scanner grows around this signature
func Scan(input string) []Token {
return []Token{{Kind: EOF}}
}
CheckpointDONE
You have an importable scanner whose token stream always ends in EOF. Commit and stop here.
Scope & extensions

Where this project stops - and where to go next

A complete, well-tested strict-JSON library - a scanner, a recursive-descent parser with positioned errors and a depth guard, a compact serializer, a pretty-printer, round-trip equality, and JSON Pointer resolution - plus a runnable jq-lite CLI that validates and pretty-prints from a file, stdin, or a built-in demo. It is honest about its data model: numbers are IEEE-754 float64 (so very large integers lose precision), negative zero serializes as 0, objects keep insertion order, and duplicate keys resolve last-wins.

Extend it next
  • Give ParsePointer and Resolve the same typed, positioned errors that Parse already produces, so a bad path reports where it failed
  • Add a streaming or token-at-a-time Decoder for large documents, instead of parsing the whole input into memory at once
  • Handle arbitrary-precision and big-integer numbers so large IDs and money values do not lose precision to float64
  • Grow the jp CLI into a small query tool: multiple pointers per run, JSON Lines input, and compact or raw output modes
  • Add a configurable serializer (sorted keys, escaped non-ASCII, custom indent) and a strict mode that rejects duplicate keys instead of taking the last
Recommended reading

Books & references that go deeper

  • Introducing JSON · Douglas Crockford

    The canonical one-page grammar, with the railroad diagrams for value, string, and number that this project scans and parses. Keep it open beside every lesson.

  • The interoperability standard: what a conforming parser must accept and reject, the rules for strings, numbers, and duplicate keys, and the guidance on I-JSON that the error and limit chapters follow.

  • Crafting Interpreters · Robert Nystrom

    The scanning and recursive-descent chapters explain the tokenizer and parser structure this project mirrors - reading one token at a time, then descending through the grammar to build a tree.

  • Parsing JSON is a Minefield · Nicolas Seriot

    A survey of where real JSON parsers disagree - number edges, deep nesting, surrogate handling, trailing content - and the test corpus that inspired this project to pin the edges, not just the middle.

  • RFC 6901: JavaScript Object Notation (JSON) Pointer · P. Bryan, K. Zyp, M. Nottingham

    The short spec for the /a/b/0 path syntax and its ~0 and ~1 escapes that the query chapter implements exactly.