build-a-csv-parser / lesson-06.md
Lesson 06 · The quoting state machine

The doubled-quote escape

If a quoted field can contain any character, it needs a way to contain a quote itself. CSV uses a doubled quote for that, and today you decode it, the last piece that makes quoted fields able to hold literally anything.

The goal

Decode a doubled double-quote inside a quoted field into a single literal quote.

Start here - the target
TO DO
Scenario: An escaped quote inside a quoted field
Giventhe quoted input he said ""hi"" with the whole thing wrapped in quotes, that is the raw characters quote h e space s a i d space quote quote h i quote quote quote
Whenit is parsed
Thenthe one field is he said "hi" with single quotes around hi
Anda quoted empty field is just two quotes, so parsing a comma "" comma (the raw characters a comma quote quote comma c) gives ["a", "", "c"]
Background

A quoted field can hold the delimiter and a newline, but it still needs to hold the one character that has special meaning inside it: the double quote. CSV’s answer is elegant and needs no backslashes. Inside a quoted field, two double quotes in a row mean one literal double quote. So ""hi"" inside quotes decodes to "hi", and the field he said ""hi"" becomes he said "hi". This doubled-quote rule is the only escape mechanism in CSV, which is part of why the format is so simple to write and so easy to get subtly wrong.

Decoding it is a one-rune lookahead inside your quoted state. When you are in a quoted field and you see a double quote, peek at the character after it. If that is also a double quote, the pair is an escaped quote: append a single quote to the field and skip past both. If it is anything else, this quote is the real closing quote of the field. That single decision handles every case, including the smallest one: a field written as just two quotes with nothing between them is a quoted empty field, which decodes to the empty string, indistinguishable in value from an absent field like the middle of a,,c.

Make it work
// in QUOTED mode, on a '"' look at the NEXT rune to disambiguate:
// if the next rune is also '"' -> it is an escaped quote:
// append one '"' and consume both
// otherwise -> this '"' is the CLOSING quote of the field
// an empty quoted field ("" then a delimiter) closes immediately -> ""
CheckpointDONE
A quoted field can contain a literal double quote via the doubled-quote escape. Commit and stop here.