Projects/Build a URL Parser

Build a URL Parser

Every value is checkable because the whole project is anchored to RFC 3986, right down to the Appendix C reference-resolution table against base http://a/b/c/d;p?q. Each lesson is one concrete spec with exact components, decoded bytes, and resolved URLs: a full URL split five ways and one missing its authority, a scheme-relative //host/path, a bracketed IPv6 host with a port, %20 decoding to a space and a lowercase %2f, remove_dot_segments turning /a/b/../c into /a/c, a default port dropped in normalization, + becoming a space in a query value, and the exact resolved results like g against the base giving http://a/b/c/g and ../../../g giving http://a/g.

25 lessonsSmall~20 min / lessonRFC 3986URI parsingPercent-encoding
The project

What you'll build over the next 25 lessons

Over 25 lessons you build a working URL/URI parser straight from RFC 3986, the standard that every browser, HTTP client, and web framework relies on. You start with the generic syntax - the algorithm that splits any URI into scheme, authority, path, query, and fragment - then break the authority down into userinfo, host, and port (handling reg-names, IPv4 addresses, and IPv6 literals in brackets), implement percent-encoding and decoding against the unreserved and reserved character sets, run the remove_dot_segments algorithm and syntax-based normalization, parse a query string into key/value pairs, and finish with the centerpiece: the Section 5 reference-resolution algorithm that resolves a relative reference against a base URI into an absolute one.

You build it as an importable library whose public API - parse a string into a URI, percent-encode and decode, normalize, parse a query, and resolve a reference against a base - is exactly what you would reach for when working with URLs in a real program. Because everything is pinned to RFC 3986, the values are all hand-checkable: the capstone reproduces the exact Appendix C examples, resolving references like g, ../g, and ../../../g against the base http://a/b/c/d;p?q to their precise absolute results.

This is a teaching-grade RFC 3986 parser: it parses, normalizes, and resolves URIs exactly as the RFC specifies, and it is honest about its limits - it implements RFC 3986 generic syntax rather than the WHATWG URL Standard that browsers use for web addresses (with its own normalization, IDNA host processing, and error recovery), it validates structure rather than every scheme-specific rule, and it treats components as UTF-8 text. What you finish with is the honest core that URL libraries share, before the scheme-specific and browser-compatibility layers they add on top.

build-a-url-parser / lesson-01.md
Lesson 01 · The five components

The URI value and a bare path

Every URI, however complex, decomposes into the same five components. Today you define the result type that holds all five and give the library its public Parse entry point, starting with the simplest input of all - a bare path with nothing else around it.

The goal

Define the URI struct with all five components and parse an input that is just a path.

Start here - the target
TO DO
Scenario: Parsing a bare path
Giventhe input string "foo/bar"
WhenParse is called on it
Thenthe result has Path equal to "foo/bar"
AndScheme is empty, HasAuthority is false, HasQuery is false, and HasFragment is false
Background

RFC 3986 describes every URI with one generic syntax: scheme://authority/path?query#fragment. Not every URI has all five parts, but every URI is some subset of exactly these five components, always in this order. The parser’s whole job is to find the boundaries between them, so the natural shape for the result is a struct with a field per component.

A few of those components can be absent rather than merely empty, and that difference will matter later - a URI with no query is not the same as one with an empty query. So alongside the string fields you carry Has... booleans for the parts that can be present-but-empty. Define the entire struct now, even though most fields stay zero today; every later lesson fills in one more boundary. Start at the far end of the grammar: an input with no scheme, no //, no ?, and no # is nothing but a path, so Parse sets Path to the whole string and leaves the rest empty.

Make it work
// the whole library grows around this one result type
type URI struct {
Scheme string
Authority string
HasAuthority bool
Userinfo string
HasUserinfo bool
Host, Port string
HasPort bool
Path string
Query string
HasQuery bool
Fragment string
HasFragment bool
}
func Parse(s string) *URI { return &URI{Path: s} }
CheckpointDONE
You have an importable Parse that returns a typed URI whose Path holds the whole input. Commit and stop here.
Scope & extensions

Where this project stops - and where to go next

A complete, correct RFC 3986 generic-syntax URI library - parse into components, recompose, percent-encode and decode, remove dot segments, normalize (case, percent-escapes, default port), parse a query into decoded pairs, and resolve a reference against a base reproducing the RFC Section 5.4 example table exactly - plus a small runnable demo that reuses it. It deliberately stops short of WHATWG URL parsing, IDNA/punycode hosts, and strict numeric IPv4/IPv6 address validation.

Extend it next
  • Add strict IPv4 and IPv6 address validation instead of falling back to a registered name for an out-of-range octet or accepting any bracketed host.
  • Offer a validating parse that surfaces a malformed percent-escape or a non-numeric port up front, rather than only when a component is later decoded or checked.
  • Round out the per-component encoders (userinfo and host) alongside the path-segment, query, and fragment encoders already built.
  • Add the inverse of ParseQuery - build a query string back from key/value pairs - so query parameters round-trip.
  • Add the non-strict resolution mode from RFC 3986 Section 5.2.2 that tolerates a legacy same-scheme reference.
  • Grow toward the WHATWG URL Standard: IDNA/punycode host processing and the error-tolerant parsing browsers use, as a clearly separate, larger project.
Recommended reading

Books & references that go deeper

  • RFC 3986: Uniform Resource Identifier (URI): Generic Syntax · T. Berners-Lee, R. Fielding, L. Masinter

    The specification this entire project implements. Appendix B gives the parse regex, Section 3 the component grammar, Section 5 the reference-resolution algorithm, and Section 5.4 (mirrored in Appendix C) the exact normal and abnormal resolution examples against base http://a/b/c/d;p?q that the capstone reproduces. Keep it open beside every lesson.

  • RFC 3986 Section 5.4: Reference Resolution Examples · T. Berners-Lee, R. Fielding, L. Masinter

    The pinned example table - normal cases like g giving http://a/b/c/g and abnormal ones like ../../../g giving http://a/g - that makes reference resolution exactly testable. Every resolution lesson asserts values straight from here.

  • The living standard browsers actually use for web URLs. Read it to see where RFC 3986 and the web diverge: WHATWG defines its own error-tolerant parser, host processing (IDNA, IPv4 shorthand), percent-encode sets per component, and a different normalization model. A useful contrast to the strict RFC parser you build here.

  • The browser URL API and its component properties (protocol, host, pathname, search, hash). Handy for comparing your parsed components against what a mainstream implementation exposes, and for the search-params model behind the query chapter.

  • The extension of RFC 3986 to non-ASCII characters via UTF-8 percent-encoding. Read it to see where the ASCII-only URI you build maps to and from internationalized identifiers - the direction a real-world parser grows.