Projects/Build a JPEG Encoder and Decoder

Build a JPEG Encoder and Decoder

Start by recognizing two marker bytes and end with a codec that decodes a real baseline JPEG to pixels and encodes pixels back into a valid JPEG. Along the way you build the hard middle from scratch - a bit reader that understands byte-stuffing, canonical Huffman decode tables, the receive-and-extend signed-magnitude decode, run-length AC coefficients, and a separable inverse DCT - then reverse every stage into a forward DCT, quantizer, and entropy encoder that produces files any JPEG viewer will open.

53 lessonsLarge~20 min / lessonDCTEntropy codingChroma subsampling
The project

What you'll build over the next 53 lessons

Over 53 lessons you build a working baseline JPEG codec from first principles, decoder first and then encoder. You begin with the container: the 0xFF-prefixed marker structure, SOI and EOI, the two-byte segment length framing, the APP0/JFIF header, and how to skip segments you do not recognize. Then you parse the tables and the frame: quantization tables in both 8-bit and 16-bit precision with the zig-zag storage order, DC and AC Huffman tables built from their 16 code-length counts into canonical decode tables, and the SOF0 frame with per-component sampling factors and MCU geometry. The centerpiece is the entropy-coded scan: an MSB-first bit reader that handles byte-stuffing, the receive-and-extend decode that turns a magnitude category into a signed coefficient, DC coefficients as differentials against a per-component predictor, AC coefficients as run-length pairs with the EOB and ZRL special symbols, and restart markers that realign the stream and reset the predictors. You dequantize, inverse-zig-zag, run a separable inverse DCT, level-shift and clamp, upsample chroma, and convert YCbCr to RGB to finish a decoder that turns a real baseline JPEG into pixels.

With a complete decoder in hand you reverse the whole pipeline: convert RGB to YCbCr, downsample chroma, level-shift, run a forward DCT, quantize with the standard example tables, zig-zag, and entropy-encode with run-length and the standard Huffman tables, then write every marker - SOI, APP0, DQT, SOF0, DHT, SOS, the byte-stuffed entropy scan, and EOI - into a valid baseline JPEG. The reference encoder you assemble writes a single-component (grayscale) baseline file, which exercises the whole DCT, quantize, and entropy path; wiring the color-conversion and downsampling stages you also build into a full 4:2:0 encoder is a natural extension. The capstone decodes an embedded baseline JPEG to pixels and proves your encoder writes a structurally valid file that round-trips back within a small tolerance.

This is a teaching-grade codec built around the real JPEG (ITU-T T.81) and JFIF specifications, and it is deliberately scoped to baseline sequential DCT only. It does not implement progressive JPEG, arithmetic coding, or 12-bit precision, and the encoder writes grayscale rather than color. Because JPEG is lossy, the codec is exact where a stage is lossless - Huffman decode, dequantization, a known coefficient block through the inverse DCT, and color conversion of exact inputs all produce exact values - while a full decode, encode, and decode round-trip is proven correct within a stated per-channel tolerance rather than byte for byte. What you finish with is the honest core that libraries like libjpeg extend with progressive scans, color encoding, optimized Huffman tables, and higher precision.

build-a-jpeg-codec / lesson-01.md
Lesson 01 · The JPEG container

The start and end markers

Every JPEG file opens with a Start Of Image marker and closes with an End Of Image marker. Today you build the smallest useful thing a decoder can do - recognize that a byte stream even claims to be a JPEG - so the codec has a front door from day one.

The goal

Report whether a byte slice begins with the SOI marker and whether it ends with the EOI marker.

Start here - the target
TO DO
Scenario: Recognizing the JPEG frame markers
Givena byte slice beginning with 0xFF, 0xD8 and ending with 0xFF, 0xD9
Whenthe marker check runs on it
Thenit reports that the stream starts with SOI and ends with EOI
Anda slice starting with 0xFF, 0xD9, or one shorter than two bytes, reports no SOI
Background

Every JPEG file is framed by two markers. It opens with SOI, the Start Of Image marker, whose two bytes are 0xFF 0xD8, and it closes with EOI, End Of Image, the bytes 0xFF 0xD9. A marker is always a 0xFF byte followed by a code byte that names it. These two are the outermost frame: everything a decoder reads lives between them.

You will not decode anything yet. You are building the one gate every decoder needs first: does this stream even claim to be a JPEG? Anything that does not begin with 0xFF 0xD8 is rejected before a single segment is read. Pin the edges now - a stream that is only one byte long must report no SOI rather than reading past its end, and a stream that starts with the EOI code 0xD9 is not a valid opening.

Make it work
// markers are two bytes: 0xFF then a code byte
const MarkerSOI = 0xD8 // Start Of Image
const MarkerEOI = 0xD9 // End Of Image
func HasSOI(b []byte) bool {
// true only if b[0]==0xFF and b[1]==0xD8
}
func HasEOI(b []byte) bool {
// true only if the last two bytes are 0xFF, 0xD9
}
CheckpointDONE
Your codec recognizes a JPEG by its opening marker and its closing marker. Commit and stop here.
Scope & extensions

Where this project stops - and where to go next

The codec fully and correctly implements baseline sequential JPEG (grayscale and 4:2:0/4:4:4 decode, grayscale encode) with graceful handling of unsupported and corrupt input, but deliberately stops short of color encoding, progressive and arithmetic decoding, and optimized Huffman tables.

Extend it next
  • Extend the encoder to write 3-component 4:2:0/4:4:4 color output instead of grayscale-only
  • Generate custom per-image Huffman tables on encode instead of always using the fixed standard tables
  • Replace nearest-neighbor chroma upsampling with bilinear filtering for smoother decoded images
  • Add progressive JPEG decoding (multiple scans, spectral selection, successive approximation)
  • Add finer-grained corruption diagnostics for malformed inputs that currently fall through to a generic error
  • Offer a streaming reader/writer API instead of requiring whole-file byte slices
Recommended reading

Books & references that go deeper

  • The authoritative JPEG standard: marker codes, the DQT/DHT/SOF0/SOS segment layouts, the canonical Huffman construction (Annex C), the receive/extend and run-length entropy decode (Annex F), and the standard example tables (Annex K). The primary reference for the whole project.

  • JPEG File Interchange Format (JFIF), Version 1.02 · Eric Hamilton / C-Cube Microsystems

    The JFIF container conventions layered on top of T.81: the APP0 segment identifier and version, the density units, and the YCbCr color model and full-range level conventions the codec assumes.

  • A famous plain-text walkthrough that decodes a baseline JPEG by hand: the marker walk, the Huffman tables, the DC/AC entropy decode with byte-stuffing, and the MCU assembly. The clearest single companion when your entropy decoder misbehaves.

  • Understanding and Decoding a JPEG Image · ImpulseAdventure (Calvin Hass)

    A readable, worked baseline-JPEG walkthrough with annotated hex - a gentle orientation to the segment stream, quantization, and the entropy scan before diving into the formal spec.

  • A broad overview of the JPEG pipeline - the DCT, quantization, chroma subsampling, and entropy coding - useful for the intuition behind why each stage looks the way it does.