Lesson 13 · The on-failure policy - Build a Process Supervisor

The on-failure policy

The most useful policy restarts a service only when it crashed - a nonzero exit - and leaves it alone when it finished cleanly. Today you decide the on-failure case, the one place the exit code changes the answer.

The goal

Decide on-failure - restart on a nonzero exit, but not on a clean exit 0.

Start here - the target

TO DO

Scenario: on-failure restarts only a crash

Giventhe on-failure policy

WhenShouldRestart(PolicyOnFailure, code) is asked for each code

ThenShouldRestart(PolicyOnFailure, 0) is false (a clean exit is left alone)

AndShouldRestart(PolicyOnFailure, 1) is true and ShouldRestart(PolicyOnFailure, 2) is true (any nonzero exit is a crash worth restarting)

Background

on-failure is the policy that matches how most people actually think about “keep it alive”: if the service crashed, bring it back; if it finished its work and exited cleanly, let it rest. The whole decision rides on the exit-code classification you pinned in lesson 11 - code != 0 is a crash, code == 0 is a clean finish - so this is the first policy where the number genuinely changes the answer.

This is exactly the edge worth testing at the boundary. A policy that restarted on any exit would be always; a policy that never restarted would be never. What makes on-failure distinct is precisely that it splits at zero: ShouldRestart(..., 0) is false while ShouldRestart(..., 1) is true. Pin both sides of that split, because a supervisor that restarts a job which already succeeded will loop it forever, and one that ignores a crash defeats its own purpose.

Make it work

// add the on-failure arm to ShouldRestart:
case PolicyOnFailure:
  return code != 0 // restart a crash, leave a clean exit alone

CheckpointDONE

on-failure restarts a crashed service but not a cleanly finished one. Commit and stop here.