Whether to respond to a contact depends entirely on how fast the two bodies are closing along the contact normal. Today you compute that single number, the input to every impulse.
Compute the relative velocity of two bodies projected onto the contact normal.
Collision response hinges on one scalar: the relative velocity along the contact normal. Take the velocity of B relative to A (their difference) and project it onto the normal with a dot product. The sign is what matters. Negative means the bodies are moving into each other and the contact must push back; positive means they are already separating and should be left alone; zero is a resting contact, two bodies in steady sustained touch, neither approaching nor parting.
That resting-contact zero is the edge to respect. Two bodies drifting together at the same velocity are touching but not colliding - firing an impulse at them would inject energy from nothing and make a stack of boxes jitter. So every impulse you compute next lesson is gated on this number being negative. Getting the closing speed right, sign and all, is the foundation of a stable solver.
// relative velocity of B with respect to A, projected onto the normalfunc NormalVelocity(a, b *Body, n Vec2) float64 {rv := Sub(b.Velocity, a.Velocity)return Dot(rv, n)}