A GitHub cleanup job overwhelmed the database its safeguard ignored

GitHub’s September incident report describes a cleanup job that watched replica lag but ignored load on the primary database. It kept writing until available connections ran out. Missing fast timeouts left requests waiting, while token-creation retries prolonged saturation. Pausing the job and reducing internal load restored service. The lesson is to throttle background work against the resource it can actually exhaust.

GitHub · Oct 14, 2026

Why it matters

A safeguard can appear healthy while a different bottleneck is already failing.

More context

Review background-job controls, connection limits and retry behavior together. This card covers the September 13 database incident; the report separately describes missing Copilot review results after a service change.

Keep in mind

The report gives inconsistent duration labels, so this summary avoids a single precise outage duration.

Original read