1min.

1MIN · PRODUCTION ENGINEERING

Stale routing records sent GitHub writes to retired servers

GitHub’s October report describes two write outages caused by a new database-routing machine starting with stale records. Once elected leader, it directed writes to retired database servers. An automated maintenance task then ignored a pause and triggered a second election, repeating the failure. GitHub reports about 19 minutes of write disruption across the two incidents, with saved data preserved.

GitHub Blog · Nov 12, 2026

Why it matters

A healthy election protocol cannot compensate for incorrect routing state or maintenance controls that do not take effect.

More context

This card covers the October 7 incidents within a November 12 retrospective. GitHub removed the new machine and tightened routing validation, retirement procedures and maintenance-pause handling. The review exercise is to test what a replacement coordinator publishes before allowing it to lead.

Keep in mind

A vendor incident account, not an independent audit. The 19-minute figure covers write disruption in these two incidents, not all degraded service periods or October’s total downtime.

Original read

More engineering ideas, one minute at a time.

Explore more stories