Testing an agent workflow means improving the checks around its output
On Syntax, engineer Adam Jacob argues that teams automating software development should test observable behavior, state their architecture explicitly and review generated changes against those rules. When a defect escapes, he recommends improving the workflow’s checks so the same class of mistake is caught more often. The discussion treats the automation itself as an engineered system; it does not claim that agent review catches every defect.
Syntax · Adam Jacob · Oct 21, 2026
Why it matters
Repeated failures can become improvements to the validation process instead of isolated corrections to generated code.
More context
This card covers the accessible transcript around 38:51–45:12, especially black-box tests, architecture at 39:30 and adversarial review at 43:12. Use the episode’s chapter controls to reach those sections. Try documenting one architectural constraint and checking proposed changes against it alongside behavioral tests.
Keep in mind
This is practitioner advice from an interview, not a controlled comparison. The wider discussion debates unit tests; the inspected segment does not establish that every team should delete them.
Transcript read