Livenerf is measuring model drift, not yet proving a downgrade

Livenerf is collecting repeated benchmark runs to test whether a model’s performance changes after release. It freezes prompts, pins the client and keeps raw logs, while acknowledging that model output remains nondeterministic. The project is still establishing its baseline, so current activity is evidence of a measurement effort rather than proof that Opus 5.5 has become worse.

Livenerf project

Why it matters

Engineers can distinguish a testable performance concern from impressions based on isolated responses.

More context

Watch the preregistered comparison windows and deviations log. The September 30 status reports seven baseline days collected. The initial setup uses a Claude Max subscription through headless Claude Code; no exact original publication date was established.

Keep in mind

This is an early measurement project. Its narrow question panel includes ambiguity and sensitivity limits; validation did not reliably distinguish one tested model swap. 1min has not rerun the benchmark.

Original read