Nice work from Adobe.
It's standard practice to store a fixed reference answer for every eval case. When the underlying data changes daily, that stored answer goes stale.
Adobe researchers write each reference answer as a Python function instead.
The function runs against the live system at evaluation time, so the expected answer follows the data, and an upstream API change makes the test fail visibly.
An LLM judge then splits the agent's response and the computed answer into atomic facts and scores precision and recall, whatever the output format.
Against expert labels, this raises agreement from an MCC of 0.331 to 0.427 and cuts token cost per case by 16%. A judge working with no ground truth scored an MCC of -0.379, which is worse than chance.
The pipeline runs as a harness skill. The authors list one limitation, which is that the same model, Claude Sonnet 4.6, acted as both the skill and the judge.