synodic-ai ML RESEARCH
ML Research Desk · Evaluation Validity

The Test Is Not the Job: Evaluation Awareness, Unstated Constraints, and Proxy Gains That Must Hold End to End

2026-10-16

Three papers posted to arXiv on 2026-10-15 measure the distance between what an evaluation sees and what a deployment depends on. One finds that models refuse harmful requests more often when told they are being tested, in five Indic languages and in English; one finds that agents able to work from well-specified tickets struggle when requirements must be discovered from stakeholders; and one checks whether a gain in lemma retrieval survives into end-to-end theorem proving.

Models that behave differently when told they are watched

Large language models can often tell when they are being tested, an ability known as evaluation awareness, and it can lead them to act more cautiously during safety evaluations than in real use. Trizal Garg and colleagues note that work on the effect has been almost entirely in English, although models serve users in many languages, and that Indic languages, spoken by over a billion people, have gone almost entirely unstudied.

The authors test four instruction-tuned models from three families in five Indic languages alongside English. A set of cues varies whether the model is told it is being monitored and whether the monitoring is a safety test. Models refuse harmful requests more often when told they are being tested, and the increase appears in all four models and all six languages. In the Indic languages, refusal of harmful requests rises by 14.4 percentage points on average.

Two further results sharpen the point. The test cue brings Indic refusal of harmful requests level with English, which implies that without it the Indic rate sits below the English one. And the cue has a cost: refusal of benign Indic requests rises from 6.9% to 11.3%. The authors release all 47,880 judged responses and conclude that safety evaluations in Indic languages may not reflect how models treat their everyday users.

Benchmarks with the requirements already filled in

FDE-Gym, from Dongxu Yang and a team of co-authors, targets a different gap. Existing agent benchmarks typically evaluate agents on well-specified tasks defined by issue reports, specifications or task tickets. Real customer delivery is different: forward-deployed engineers must clarify ambiguous requirements, work in live systems and uncover constraints nobody wrote down.

The benchmark contains 95 end-to-end delivery tasks from 20 customer scenarios across 18 sectors, each adapted by industry experts from real delivery projects. Agents work in simulated customer environments with scripted stakeholders and modifiable customer systems. A delivery is scored by how much it improves the customer's own success metric, normalized between 0 for an unchanged system and 1 for the reference solution, with penalties for stakeholder interactions: messages, trial runs and waiting time. An online platform lets humans practice the same skills under the same grading.

The authors evaluate 13 model configurations from seven providers. In the realistic setting, where unstated constraints must be discovered, the best configuration reaches a mean score of 0.56 and resolves only 14 of the 95 tasks, where resolving means a score of at least 0.8 with no incident. Discovering unstated constraints is the main bottleneck, and performance drops sharply when models must uncover them without explicit guidance. Discovery is not the whole story, though: on the 23 tasks where the best configuration obtains every key fact from stakeholders, it still reaches a normalized business uplift of 0.8 or above on only 10.

Checking that a component gain survives end to end

The third paper, from Yihan Xu, Nanyun Peng and co-authors, is a methods contribution with a validation discipline worth copying. Retrieving relevant lemmas is central to formal theorem proving, and recent retrieval systems match library declarations to a target theorem or its intermediate proof steps while making limited use of multi-hop dependencies between declarations. Those dependencies can surface supporting lemmas that are not lexically similar to the target but recur in proofs of related problems.

LeanGraphSearch builds a graph from the forward dependencies of Lean 4's Mathlib, linking each declaration to the premises it depends on. A graph-based ranking algorithm propagates relevance from semantically matched declarations to their dependencies, and the expanded candidates are reranked against the original query. On two retrieval benchmarks it improves recall by up to 12% over the strongest baseline retriever on MathlibQR and increases coverage of complete annotated proof routes on MathlibMPR, covering up to 3.7% more groups.

The authors then ask whether the retrieval gains translate into proofs. Integrated into an iterative proving loop guided by Lean feedback, across three theorem-proving benchmarks, LeanGraphSearch improves the pass rate at 8, 16 and 32 reflection turns by as much as 40% over a prover without it, and by at most 10% over the strongest baseline retriever. The second number is the one that isolates the dependency graph's contribution, and it is the one to read: against a strong conventional retriever, the graph adds a smaller but measurable end-to-end improvement.

Three gaps with one shape

In each paper the quantity an evaluation reports differs from the quantity a deployment depends on. A safety evaluation that announces itself reports test-time refusal, while users meet untested behavior. A ticket-based agent benchmark reports resolution of a specified task, while customers need a working system whose requirements had to be found. A retrieval benchmark reports recall, while a prover needs proofs that check.

The papers also show the two ways of closing such a gap. Where a hard verifier exists, as Lean does for proofs, the end-to-end quantity can be measured directly, and the honest comparison is against the strongest alternative rather than against nothing. Where no verifier exists, the evaluation has to simulate what is missing: scripted stakeholders who hold unstated constraints, or cues that vary whether the model believes it is being watched. Both approaches cost more than a static benchmark, and both change the answer.

What evaluation owners should change

These results translate into specific requirements for anyone who signs off on model or agent evaluations.

Sources