synodic-ai ML RESEARCH
ML Research Desk · Evaluation Methodology

What the Signal Actually Checks: Mechanical Verification, Recoverable Teachers, and Interface-Invariant Agents

2026-10-15

Every training or acceptance pipeline rests on a signal that is assumed to measure the thing it is meant to measure: a judge's verdict, a teacher's token-level supervision, a per-sample score. Three preprints from 2026-10-14 each find a place where that assumption breaks, and each proposes a check that does not share the failure it is meant to catch.

A supervision signal is a measurement, and measurements have blind spots

Model-based judges, teacher models, and per-sample rewards are now the default way to decide whether an output is acceptable and what a model should learn from it. Their convenience hides a dependency: if the signal shares a blind spot with the system it grades, errors pass through as successes, or are learned as targets.

The three papers below find that dependency in three different places. In autoformalization, a language-model judge accepts formal statements that are wrong. In on-policy distillation, a teacher that can recognize a student's error nevertheless continues from it. In agent post-training, scoring each sample on its own ties the agent's competence to the surface form of the interface it was trained on. Each paper's fix adds a signal that fails differently from the one it supplements.

Beyond the LLM judge: mechanical checks for Lean formalizations

Anna Mei and Michael R. Douglas address the verification of Lean formalizations, where the usual test of whether a formal statement faithfully captures its informal source is a judgment by a language model or another trained model. Their method keeps an LLM-based semantic review but adds mechanical numerical and vacuity checks, which provide checkable feedback and detect errors that LLM judges miss.

They apply the method, together with their Extract-Verify-Proof autoformalization pipeline, to build the Formalized Library of Mathematical Functions (FLMF), a partial formalization of the NIST Digital Library of Mathematical Functions. On three labeled datasets the method achieves the lowest accepted-error rate among the evaluated methods, where the accepted-error rate is the fraction of incorrect statements that a verification method accepts. Expert review of the results also identified translation and source defects in existing datasets, a reminder that the labeled data used to score verifiers carries errors of its own. The authors have released their code.

CriticOPD: the teacher that follows the student's mistake

On-policy distillation (OPD) trains a student on its own rollouts with token-level supervision from a teacher. The method assumes what the CriticOPD authors call the recoverability of the teacher: that the teacher can reach a correct answer from any prefix the student writes. They show that this property fails on incorrect rollouts. The teacher follows the student's errors, even though it recognizes them.

CriticOPD uses that recognition to restore recoverability. The teacher critiques an incorrect rollout, and the student rewrites its solution following the error. The teacher scores the resulting output with the critique included as privileged information, while the student trains on the revision without the critique. On two student models and six mathematical reasoning benchmarks, CriticOPD improves on OPD, with the largest gains on competition problems.

The diagnosis matters beyond this method. A teacher conditioned on a wrong prefix is a grader evaluating from inside the error, and token-level supervision from that position can reinforce the mistake rather than correct it. The fix separates the teacher's ability to recognize the error from its role in continuing the text.

CIPR: competence bound to an interface

Artin Tajdini, Manpreet Kaur, and Peng Qi start from an observation about agents: post-training binds an LLM agent's competence to the interface it was trained on. Semantics-preserving perturbations of that interface, such as renaming an action or changing the notation, leave the underlying task unchanged yet make the agent worse at it. Because each sample is scored independently, randomizing the interface during training only broadens the dependence from one interface to the training distribution of interfaces, and out-of-distribution perturbations still degrade the agent.

Cross-Interface Policy Regularization (CIPR) is a self-supervised objective that replaces action names with uninformative identifiers and presents each training sample under two interfaces in which equivalent actions are identified by construction. It penalizes the KL divergence between the action distributions the two interfaces induce, so the policy behaves consistently across them. The authors evaluate it in both supervised and reinforcement-learning post-training and report improved robustness in both. On agentic function calling, held-out interface perturbations reduce the accuracy of a post-trained agent by nearly 25%, and CIPR recovers 10.5% while leaving accuracy on the unchanged interface intact.

Reading the three results together

All three are preprints, and their figures are the authors' own, measured on their chosen datasets, models, and baselines. The verification result is reported on three labeled datasets and a partial formalization of one reference library. CriticOPD is evaluated on two students and on mathematical reasoning only. CIPR's headline numbers come from agentic function calling, and it recovers part, not all, of the accuracy lost to held-out perturbations.

The common thread is a change in what counts as evidence. The mechanical checks add numerical and vacuity tests whose outcome does not depend on a model's judgment. CriticOPD stops treating a teacher's continuation of a wrong prefix as a target. CIPR scores a policy by its consistency across equivalent presentations of the same task, not only by its success on the presentation it saw. In each case the added signal targets a specific, named failure of the signal already in use.

Practical reading for evaluation teams

The three papers suggest audits that evaluation teams can run on their own systems.

Sources