What Is a Shadow Evaluation?

A shadow evaluation gives an AI agent the central question from a high-quality unpublished study without providing the original paper, solution or findings.

The agent works independently, and researchers with expertise in the same question assess its output. The method tests judgment, research design, adaptation and final quality—not only whether the agent can complete technical activities.

Case Study Overview

AI agents can search literature, write code, operate tools, run experiments and produce complete documents. These capabilities create an important evaluation question:

The source study examined this question through two real research problems taken from unpublished NeurIPS 2026 submissions. The agents completed substantial engineering work, but neither final paper met the standard expected by the original researchers.

Learning Objectives

By the end of this case study, participants will be able to:

1. Distinguish narrow, verifiable AI evaluation from open-ended evaluation.
2. Describe the components of a shadow evaluation.
3. Separate technical completion from substantive quality.
4. Identify the five failure modes reported in the study.
5. Assess the method’s strengths and limitations.
6. Apply shadow evaluation to an M&E workflow.
7. Design human oversight and quality gates for an AI agent.

1. Why Open-Ended AI Performance Is Difficult to Evaluate

Many AI evaluations use fixed tasks with results that can be checked automatically. An agent may be asked to improve a score, reproduce a computational result, solve a coding problem or reduce training loss.

Open-ended research requires additional forms of judgment:

• Selecting worthwhile hypotheses
• Determining what evidence would answer the question
• Choosing appropriate data and methods
• Recognizing when an experiment is inadequate
• Deciding when to abandon an approach
• Prioritizing major criticism over minor comments
• Assessing whether the final contribution is significant and original

2. Three Approaches to Evaluating AI Research Agents

Automatically Verifiable Tasks

These are objective, repeatable and scalable, but normally require success to be represented by a predefined metric.

Blind Peer Review

This assesses a complete research paper, but review quality may vary and successful submissions may not reveal how many unsuccessful papers were generated.

Shadow Evaluation

This uses a real, unpublished question and expert assessment. It supports detailed analysis of open-ended work but is resource-intensive and difficult to scale.

3. The Shadow-Evaluation Process

4. The Central Evaluation Question

The evaluation distinguished four levels of performance:

• Activity: Did the agent search, code, experiment and use tools?
• Output: Did it produce a complete research paper?
• Quality: Were the methods, evidence and conclusions sound?
• Outcome: Did the work make a significant and sufficiently original contribution?

5. The Two Research Questions

Personas

The first question concerned the structure and controllability of personas in large language models.

TabPFN

The second question concerned the design of a distribution-shift detector for tabular foundation models.

Both questions came from papers that were unpublished when the principal experiments began, reducing the possibility that the agents could retrieve the researchers’ findings online.

6. Resources and Human Interventions

Each main experiment received:

• Six days of wall-clock time
• USD 3,000 in model API credits
• Additional GPU-compute credits
• A Linux virtual machine and open-web access
• Subagents, research logs and AI review tools
• A conference-quality research target

The study also reported three human interventions:

1. Correction of a technical problem affecting the agent scaffold
2. A 24-hour deadline extension
3. A request for a final readability rewrite

7. How the Agent-Generated Papers Were Assessed

The original researchers assessed the papers using conference-style criteria including quality, clarity, significance and originality.

The reviewers identified poorly justified data and experimental choices, conclusions that did not adequately follow from the evidence, limited contribution and unclear presentation.

8. What the Agents Did Successfully

The cases should not be reduced to a simple statement that the agents failed. They successfully:

• Conducted substantial literature reviews
• Managed research repositories and computing environments
• Debugged infrastructure and GPU problems
• Ran experiments and robustness checks
• Coordinated subagents and external review tools
• Produced complete LaTeX research papers
• Provided reproducibility code
• Detected some errors produced by subagents
• Reported negative results rather than fabricating positive findings

9. The Five Reported Failure Modes

1. Poor judgment about the quality threshold.
   The agents did not adequately recognize what was required for a strong, publishable contribution.
2. Weak creative response to design problems.
   When initial ideas failed, the agents struggled to develop substantially different strategies.
3. Ineffective project-level backtracking.
   They corrected local errors but did not effectively restart the overall project.
4. Poor resource and context awareness.
   They did not manage the relationship between time, compute, budget and expected quality effectively.
5. Instruction drift.
   Requirements recognized early in the process were not consistently followed throughout the multi-day task.

10. The Generator–Verifier Gap

The AI review systems frequently identified weaknesses in the papers. The more difficult problem was whether the research agent could prioritize the criticism and take an appropriate corrective action.

The agents struggled to:

• Separate critical problems from minor comments
• Treat repeated rejection as a redesign signal
• Avoid overweighting favourable feedback
• Convert criticism into a fundamentally different research plan

11. Strengths and Limitations of Shadow Evaluation

Strengths

• Tests open-ended decision-making
• Uses questions that were not publicly available
• Uses reviewers with direct expertise in the question
• Examines process logs as well as final outputs
• Can be repeated with new models and questions
• Supports detailed qualitative analysis

Limitations

• Only two principal research questions were studied
• The reviewers were not blind
• There was no single objective ground truth
• The selected questions may not represent all research tasks
• Model and scaffold choices may have influenced performance
• The original human researchers worked on the questions for longer

12. What Can and Cannot Be Concluded?

Supported by the Cases

• The tested agents performed substantial research-engineering work.
• They did not produce acceptable answers to the two selected questions.
• Similar failure patterns appeared across the two cases.
• Output completion was not an adequate measure of research success.
• Process logs revealed problems that final-output review could miss.

Not Established by the Study

• That no AI agent can conduct open-ended research
• That AI agents will never become autonomous researchers
• That all professional fields will produce the same results
• That shadow evaluation is free from bias
• That AI agents are not useful to human researchers or evaluators

13. Application to Monitoring and Evaluation

An AI agent may complete a visible M&E activity while still failing to achieve the intended outcome.

For example, an agent may:

• Produce an evaluation matrix using unsuitable methods
• Code interviews without identifying meaningful patterns
• Draft a report without adequately supporting its claims
• Calculate an indicator correctly but misinterpret its meaning
• Review a donor call while missing a critical requirement
• Generate a complete proposal that does not address the evaluation criteria

A strong AI-agent evaluation should examine:

• Inputs: Data, tools, permissions and instructions
• Activities: Searches, calculations, analyses and reviews
• Outputs: Documents, datasets or recommendations produced
• Quality: Accuracy, completeness, clarity and compliance
• Outcome: Whether the task supported the intended decision
• Adaptation: Whether the agent responded appropriately to feedback
• Efficiency: Use of time, tools and budget
• Safety: Protection of evidence, data and credentials

14. Converting the Findings into AI-Agent Controls

• Premature commitment: Require an alternatives register and minimum exploration stage.
• Weak design: Require human approval before the main method is executed.
• Repeated negative reviews: Create a stop-and-redesign rule.
• Poor feedback prioritization: Classify issues as critical, major or minor.
• Resource problems: Add scheduled budget and time checkpoints.
• Instruction drift: Use a persistent compliance checklist.
• Unsupported conclusions: Require an evidence-to-claim verification table.
• Weak presentation: Separate methodological, technical and editorial review.

Possible tasks include:

• Developing an evaluation matrix
• Analyzing qualitative interview summaries
• Preparing an evidence-synthesis brief
• Reviewing an indicator framework
• Drafting an evaluation-report section
• Assessing a donor call for compliance
• Conducting a data-quality assessment

1. Select a completed professional task with a credible human-produced output.
2. Define the intended outcome, not only the required document.
3. Provide realistic evidence, time, tools and permissions.
4. Withhold the completed human answer.
5. Select expert reviewers.
6. Collect process logs, decisions, reviews and resource-use records.
7. Apply quality, traceability, adaptation and safety criteria.
8. Document limitations and avoid unsupported generalization.

Shadow Evaluation Design Template

Case Discussion Questions

1. What capability was the study attempting to measure?
2. Why would a standard coding benchmark have been insufficient?
3. Should the successful engineering work count as partial success?
4. Why is “paper completed” an inadequate primary outcome?
5. What should have triggered project-level backtracking?
6. How should conflicting AI reviews be prioritized?
7. Which decisions should remain with a human expert?
8. How might non-blind review affect the results?
9. What evidence would be needed before generalizing the findings?
10. Which M&E task could your organization test through shadow evaluation?

Key Takeaways

1. Open-ended capability cannot always be assessed through a fixed metric.
2. Technical completion is not the same as substantive success.
3. Expert review can reveal weaknesses that activity counts miss.
4. Process logs help explain why an agent succeeds or fails.
5. Review tools are insufficient unless feedback produces appropriate action.
6. Shadow evaluation is useful but resource-intensive and dependent on expert judgment.
7. Two cases provide early evidence, not a universal conclusion.

Main Sources and Further Reading

The original paper remains the primary source for the research design, findings, figures and limitations.

• Original research paper — arXiv:
  
  https://arxiv.org/abs/2607.27191
  
• Original research paper — PDF:
  
  https://arxiv.org/pdf/2607.27191
  
• CRUX evaluation project and reproduction materials:
  
  https://cruxevals.com
  
• NeurIPS 2026 reviewing guidance:
  
  https://neurips.cc/Conferences/2026/ReviewerGuidelines
  
• NeurIPS paper checklist:
  
  https://neurips.cc/public/guides/PaperChecklist
  

Frequently Asked Questions

Did the study prove that AI agents cannot conduct research?

No. It provided early evidence from two demanding questions. The tested agents completed substantial engineering work but did not produce papers accepted by the original researchers.

Why were unpublished questions used?

This reduced the chance that the agents could retrieve the original researchers’ findings online or reproduce a known answer.

Why is task completion an insufficient metric?

An agent can complete searches, experiments and documents while still using weak methods, misinterpreting evidence or failing to achieve the intended outcome.

Can shadow evaluation be applied to M&E?

Yes. It can be adapted to evaluation-matrix design, evidence synthesis, qualitative analysis, indicator review, compliance analysis and report drafting when a credible human comparison exists.

Continue Your Learning

Develop Practical and Responsible AI Skills for Evaluation

Continue with the EvalCommunity Academy pathway that best matches your professional learning goals.