An AI agent worked for up to six days, spent thousands of dollars in compute, and completed every piece of engineering — with zero human intervention.
Then the original paper authors reviewed the results and rejected them, no ambiguity.
The experiment was run twice. Same outcome both times.
This isn’t a test of whether AI can write code. It’s a harder question: when code is no longer the bottleneck, can AI find research questions worth answering — and recover when a line of investigation fails?
The latest answer: not yet.
A Different Kind of Evaluation
Judging whether AI can do research has always been tricky.
Give it a problem with a known answer, and scoring is easy — but that tests problem-solving, not research. Real research has no answer key, and sometimes nobody even knows whether the question is worth asking.
Another approach is to have AI write papers and submit them for peer review. Sounds direct, but peer review is noisy, stochastic, and already overloaded.
This study tried a third method: shadow evaluation.
The researchers took two unpublished NeurIPS 2026 submissions, extracted the core open research question from each, and handed it to frontier agents. The agents never saw the authors’ solution. They had to design their own approach, write code, run experiments, and analyze results — like actual researchers.
Each agent had up to six days and thousands of dollars in compute. Afterwards, the original paper authors graded the output.
The strength of this design: the graders know what real progress looks like, and which approaches look busy without touching the core problem.
The limitation: only two papers. The authors themselves call their findings “early evidence.”
Engineering Complete, Research Stalled
Both agents completed all engineering work with zero human intervention.
That sentence alone is remarkable. They can set up environments, implement code, launch experiments, and process results. A few years ago, any one of those tasks would have been a showstopper for a model. Now, agents sustain long-running tasks and even finish an entire research engineering pipeline.
But the authors’ verdict was unambiguous: the agents made no substantive progress on the core research questions. Both submissions were rejected.
The team ran a robustness check with a different model and agent scaffold. The same failures reappeared. This isn’t just one model having a bad day — and it’s not just a prompt engineering problem.
The most dangerous misinterpretation is right here.
If you look at the process logs, you’ll see intense activity: code growing, experiments running, figures generating, errors getting patched. The system looks incredibly productive.
But research progress isn’t measured by activity volume.
Lines of code, number of experiments, tokens consumed — none of these equal proximity to the answer. An agent can work for six days. It can also work for six days in the wrong direction.
Five Failures Worse Than “Can’t Write Code”
The paper categorizes the recurring problems into five types.
Type 1: Can’t Tell What Counts as Publishable
Agents complete tasks but struggle to judge whether the output has research value.
Normal software tasks have reasonably clear stopping conditions: tests pass, the API returns correct results, the page matches the design spec. Open-ended research has none of this. A positive experimental result doesn’t mean the conclusion is novel. An improved metric doesn’t mean the improvement is meaningful.
Agents are sensitive to “task complete” but lack stable judgment about “research bar.” They easily mistake a formally complete result for a publishable one.
Type 2: No Creative Recovery When the Design Fails
Mediocre experimental results are normal. What separates researchers is what happens next.
Experienced researchers might question the hypothesis, redefine variables, switch the evaluation method, or realize the question was wrong. Agents tend to keep tinkering within the original framework: tweak another parameter, run another experiment, make the results look a little more complete.
They can optimize within a path — but they’re bad at deciding whether to abandon the path.
Type 3: Can’t Backtrack From Dead Ends
In coding, backtracking is usually concrete: revert a bad commit, restore passing tests.
In research, backtracking might require admitting that two days of experiments had zero information value, re-examining initial assumptions, or throwing out a large volume of completed work.
Humans struggle with sunk cost too. Agents just accumulate it faster — they produce more “completed work” more quickly, so the sunk cost on a wrong direction builds faster. When continuous execution is the default goal, persistence is easily mistaken for progress.
Type 4: No Resource Awareness
Six days and thousands of dollars sounds generous, but it’s finite for open research.
A reliable researcher constantly estimates: how long will this experiment take? Is there enough compute budget left? If this round fails, is there time to switch approaches? Which uncertainty is the most valuable to resolve first?
Agents can invoke resources but don’t understand their opportunity cost. They can burn large budgets on low-value experiments, and by the time the critical question surfaces, there’s no time left to start over.
Type 5: Working Longer, Drifting Further — Instruction Drift
Agents encounter countless local problems during execution: environment won’t install, metrics look wrong, scripts crash, results are unstable. Each fix adds new context to working memory.
Over time, the agent may still be working hard — but it has quietly rewritten the task. From “answer the core research question” to “make this experiment pipeline run smoothly.” From “validate whether the new method works” to “produce a set of results that look explainable.”
Local goals grow more concrete. The original goal grows more blurry.
Programmers know this state intimately. You fix and fix until all tests pass — then look up and realize the user’s original problem is still there.
“Can Do” and “Knows What to Do” Are Diverging
We used to worry that AI couldn’t handle long tasks: context would be lost, tools would throw errors, code wouldn’t run.
Those problems haven’t disappeared, but the capability frontier has moved. Agents now complete increasingly long engineering chains. So a new bottleneck emerges: they execute fast, but don’t necessarily know what’s worth continuing.
This study doesn’t prove AI can never do research. It proves that, in these two open-ended cases, engineering automation did not automatically produce research automation.
Between the two sits judgment.
Research judgment includes choosing problems, setting evidence thresholds, recognizing when a line of investigation has failed, deciding when to backtrack, and prioritizing which uncertainty to eliminate first within a limited budget. These activities rarely produce code — but they determine whether the code has value.
That’s why “AI completed all the engineering” and “AI didn’t complete the research” can both be true at the same time.
This Hits Closer to Home for Programmers
People might think AI doing research is far removed from everyday development. But the paper’s five failure modes map almost directly to software projects.
Ask an agent to “finish the signup flow” — it might complete the API, pages, and tests, without realizing the real goal was reducing new user churn.
Ask it to “fix the concurrency bug” — it might make the tests stop failing, without confirming who actually owns the state, because the race condition was just masked by slower execution order.
Ask it to “refactor this module” — it can produce thousands of lines of elegant code, without proving that the new structure is easier to maintain than the old one.
These aren’t coding ability problems. They’re acceptance criteria and direction judgment problems.
When AI writes code faster and faster, teams need to actively suppress low-value output. Cheap code means wrong-direction code also becomes cheap.
The more complete the output looks, the higher the psychological cost of throwing it away.
How to Stop Agents From Busily Doing the Wrong Thing
If we accept that agents aren’t yet good at open-ended judgment, we can’t just write a cleverer prompt. The workflow needs deliberate brakes.
1. Write success criteria AND failure criteria.
Don’t just tell the agent what to deliver. Define what evidence would prove the approach is not working. Tasks without failure criteria drift into infinite patching.
2. Slice long tasks into falsifiable stages.
Stage 1 validates the requirement. Stage 2 validates the riskiest assumption. Stage 3 expands to full implementation. Every stage allows stopping — don’t assume the agent must run all the way to the final deliverable.
3. Require it to log assumptions, not just progress.
“Added 12 files today” has little decision value. “We assume the bottleneck is database queries, but existing data doesn’t yet support this” has value. The latter helps a human decide whether to change direction.
4. Predefine backtrack conditions.
For example: if two consecutive experiments show no improvement, return to approach selection. If 40% of budget is spent without validating the core hypothesis, stop expanding features. If a key result can’t be reproduced by an independent method, don’t proceed to the release stage.
5. Have someone who didn’t do the work review the result.
An agent that generates its own plan, implements it, and declares success can easily mistake process completeness for correctness. Independent review forces the system back to the original goal, rather than continuing to tell a coherent story along the execution log.
Code Isn’t Research. Busy Isn’t Progress.
What bothers me most about this experiment isn’t “both papers were rejected.” It’s that agents are now strong enough to produce output that looks exactly like progress.
They write code. They run experiments. They fix errors. They produce structurally complete results. Looking only at the process, it’s easy to believe they’re almost there.
The authors’ review punctured that illusion: all the engineering was done, and the core question hadn’t moved an inch.
For AI teams today, this is more dangerous than “the model occasionally writes buggy code.” Code bugs get caught by tests. Direction errors can pass through with passing tests, documentation, and polished charts — right into an internal review.
The future of human-agent collaboration may not be “human writes requirements, AI writes code.” A more reliable division might be: AI expands execution capacity. Humans choose the problems, set the stopping conditions, and judge whether the result genuinely changes what we know about the problem.
Agents can be very diligent. Our job is to make sure they’re not diligently heading in the wrong direction.
Source
- Peter Kirgis, Sayash Kapoor, Arvind Narayanan et al., Can AI agents conduct open-ended AI research? Early evidence from two case studies
- arXiv: 2607.27191
Note: This study covers only two cases and constitutes early evidence. Do not extrapolate to all models, agent frameworks, or research domains. “Completed all engineering” refers to agent execution without human intervention under the paper’s experimental conditions, not to completing full research.