AI agents struggle on end to end physics research benchmark
A new benchmark designed by UC Berkeley researchers reveals that top artificial intelligence models fail to complete most end to end physics research tasks.
FTMQ SI, written by our newsroom0 views
No artificial intelligence agent passed more than 46 of 168 attempts on FrontierPhysics, a new benchmark designed to evaluate end to end physics research performance. Tech Times reported that doctoral candidate Bingran You of UC Berkeley and 71 co-authors submitted the study to arXiv on November 4. The benchmark consists of 56 tasks drawn from 42 physics papers to test models across literature review, experiment planning, execution, and manuscript preparation. [1][3]
Anthropic's Claude Opus 5.5 achieved the highest success rate among tested models, passing 27.4 percent of its attempts. Tech Times reported that OpenAI's GPT-6 Astra failed due to a lack of expert intuition and weak analytical rigor during the testing process. [1]
The benchmark findings highlight the practical limits of using automated models for complex scientific tasks. Tech Times noted that these low pass rates show current agents still falter when trying to manage the full loop of scientific research within laboratory settings. [1]
Separately, a post on LessWrong AI discussed experiments conducted during the Anthropic Fellows Program regarding agent fleets. The report described researchers using multiple autonomous agents simultaneously to run experiments while human operators monitor their progress. [2]
In short
- No AI agent passed more than 46 of 168 research attempts on the FrontierPhysics benchmark.
- Bingran You of UC Berkeley and 71 co-authors submitted the FrontierPhysics benchmark to arXiv on November 4.
- Anthropic's Claude Opus 5.5 achieved the top score on the benchmark with a 27.4 percent success rate.
- The FrontierPhysics benchmark contains 56 tasks derived from 42 physics papers.
Sources
Every paragraph above points to the numbered items it rests on. Read the originals here.
- [1]No AI Agent Passes More Than 46 of 168 Physics Research Attempts in New FrontierPhysics BenchmarkTech Times, 11d ago (the report this story comes from)
- [2]How to Run Better Research With Agent FleetsLessWrong AI, 11d ago
- [3]FrontierPhysics: Evaluating Agents for End-to-End Physics ResearchalphaXiv, 12d ago
Our newsroom writes these reports with the help of software, from the 3 sources listed and nothing else, and checks them against those sources. Facts can still be wrong or move on; the originals are the record. Spotted a mistake? Write to daniel@monsterkong.com.
Related from FTMQ SI
Earlier reports of ours on the same people and subjects.
- AI agents fail to pass Department of Energy scientific paper benchmarkResearch Papers, 8d ago
- Microsoft and UC Berkeley release TokenEconomizer plugin for AI coding agentsModels & Products, 18d ago
- Anthropic launches Claude Duet agent to rival Meta and OpenAITop Stories, 27d ago
- OpenAI announces three scientific discoveries developed alongside external academic shepherdsTop Stories, 22d ago
More in Research Papers
- Google DeepMind Releases Agentic Cheminformatics Tools for Google Antigravity8d ago
- CERN LHCb detector captures ultra rare B meson decay evidence8d ago
- AI agents fail to pass Department of Energy scientific paper benchmark8d ago
- OpenAI GPT-6 Astra achieves 70 percent on theorem proving benchmark9d ago
- Google releases Gemini Omni 1.2 Flash video AI model10d ago
- Google DeepMind releases Gemini 4 Carbon frontier model16d ago
Get the day in one email
Reports like this one, the top news of the last 24 hours, every morning. Free, one email a day; readers can comment under every report.
By signing up you agree to our terms and privacy policy. Unsubscribe any time.

Comments
Loading
Join the conversation
Comments are open to readers of our daily email: the top news of the last 24 hours, every morning, free. Sign up and the comment box opens.
Already on the list? Enter the same address and we will send a sign-in link.
By signing up you agree to our terms and privacy policy. Unsubscribe any time.