📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
AI systems are now capable of automating much of AI engineering, including reproducing research and optimizing kernels. However, AI-driven research itself remains partially human-led, though this may change soon.
Recent advances in AI capabilities demonstrate that large language models and automation tools have reached a point where they can handle most core engineering tasks involved in AI research, effectively automating the engineering side of AI development. Experts say that while AI can now reproduce research and optimize infrastructure with high reliability, the creative and exploratory aspects of research remain less automated, although this gap may narrow rapidly.
Multiple recent benchmarks, including CORE-Bench and MLE-Bench, show AI systems achieving near-complete automation of core engineering skills. For example, CORE-Bench, which measures the ability to reproduce research, has seen performance rise from 21.5% in September 2024 to over 95.5% by December 2025, with the benchmark’s own authors declaring it ‘solved.’ Similarly, in Kaggle competitions, AI agents now reach bronze-medal levels in about two-thirds of tasks, indicating a significant leap in automation of data science and model optimization tasks.
Experts such as Thorsten Meyer interpret these developments as evidence that the bottleneck in AI progress is shifting from engineering to research itself. Clark’s analysis points out that the progress across multiple independent benchmarks suggests a saturation point in engineering capabilities, which are now approaching a plateau. Meanwhile, research—particularly the creative and hypothesis-driven aspects—remains less automated but is likely to be affected as AI systems become more capable of generating novel ideas and experimental designs.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
AI Tools for Finance and Accounting Professionals: Automate Tasks, Save Hours, Work Smarter
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
![Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results](https://m.media-amazon.com/images/I/415+fSJacsL._SL500_.jpg)
Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
n8n Automation Platform Applications: Workflow Design and System Integration for Experienced Developers and Businesses
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
Hands-On LLM Serving and Optimization: Hosting LLMs at Scale
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications of Automation for AI Development Pace
The ability to automate core engineering tasks dramatically reduces the time and cost required to develop and reproduce AI research, potentially accelerating innovation cycles. This shift could lead to a future where human researchers focus more on hypothesis generation and less on routine implementation, fundamentally changing the landscape of AI R&D. However, it also raises questions about the future role of human creativity and whether AI will eventually automate research itself, not just engineering.
Progression of AI Capabilities in R&D Tasks
Over the past 18 months, multiple benchmarks have shown rapid improvements in AI’s ability to perform tasks traditionally done by human researchers, such as reproducing experiments, optimizing kernels, and participating in competitions. The benchmarks, including CORE-Bench and MLE-Bench, have demonstrated that AI systems are approaching or have reached the upper limits of these measurement scales. The broader research community has noted a pattern of saturation, indicating that AI’s engineering capabilities are nearing full automation, with some experts declaring certain tasks ‘solved.’
While these advances are recent, they follow a trajectory of consistent improvement since 2024, driven by larger models, better training techniques, and increased automation in infrastructure design. The question remains whether research—particularly hypothesis generation, creativity, and exploration—will follow this same rapid progression or lag behind due to its inherently less structured nature.
“The pattern across multiple benchmarks indicates that AI’s engineering skills are approaching saturation, effectively automating most core tasks involved in AI development.”
— Thorsten Meyer
Unclear Extent of AI’s Research Automation Potential
While engineering tasks have shown near-complete automation, it is not yet confirmed how much of the research process—particularly hypothesis generation, experimental design, and creative insight—can be automated. Experts acknowledge that research may itself be a form of engineering at scale, which could accelerate automation in this area, but evidence remains limited and speculative at this stage.
Next Steps in AI R&D Capability Assessment
Researchers and industry leaders will likely focus on developing benchmarks and tools to measure AI’s capabilities in research-oriented tasks, including hypothesis creation and scientific exploration. Monitoring progress over the next 12-24 months will clarify whether AI can fully automate research, potentially transforming the pace and nature of scientific discovery.
Key Questions
What specific tasks in AI research are currently automatable?
Reproducing research experiments, optimizing code and infrastructure, and participating in data science competitions are now largely automatable, as shown by recent benchmarks.
Will AI eventually automate all aspects of research?
It is uncertain. While engineering tasks are approaching full automation, the creative and hypothesis-driven parts of research are less developed in AI, but this may change as capabilities improve.
What are the implications for human researchers?
Automation of engineering tasks could free researchers to focus on high-level hypothesis generation and exploration, potentially accelerating innovation but also raising questions about the future role of human creativity.
Are there risks associated with fully automating AI research?
Potential risks include loss of human oversight, ethical concerns, and the possibility of AI generating unintended or harmful research directions. These issues require careful management as automation advances.
Source: ThorstenMeyerAI.com
