arrow_backNeural Digest
METR time horizon plot showing AI capability progress
Research

METR Updates AI Progress Graph: Coding Leads, Judgment Lags

MIT Technology Review
auto_awesomeAI Summary

“METR, an AI evaluation nonprofit, has updated its influential time horizon plot, confirming that certain AI capabilities continue on an exponential growth curve. However, the report reveals a meaningful gap: models perform significantly better at coding tasks than at tasks requiring nuanced judgment. This distinction matters because real-world deployment often demands exactly the judgment skills where AI currently falls short.”

Key Takeaways

  • METR's updated time horizon plot shows some AI capabilities still growing exponentially.
  • AI models perform notably better at coding than at tasks requiring judgment or reasoning.
  • The time horizon plot is considered one of the most widely cited yet misunderstood graphs in AI evaluation.

AI capabilities grow exponentially in coding but stumble where human judgment matters most.

Part of the 20 November 2026 AI news digest

trending_upWhy It Matters

The gap between coding performance and judgment performance has direct implications for anyone deploying AI agents in complex, real-world workflows. Organizations building autonomous systems may be overestimating model reliability in ambiguous situations based on strong benchmark scores. For policymakers and safety researchers, this uneven capability profile complicates risk assessments. Watch for how labs respond to the judgment gap — whether through new training approaches, retrieval augmentation, or more cautious deployment guidelines.

FAQ

What is METR's time horizon plot?

The time horizon plot is a graph developed by the AI evaluation nonprofit METR that tracks how AI models improve over time on autonomous task completion. It is widely cited in AI safety and capability discussions, though often misinterpreted.

Why do AI models struggle more with judgment than coding?

Coding tasks have clear, verifiable right answers, making them easier to train on and evaluate. Judgment tasks — such as weighing competing priorities or navigating ambiguous instructions — require contextual reasoning that current training methods do not reliably instill.

What does exponential capability growth mean for AI safety?

Exponential growth means capability gains accelerate rather than plateau, which compresses the time available for safety researchers to study and mitigate risks. It raises urgency around evaluation standards and deployment guardrails before models reach higher autonomy thresholds.

This summary was AI-generated. Neural Digest is not liable for the accuracy of source content. Read the original →
Read full article on MIT Technology Reviewopen_in_new
Share this story

Related Articles