AI Beats Chess but Struggles with Modern Video Games
Key Points
- AI has achieved superhuman performance in chess and Go.
- NYU researchers highlight AI's inability to adapt to new video games.
- Current systems rely on fine‑tuning for a single game and collapse with minor changes.
- Reinforcement learning requires millions or billions of simulated runs to succeed.
- Large language models perform poorly on unfamiliar games without custom scaffolding.
- True general game‑playing AI would need to learn new games within tens of hours.
- Limitations in gaming suggest broader challenges for AI in real‑world situations.
Recent breakthroughs have shown artificial intelligence surpassing human performance in games like chess and Go, yet researchers highlight a significant weakness: AI cannot readily adapt to new video games it has never encountered. A study from NYU points out that many AI successes rely on systems finely tuned to a single game, and performance collapses when rules or environments change. The research argues that true general intelligence would require learning a new game from scratch within a timeframe comparable to a skilled human player, a capability current AI lacks.
Impressive Wins in Classic Games
Artificial intelligence has captured headlines by achieving superhuman performance in well‑defined games such as chess and Go. These victories demonstrate the power of reinforcement learning and massive computational resources when the environment is fixed and the rules are unchanging.
Limitations Exposed by Modern Video Games
Despite these successes, a recent NYU paper stresses that AI’s abilities do not extend to the flexible, unpredictable nature of contemporary video games. Modern games demand a wide range of skills—including spatial reasoning, long‑term planning, trial‑and‑error learning, and even social intuition—that go beyond the narrow focus of classic board games.
The researchers note that AI systems excel only when they are meticulously engineered for a specific game. When even minor alterations occur—such as shifted colors or repositioned objects—their performance can deteriorate dramatically. This fragility reveals a gap between headline‑making achievements and genuine, adaptable intelligence.
Reinforcement Learning’s Trade‑offs
Reinforcement learning can produce remarkable results, but it typically requires millions or billions of simulated runs to reach acceptable performance levels. The resulting agents become experts in the exact scenarios they were trained on, yet they fail to generalize when faced with novel situations.
Large Language Models Also Fall Short
The study further observes that large language models (LLMs) perform poorly on unfamiliar games. When they do succeed, it is often because they rely on custom, game‑specific scaffolding that interprets game states, manages memory, and executes actions. Stripping away this support quickly reduces their effectiveness.
What True General Game‑Playing AI Would Need
According to the NYU researchers, a genuine game‑playing AI would need to learn a new game from scratch in roughly the same amount of time as a skilled human player—potentially tens of hours—without relying on massive simulation or prior exposure. Current systems are far from achieving this capability.
Implications Beyond Gaming
The inability of AI to adapt to brand‑new video games suggests broader challenges for handling real‑world unpredictability. While victories in chess and Go make compelling headlines, the performance gaps exposed by modern video games indicate that artificial intelligence still has a long way to go before reaching flexible, human‑like intelligence.