Google’s Chess AI Experiments Reveal Secrets to Boosting Artificial Intelligence

Diversifying AI Strategies for Creative Problem-Solving in Chess

Tom Zahavy, a researcher at Google DeepMind, recently rekindled his passion for chess after watching “The Queen’s Gambit” on Netflix and reading Garry Kasparov’s memoir, “Deep Thinking.” While Zahavy admits he’s not a great player, he enjoys solving chess puzzles that challenge players to find creative ways to gain an advantage.

The Limitations of Chess Programs Revealed

These puzzles have exposed the hidden limitations of chess programs. In 2017, mathematician Sir Roger Penrose devised a notorious puzzle that placed stronger black pieces in awkward positions. While an experienced human player could easily steer the game into a draw, powerful computer chess programs would incorrectly assess black as having a clear advantage. This discrepancy suggests that despite computers defeating the world’s best human players, they struggle to recognize and work through every type of tough problem.

Developing Diverse AI Systems for Creative Problem-Solving

Zahavy suspected that a program composed of multiple diverse systems, working together as a group, could make headway on these puzzles. He and his colleagues developed a method to weave together up to 10 decisionmaking AI systems, each optimized and trained for different strategies, starting with DeepMind‘s powerful chess program, AlphaZero. The new system, they discovered, could solve the Penrose-style puzzles more effectively than any single AI system.

With a population of agents, there’s a higher probability that the puzzles are in the domain that at least one of the agents was trained in.

This work suggests that teams of diverse AI systems could efficiently tackle hard problems well beyond the game board, providing benefits by looking for more than one way to solve a problem.

The Glitches in Reinforcement Learning

However, reinforcement learning doesn’t always lead to strategies that reflect a general understanding of the game. Peculiar glitches can occur, such as systems finding loopholes to cheat or skip levels in video games or getting stuck in repetitive loops. Zahavy suspected that AlphaZero’s blind spots might be tied to the system’s internal rewards, as deep reinforcement learning systems don’t know how to fail or recognize failure – a crucial aspect of creative problem-solving.

Embracing Failure for Creativity

As Garry Kasparov wrote in “Deep Thinking”:

Creativity has a human quality. It accepts the notion of failure.

AI systems typically don’t recognize failure and may continue trying the same unsuccessful strategies. Zahavy believes these “weird kinds of intrinsic rewards” developed during training lead to dead ends and getting stuck on challenges.

Antoine Cully, a researcher at Imperial College London, has built robots that can effectively brainstorm multiple different solutions to a given problem. He has also collaborated with researchers to use diversity in identifying promising new drug candidates and developing effective stock-trading strategies.

The Potential of Diversified AI Systems

Zahavy suspects that for AI systems to think creatively, researchers need to get them to consider more options. This hypothesis suggests a curious connection between humans and machines: intelligence may be a matter of computational power, and creativity could boil down to the ability to consider and select from a large enough buffet of options.

While a diversified AI system may not completely resolve the broader generalization problem in machine learning, it is a step in the right direction. There is still room for improvement, as the diverse approach is currently computationally expensive, and Zahavy believes there may be more solutions to explore.

I still [think] there is room to find different solutions. It’s not clear to me that given all the data in the world, there is [only] one answer to every question.

Original story reprinted with permission from Quanta Magazine, an editorially independent publication of the Simons Foundation whose mission is to enhance public understanding of science by covering research developments and trends in mathematics and the physical and life sciences.