The AI-powered robot that learned curling using Adaptive Deep Reinforcement Learning

The AI-Powered Robot That Learnt Curling Using Adaptive Deep Reinforcement Learning

In curling, a sport referred to as ‘chess on ice’ for its strategy and precision, a robot named Curly beat the Korean national teams in three of the four official matches. Robots have certainly come a long way, but they are still quite clunky, and most lack the dexterity of the human body. So Curly, the robot that mastered curling, is pretty impressive.

What is curling?

To fully appreciate this technological achievement, it is important to understand the sport of curling. Curling requires the physicality of bowling as players push a 40-pound stone into an ice pack from a boundary called a hogline after a target 30 yards away. The target for the stone is called the house with concentric circles – the closer you get to the target, the more points you get.

In curling, you play against a team whose players also try to throw their granite puck closer to the goal or take out yours to earn the most points. The curling strategy is about figuring out how to keep your opponent’s stone away from the house by knocking it out of position while doing this with enough finesse that your stone is aligned in an optimal position in the house. The trick is that the friction of the stone and ice causes the elements that the participants are confronted with always shifting during the match. Curling is not easy for humans and an incredible feat for a machine.

Curly and its creators

Klaus-Robert Müller of the Berlin Institute of Technology in Germany and his colleagues are behind Curly’s creation. Curly is powered by artificial intelligence, specifically an adaptive deep empowerment learning framework. The robot has two wheels at the front and a castor wheel at the back. It has a telescopic camera that reaches 7 feet in the air to help the robot see the house and another right above the front wheels to be able to see the hogline. Together with four smaller wheels in the shape of a U and driven by a conveyor belt, the robot grabs the stone with its front wheel. It is the U-shaped wheels that enable the robot to turn the stone, the curl that makes the stone turn to the right or left, a crucial technique in the sport.

To help Curly learn curling strategy, the development team created a simulation of a curling game that Curly could compete with and learn from. The challenging thing to simulate was the ever-changing conditions that occur in every game – the ice conditions and the shine of the stone and other physics of the sport. Human competitors must constantly adapt to changing circumstances. As a result, there was a gap between simulation and reality.

Before a match begins, competitors may be given test throws to learn about current conditions. Curly also did the test throws and then had to match the hands-on experience with the mathematical models it learned from. It was programmed to compare the current conditions experienced during the test throws with the training model and adjust as necessary.

In addition, during the match, Curly had to learn the best moves for the next throw, depending on the position of the competitor’s stones. In the simulation, Curly was given different scenarios and considered different throws, ultimately estimating the risk of each type. Using the knowledge it had acquired during training and then adapting to real-world conditions and the course of the competition, the robot adapted its plan accordingly to achieve success.

It is important to note that these matches between Curly and the Koreans were not an exact replica of the sport, as there was no swiping – the process of teammates sweeping a broom in front of the stone to scrub the ice to reduce friction and the stone to make. walking a straighter course – done by Curly or the Korean competitors in these competitions.

Ultimately, Curly’s training and development resulted in success, showing that artificial intelligence can adapt to real-world conditions. Olympic-level curling participants learn the nuances of their sport for 15 to 20 years. It was truly remarkable that a robot powered by AI achieved so much in such a short time and successfully adapted to the many variables that make up a curling competition. Curly showed that even if there is a gap between physics-based simulators and real-world conditions, artificial intelligence can bridge it. This will be important as other artificial intelligence systems are developed.

Leave a Reply

Your email address will not be published. Required fields are marked *