The power of man and machine to innovate together…at AWS re:Invent 2017 and beyond
At AWS re:Invent 2017, we set out to create an unforgettable experience for attendees. The objectives? Having fun, yes, but primarily demonstrating how the combined power and innovation of AWS and Accenture can deliver intelligent services in the cloud – services that have the potential to transform our clients’ businesses.
In just 10 weeks, using AWS technology, Accenture teams developed an immersive game called “AI-CE”. This uses deep reinforcement learning to train AI how to play a Pong-like game with people. Some of us are old enough to remember this arcade classic. We reinvented it, with real-time back-end updates so that the game would continue to pose new challenges.
So how does AI-CE work? Detected by motion sensors in a connected device, players’ hand gestures move a paddle left and right. Onboarding via iPad, they’re recognized with AWS Rekognition, before playing three games against the AI player, trained with a reinforcement learning (RL) algorithm. During games, Amazon Rekognition real-time sentiment detection captures players’ emotions.
Game stations at The Quad at Aria
At AWS re:Invent, game data was sent from the four on-site gaming stations to the AWS cloud and displayed at “Mission Control Center” showing all the real-time game metrics, system architecture and algorithms.
“Mission Control Center” at Executive Summit
So how did it go? Over the course of four days, we captured insights from all 4,885 rounds played. More than 1,200 unique players completed all three game levels – Basic, Intermediate, Advanced – in consecutive order. 212 of them played more than once…nine played more than 10 times. Over 900 companies took part, with Accenture, Amazon, Capital One and Google in front with most games played.
Continuously learning from human input, the AI got smarter and smarter as AWS re:Invent went on. With every new player, it picked up new insight and became more skillful. So skillful, in fact, that the team had to tweak the code to make sure it was still fun for participants to play. It was a great way to demonstrate our leadership in innovation…and what’s possible when we use AWS technology to apply the new now.
So how does it work?
The core technology behind AI-CE is Deep Reinforcement Learning, which brings together deep learning (DL) and reinforcement learning (RL). These technologies are currently the cutting-edge of AI R&D in Silicon Valley and leading academic communities. With RL, instead of humans giving explicit instructions, business logic or operation rules, the technology itself trains AI models to perform intelligent tasks.
With traditional supervised machine-learning algorithms, a model is created offline by training a large dataset as a batch. The big advantage with RL is that it gets computers to learn like people – continuously, from incoming streaming data.
Although RL has been around for a while, it really started making waves when it was combined with DL (networks that are capable of learning unsupervised from unstructured data). That was a real game-changer. And when AlphaGo, a program trained using deep reinforcement learning, won a three-match series against the world’s best Go player in 2017, people really began to take notice. It was a breakthrough moment that prompted MIT Technology Review to identify RL as one of 2017’s “10 Breakthrough Technologies”.
To create AI-CE, we implemented deep reinforcement algorithms with TensorFlow (the most popular deep learning framework available today) and trained them on Amazon EC2 P3 with Amazon Deep Learning AMI. It was a powerful demonstration of what AI applications built on AWS can deliver.
Amazon EC2 P3 enables the ultra-high speed GPU-to-GPU communications that are so critical when training deep learning models with TensorFlow or other DL frameworks. Usually building a model requires significant computing resources. For our AI game, we ended up training hundreds of models with various hyper-parameter combinations in a simulated gaming environment. It was the immense power of EC2, combined with its flexibility, that allowed us to create these models in such a short timeframe for AWS re:Invent.
The other core AWS technology in AI-CE is Amazon Rekognition. Instead of people having to go through conventional user authentication (typing username/password) before playing, AI-CE admits them through face recognition. And with more than a thousand people using the service at AWS re:Invent, it got it right almost every time. Although there were a few pre-registered faces that it couldn’t identify, as more advanced hardware (like 3D cameras) is introduced, we’d expect Rekognition’s accuracy to improve to the point where it will be able to satisfy more security-sensitive use cases.
Into the future
AI-CE uses DL and RL to demonstrate forcefully just how much these technologies can achieve when they’re combined with human ingenuity. And of course, it’s about much more than gaming. When intelligent technology and people’s innovative creativity are applied at the core of business – across every function and process – they can address even the most complex challenges.
And although there are few large-scale industrial RL applications today, the technology’s ability to continuously learn, in collaboration with people, will soon be transforming applications in areas from robotics, autonomous vehicles and simulation of manufacturing processes, virtual agents, and supply chain logistics.