What gaming tells us about the future of work
May 5, 2020
Jan is on a mission that requires intense concentration, constant multi-modal communication, rapid and coordinated planning, and mastery of complex controls. Physically, Jan’s teammates may be far away, but they are all connected via digital channels. A collection of displays shows real-time point-of-view imagery, as well as maps that identify where each team member is, and other relevant data. Success depends on everyone executing their role while staying coordinated, following the plan, and adapting to changing circumstances in real time. Some of Jan’s teammates are other people, but others are semi-autonomous robotic avatars.
Jan could be a teenager playing a multi-player online game (MPOG) like League of Legends. Or Jan could be a member of a human-robot team, controlling submersible robots on scientific missions.
Technology is letting organizations reimagine the future of teams and teamwork. As robots become more capable, human-robot teaming (HRT) is becoming more common. Soon, the typical work crew—whether it’s a surgical team or a firefighting brigade—could include people that never meet face-to-face, and teammates that aren’t actually people. Robot team members will take on physical work that is dangerous, distant, unpleasant, or that involves strength or senses that humans lack.
But are organizations prepared for the challenge of incorporating robots into their operations?
Photo credit: Chad Tubbs
What will it take to develop effective human-robot teams?
We recently collaborated with researchers at UC Santa Cruz, observing and interviewing robot operators to study how humans and robots work together today. These interviews suggest there’s a lot to learn from gamers: several skills and mindsets involved in playing MPOGs map directly onto the way future human-robot teams will operate.
Robot operators described themselves playing video games to enhance or prime their spatial/cognitive abilities. Drone operators we spoke to said it was common to get “in the zone” before a work shift by playing action and motion intensive games like Grand Theft Auto, even if those games don’t directly mirror their specific missions or the robots they work with. One operator said:
“I've been gaming all my life, too. Of course, all the pilots I know are huge gaming nerds... because... you're just decoupling from your body. Right, just your mind is connected to the vehicle remote and when you're playing these games like that, well, you're kind of doing the same thing. And it's a lot of controls... you're translating your hands into these different controllers, and that translates to some action on the screen. And it's very, very similar. So if you've been playing games your whole life, like I am, it really is no different.”
When cross-disciplinary teams of digitally-connected humans and robots become the norm, successful organizations will be the ones who build teams—including both human and robotic teammates—that can work effectively together. The connection to video games could influence how organizations approach this, from recruiting, to training, to building the support teams that will operate in this future model of work.
Mechatronic mind meld
Consider this: In addition to building the physical reflexes needed to rapidly control their virtual avatars, many MPOGs also help players build the kinds of communication and coordination skills that are going to be increasingly important in the HRT world. In many MPOGs, players need to track the progress of tasks carried out by human and robot teammates who are located elsewhere. That means using a combination of verbal interaction and available data displays and adapting plans effectively as circumstances dictate.
To do this well, both gamers and robot operators need to be able to “jump into” different perspectives, like the body of a game character or a robot teammate, and map their controls to that character or robot’s movements. They also need to be able to switch seamlessly between first-person and third-person perspectives to maintain a comprehensive understanding of their avatar and surroundings. Moment-to-moment movements to execute a plan may require seeing through the avatar’s eyes, while a strategic sense of the plan’s progress requires a third-person view. In all these cases, gaming experience offers lessons for the humans in human-robot teams and for the companies that need to hire them.
Enabling the future workforce
Of course, as HRT evolves going forward, the dynamics of participating in a human-robot team are likely to change. Today’s professional robot operators are self-selected specialists. Many have spent years developing the necessary reflexes and the mindset to use complex controls and rapidly digest a chaotic collection of incoming data. But as robots become more affordable and capable, demand for human robot teaming will increase. The role of robot operators will go from a specialized niche to a core capability for much of the future workforce. This has big implications for the design of robot communications and control interfaces. Designing for usability will be key: leveraging common interfaces and standard game controllers, or reducing information overload, will help empower a broader range of people who haven’t necessarily spent years developing these skills.
Increasing robot autonomy will also mean that the nature of teaming with robots will shift. Today’s robots—and many of the virtual avatars in today’s computer games—are usually under relatively direct control. Both operators and gamers must essentially micro-manage their robot teammates. But robots are already capable of making some maneuvers on their own, like avoiding obstacles. As autonomy increases, human operators can take on roles that are more like team leads. Over time, human operators might not be “operators” at all, but managers of fleets of robots rather than directly controlling just one. Even then, games will still have value for preparation and practice in this evolving space: today’s strategy games, where a player has more global responsibilities over non-player characters, may be the closest match to these skills.
Organizations are still learning what it takes to select and support today’s robot operators and help them work effectively with a distributed team. Understanding the connection between human-robot teaming and multi-player online games offers a path forward – today and for the evolving future of work.
We'd like to thank our collaborators in this work, Leila Takayama and Kevin Weatherwax from the University of California, Santa Cruz. We would also like to thank Naomi Nishihara for her help with this post. To learn more about human-robot teaming, check out our paper Humans, Robots, Teaming and the Future of Work. For more information, contact Alex Kass and Dave Vinson.
