Uncovering Human+AI opportunities through experiments
March 29, 2021
March 29, 2021
We’ve talked a lot about the human+AI approach, and for good reason. Humans and artificial intelligence working together can accomplish so much more than either can do apart. But it’s not always clear how to begin harnessing the potential of these powerful collaborations.
That’s where experimentation comes in. Purposeful experiments on the use of human+AI systems in your workplace can help you uncover the path to new business value. We’ve done experiments with the human+AI approach, and the lessons we learned can help you to craft your own experiment and use it to ensure that when there is change, it’s for the better.
And it’s more important than ever to ensure that we’re implementing AI thoughtfully. 2019 focused the business world’s attention on the needs of stakeholders beyond the shareholder, including the need for greater security for employees and greater equality of opportunity in communities. The pandemic and recession of 2020 deepened those needs. Then, at the end of 2020, research revealed that machines were taking over tasks from humans faster than ever, the result of shifts driven by the pandemic and recession. And it’s well understood that those with lower levels of skills and education are at the most risk, since automation and AI take on lower complexity tasks most readily.
As an AI tech researcher, building and promoting AI systems, I might have wondered whether we’re only making matters worse if I hadn’t learned otherwise from a prior experiment. It was run at our Lab and Accenture’s innovation hub, The Dock, in Dublin. My colleagues and I had co-created and tested a new role for workers who had an AI teammate. You can read more about our experiment at Sloan Management Review, but in short: bringing AI into the workforce doesn’t have to be a zero-sum game. It’s not human versus AI.
<<< Start >>>
<<< End >>>
In fact, our experiment proved that people gained valuable skills and knowledge while training an AI system designed to help them in their roles. People were enabled to teach their AI colleague, improving the accuracy of the process for customers. We then saw how the up-skilled workers could support the development of new products and services. Employee up-skilling, customer service improvement and enterprise value – all as a result of how we had experimented with AI.
For me personally, this was a uniquely interesting project. People often cite lessons from history that technology innovations create more jobs than they eliminate – but these had fallen short of being completely reassuring for me. What if this time it’s different? It certainly feels different from previous waves of automation. Machines are getting ever better at mimicking activities we consider especially human, like talking and reasoning. Can we make sure that new jobs are created, and that it happens in time to not leave people behind?
We ran our project as an experiment precisely because we had more questions than answers (these and many others). As you’ll see below, the experiment showed that the outcomes we valued were possible and illuminated practical steps to make them real. You can use a similar approach to map the steps from your organization’s shared values to new human+AI value in your business. Where do you start? Here are guidelines based on our experience, along with examples of what they helped us to learn.
For many teams, experiments are unfamiliar. Projects with more certain outcomes are the norm. Some may use “A/B testing” of alternatives, but that essentially precludes learning about “unknown unknowns.” Others might use the word experiment to mean “let’s try X, and who knows what will happen!” But if you haven’t done the homework to shape a crisp hypothesis, you might not be asking the questions that will result in the greatest learning.
We planned to test our hypothesis by enhancing an AI system that was already in place. Registered nurses working as medical coders had been using the existing AI system to help them annotate medical records. The records, produced as patients interact with care providers, are often unstructured. Annotating them with standardized codes helps with payment processing and data analysis that can improve patient care.
We found that the existing AI was not a medical know-it-all. Rather, it learned from the medical coders, who had extensive medical knowledge and clinical experience and used it to help improve the system’s outputs. That learning was a clunky process, though. It required coders to correct the AI several times before a data scientist was alerted to update the AI’s knowledge base. So, the relationship between the old system and the medical coders did not fit our vision of being “symbiotic.” To create symbiosis, we needed to tighten the loop between the coders and the AI. The AI would continue help the coders, but the coders would take on a direct role in teaching the AI.
Ultimately, the new role was a success. Medical coders who had no previous data science training learned to take on the role of training the AI. The accuracy of AI training by the coders was high, and they strongly agreed that they were acquiring new knowledge and skills; meanwhile, assessments also showed that coders could apply the new concepts to other scenarios. Eight out of nine coders were more positive about working with AI than they had been prior to taking part in the experiment.
It’s going to be quite some time until AI surpasses human abilities in many areas. Meanwhile, there’s much value to be had by teaming humans and AI together. Spotting that value and how to tap it sustainably and responsibly in your context might be best discovered through an experiment like ours.
This research was a collaboration between colleagues at Accenture Labs, The Dock, Accenture Research, Accenture Insights Driven Health and Dr. Claire O’Connell, Irish Science Writer of the Year 2016.
To learn more about our ongoing work in the human+AI space or about how an experimental approach can help you unlock its value for your business, contact Diarmuid Cahalane.