Over the next five years, federal agencies will need to master an entirely new world – the virtual one. Fueled by advances in artificial intelligence (AI), 5G, and miniaturization, extended reality (XR) technologies are poised to become more powerful and more commonplace. For example, the U.S. Army is planning to acquire up to 120,000 Microsoft HoloLens headsets worth almost $22 billion to improve soldiers’ situational awareness and training. Marco Tempest, a creative technologist with NASA’s Jet Propulsion Lab and Accenture Luminary for XR, notes that these technologies allow organizations to “invent the impossible” by bringing the virtual world to life. All of this raises an important question for federal leaders – what should this virtual world look like and how can they use it to advance the mission and serve the interests of the American public?
In today’s complex, digital-first world, XR will redefine how government trains its workers, delivers services to customers, and operates remotely. With disciplines like digital engineering, telehealth, and data fusion taking center stage, the ability to work virtually in a digital world has never been more important. XR is helping federal agencies reimagine how they can operate while making data more accessible and digestible:
As agencies seek to recruit and reskill workers for increasingly diverse roles, XR can deliver training that is more efficient, effective, and engaging.
2. Field operations
XR can improve field operations by precisely and intrinsically guiding workers through remote processes and bringing together expertise from anywhere to collaborate.
3. Situational awareness
XR can advance situational awareness and improve real-time decision-making through more impactful, data-driven visualizations.
4. Digital twins
Finally, XR can bring digital twins to life, with immersive experiences enhancing understanding of virtual models.
These tools are already enabling federal agencies to reimagine how they deliver on their missions. For example, a large federal agency used mixed reality headsets that project 3D images into a physical space to overcome travel constraints created by COVID-19. In this case, U.S.-based members of the agency conducted real-time visual inspections of facilities on the other side of the world, supported by visual data delivered by an on-site inspector — a vivid demonstration of the power of XR to enhance government operations regardless of geographic limitations.
In the post-pandemic world, federal agencies cannot afford to be followers when it comes to XR, and they no longer have the luxury to wait and see. Government will need extended reality to meet mission needs. To better understand XR’s potential for federal government, Accenture surveyed 115 federal technology leaders to understand how they are using or planning for extended reality; this report shares the findings, as well as XR use cases, challenges, and recommended next steps for federal agencies.
In the post-pandemic world, federal agencies cannot afford to be followers when it comes to XR, and they no longer have the luxury to wait and see.
To make the most of this opportunity, federal leaders need to consider how XR will impact and can enable their mission, and what technologies and skillsets are needed to master this new domain. They can begin by looking at their current business challenges, and exploring new ways of enabling services for employees and customers using the latest technologies. Leaders must begin a process of cultural change, looking for opportunities to leverage XR in support of increasingly diverse mission needs.
Key concepts in XR
Extended reality is an umbrella term covering a range of immersive experiences. All represent digital extensions of the real world, but their uses may vary. Among federal agencies already working in the XR space, Accenture found 60 percent have used augmented reality, followed by mixed reality at 54 percent, and virtual reality at 41 percent.
Virtual reality (VR) represents the far end of the spectrum: It offers a fully immersive experience, with the user deeply engaged in a simulated environment. In this mode, users operate within an artificial three-dimensional milieu via a headset or special goggles. Fully immersive experiences can be supplemented by sensor-enabled haptic gloves and other technologies that help deliver a realistic, albeit digitally simulated, experience.
Augmented reality (AR) offers a blending of the physical and digital worlds. Users can still see their actual surroundings, but the scene is overlaid with three-dimensional, interactive digital content. This includes concepts of mixed reality using advanced sensor and imaging technologies that enable spatial awareness, allowing the user to interact with that digital content naturally while still operating in real-world space.
VR can be especially useful in cases where the user needs to engage fully in a scenario that might be difficult to create in the real world. For example, when training for dangerous or highly unusual missions, it may be easier, safer, and more cost effective to ramp up a simulated digital experience than to pursue conventional training. In addition, VR can enhance remote collaboration by bringing geographically disparate groups together with less cost and travel.
AR can be valuable when you need to execute complicated procedures with specialized equipment, knowledge, or experience. In this scenario, the workers’ real-world efforts can be informed and guided by a digital overlay, with contextual data and other key content made available in real time to increase operational efficiency and adherence to procedures. This can include on-demand remote collaboration and digital step-by-step guides that bring the expertise required to complete complex tasks.
How XR can transform federal government
Now is the time for federal agencies to explore XR. The devices and technologies that support XR are fast becoming more affordable; you can buy VR headsets for just a few hundred dollars. At the same time, the simulations themselves are increasingly sophisticated, with a depth of detail and physical realism that mirrors the real world ever more closely.
The commercial world is already racing to XR. Research firm IDC reports that worldwide spending on AR and VR will see strong growth, forecasting a CAGR of 66.3% over the next five years. Over this period, spending by federal and central governments is forecast to grow by 90.5% CAGR.1 Stacey Soohoo, research manager of Customer Insights and Analysis at IDC, wrote in November 2020: "2020 has become a major turning point where enterprises and organizations across all verticals are embracing the unarticulated need for augmented, mixed, and virtual reality."
XR’s momentum in government
Most in government already recognize the need for bold action when it comes to integrating XR and see the application of XR expanding over time.
of federal technology leaders say XR is very or extremely important for meeting agencies’ mission needs.
of federal technology leaders say it will be very or extremely important by the end of 2026.
Empowered with tools to extend the human senses and reshape our relationship to our physical environments, government has the opportunity to dramatically enhance its performance through XR. The technology can be a powerful enabler of training, bring digital twins to life in complex ecosystems, and empower workers to increase their expertise, speed, and precision. From field inspectors to warfighters, the potential benefits extend across a range of federal missions.
The best way to understand the moment we’re in is to view it as similar to the advent of the smartphone. No one at first predicted how disruptive a portable computing device would be; yet today, smartphones support the federal government in any number of areas, driving business processes that support ever-higher levels of productivity. In the future, will we always carry a smartphone to perform many day-to-day functions, or will that interface be replaced by simple, smaller headsets that expand our interactions using XR?
The impact of XR promises to be just as far-ranging and equally disruptive, revolutionizing the employee experience. Digital simulations will give workers new tools in support of their efforts, while simultaneously empowering those same workers to deliver vastly improved services. In the “new normal” — at a time when remote connections are essential — agencies can leverage XR to maintain critical business operations and services where they are needed. Digital experiences will loosen the shackles of geography, freeing federal workers to access information from anywhere; to collaborate through tangible, hands-on interactions; to interact with each other and with citizens in profoundly powerful new ways.
Federal use cases for XR
Government leaders have begun to envision or adopt new use cases for XR, with successful implementations already driving enhanced employee and customer experiences. Accenture found that 59 percent of federal technology leaders use XR for remote work and collaboration, while 53 percent use it for operations/inspections, and another 53 percent for security and surveillance.
Federal use cases underscore XR’s potential, making clear that:
XR increases the efficiency, effectiveness of training
Faced with a potential looming wave of retirements, and under pressure to increase their pipelines for talent, federal agencies can leverage XR to transform the way they deliver training. Virtual experiences offer the promise of higher-value learning experiences, especially in scenarios that are inherently dangerous or difficult to recreate. Personnel can learn without having to worry about making mistakes that would be costly in the real world and can even come together virtually to train regardless of geographic distance.
XR training also represents a potential cost savings: It cuts down on travel time and reduces the expense associated with setting up and executing training experiences. Moreover, XR training is immersive, and it can be easily repeated. These traits tend to make it highly effective. Research from Stanford University and Technical University Denmark found learners recall more when using virtual teaching methods than with traditional methods, resulting in a 76 percent increase in learning effectiveness.
Evidence from a range of industries support these findings. The Society for Human Resource Management reports on a financial services firm that used VR tools to train its customer service teams, thus reducing the average time customers spend on hold by 50 percent. Harvard Business Review points to a supermarket chain that used VR to impart its core values to employees. The result: 48 percent of VR trainees learned all six key concepts perfectly, versus just three percent who trained using traditional methods. At the University School of Medicine in Atlanta, VR-trained surgeons make 40 percent fewer mistakes than surgeons who are conventionally trained.
Immersive training makes the trainee an active participant, driving deeper levels of learning and retention. And it doesn’t just benefit individual training but provides a more powerful approach for team training as well. Imagine a critical program milestone or emergency team deployment where virtual team members can come together in their target environment from anywhere in the world to meet, collaborate, and even practice likely scenarios they may encounter in a physical deployment together. The potential to integrate these teams virtually and practice in a digital world provides a foundation for improved outcomes and mission success.
XR training provides “in-the-moment” immersive experiences
XR training can also be a powerful tool for encouraging empathy. “The immersiveness of extended reality (XR) can help employees see things from one another’s perspective in even more vivid ways,” according to the authors of a Forrester report.2
For example, Accenture teamed with Goodwill Industries International to develop an innovative virtual experience called Project Overcome, designed to support people impacted by the criminal justice system who want to enter the workforce. It delivers a simulated interview experience in which users speak face-to-face with a human resources manager and hear from individuals who overcame challenges they faced when seeking and earning employment after incarceration.
The same approach could be used to help federal workers be more responsive in their own interactions with customers. For example, Accenture supported virtual training to help social service case workers respond to a range of possible scenarios. With storytelling and interactive voice-based branching scenarios, these virtual tools help front-liners identify their own biases and sharpen their decision-making abilities, improving their ability to fulfill their jobs.
XR augments field operations with critical skills, improved collaboration
The U.S. federal government supports diverse operations in the field, including in remote locations and around the world. In many cases, these activities depend on specialized skill sets, often deployed in unpredictable ways. XR can empower local employees with the missing expertise needed to execute complex functions, such as triaging a wound or fixing machinery.
For example, Accenture supported Airbus in creating a wearable XR system with smart glasses that display crucial information for manufacturing operators. The system improved production time on the A330 aircraft, enabling operators to mark seat placements six times faster and reducing errors to zero on the final assembly line.
The Department of Veterans Affairs is using 5G-enabled augmented reality to help doctors analyze and manipulate large imaging files, like MRIs or CT scans. Called Project Convergence, the work so far has mainly centered on training, education, and pre-surgical planning. However, the same technology is starting to be applied today during patient procedures to make them safer and more effective by providing advanced surgical 3D visualization and navigation.
There’s a powerful “over the shoulder” potential here as well. Picture a scenario in which a subject matter expert can virtually step into a situation and help to execute on a challenging aspect of the mission. In this way, XR can support complex tasks, integrating expertise where and when it is needed, in a way that is at once deeper and more tangible than what could be delivered in a mere phone call. Imagine this in the context of an aircraft mechanic or a heart surgeon: There’s compelling potential here to drive significantly higher levels of performance when XR enables real-time, hands-on collaborative encounters.
XR advances situational awareness
By redefining the way data is presented, XR can improve insights and deliver actionable intelligence in real time, heightening situational awareness and enhancing environmental visuals. This can have immense benefits for defense, law enforcement, and public safety agencies or any user seeking enriched perception for more effective and rapid decision-making.
Visual representations can lower the cognitive load for the user. From a first responder to a front-line warfighter, information shared through XR can be more digestible and accessible, enabling faster and more informed decisions when it matters most.
For example, the Drug Enforcement Administration is using AR to provide contextual information in the field by overlaying data such as street names, addresses, parcel data, business names and important landmarks directly over live video.
And the Army is developing its Integrated Visual Augmentation System (IVAS) to address capability gaps in the dismounted close combat force. Soldiers can use the system to “see through” a vehicle in order to access what the external sensors are seeing, thus dramatically enhancing their situational awareness.
XR brings digital twins to life
Digital twins, or virtual models of objects, processes, and ecosystems fed with real-time data, are improving government’s ability to understand and predict outcomes. XR can bring these models to life as immersive, 3D digital twins in both live operational environments and simulated exercises with synthetic data used to model any number of scenarios.
Consider how a digital twin of a warehouse’s manufacturing processes or a city’s transportation networks could be made more interactive through XR – allowing users to step into the model, view from different perspectives, and potentially even interact.
These visualizations can include representations of both the physical and the digital. An AR-supported digital twin could, for example, be used to visualize an IT system to bolster cybersecurity. “While a system’s imminent failure due to attacks might not be visible to an operator’s eye on site, adding additional information from the [digital twin] via an AR device might reveal previously invisible misoperations,” write researchers in the Journal of Cybersecurity and Privacy. “Moreover, AR-[digital twin]-intertwined devices might display visual representations of the system’s logs or network traffic between systems imposed on the physical devices, enabling visual and contextual intrusion detection right next to the physical systems…” This allows for direct intervention with them.
Federal XR pioneers in action
A variety of other early use cases demonstrate how agencies with a broad range of mission sets are already leveraging XR.
The VA is using XR to treat PTSD. It is leveraging virtual reality to deliver “prolonged exposure therapy,” which involves recalling a traumatic memory while talking through the nuances of that memory with a therapist. VA reports it is easier for some veterans to confront their memories and talk through their experiences in a virtual space. In addition to PTSD, the VA is leveraging XR as a possible means to treat anxiety, depression, and chronic pain. The Veterans Health Administration is using XR at more than 50 sites, with over 200 VA employees actively involved with the effort.
Meanwhile, NASA has tapped virtual reality to drive scientific discovery. Their VR study of groups of stars has revolutionized the classification process, helping us to better understand how our galaxy evolved.
We also see a wide range of uses emerging in the national defense and homeland security sectors:
The Homeland Security Department’s Federal Protective Service has put out a request for proposals to equip law enforcement personnel with body-tracking devices and head-mounted displays: Virtual immersion would be used to train officers on use of force and de-escalation techniques.
The U.S. Air Force Academy is using VR to transform the way B-52 Stratofortress student-pilots train for combat. Instructors report the virtual training helps reduce human bias in instruction, provides better access to training for student pilots, and gives students immediate feedback that lessens the chance they develop poor habits in the early phases of training.
The Defense Innovation Unit backed an experimental “sensor suit” that combines augmented reality, haptic signaling, and hands-free communication, enabling more advanced human and machine interactions.
Challenges to federal XR adoption
While emerging use cases help to demonstrate the promise of XR, there are still technical and cultural challenges that need to be overcome in order to achieve its widespread implementation.
Agencies’ top obstacles
In Accenture’s research, federal leaders raise a number of key concerns with increasing their XR adoption.
Existing IT policy restrictions/compliance
Security and privacy concerns
Insufficient network bandwidth/processing power
Lack of enterprise-grade hardware/software partners
In terms of policy, IT teams should be thinking now about the ramifications for security and device management. XR technologies including goggles and headsets represent yet another endpoint in the ever-expanding galaxy of devices. They’ll need to verify that those devices are secure, and they will need to implement policies and procedures for device management — ensuring devices are all accounted for and up to date. A robust device management plan ensures devices can be appropriately distributed, patched, maintained, and wiped if necessary. Agencies have a head start here: The same practices that currently support vast inventories of agency-issued mobile devices likely will extend into the XR realm.
Bandwidth considerations also come into play. Agencies deploying XR will need to ensure users have ample connectivity in support of an immersive experience. Depending on the volume of XR experiences deployed in a given location, network management may require some special attention. Agencies should be looking now at 5G network connectivity to deliver the low latency and high bandwidth needed to enable XR in edge and remote scenarios.
Finally, the race is on for the next wave of hardware and software. Companies like Facebook, Microsoft, HTC, and Varjo are heavily investing in the space which will drive out smaller, more flexible headsets. Apple and Google are other examples of companies that continue to invest in XR, including expanding software capabilities like ARKit and ARCore.
It’s significant that we are seeing the presence in the market of such major players. They and others are delivering software development kits for building AR capabilities, with ready-to-deploy applications already on the market. And the hardware constraint is rapidly disappearing: With XR no longer a niche space, there are commercial products available that can be made government-friendly for federal implementations.
How to get started with XR
For federal agencies looking to leverage the emerging power of XR, strategic investments today can help to set the stage for tomorrow’s successes. They can begin with some key steps:
1. Determine where XR is needed most
Build a three-year roadmap aligning potential use cases with emerging capabilities.
Agencies can begin their XR journey by defining the mission cases where it will be most impactful. The time is ripe to initiate a culture shift from leadership out to the field, with all stakeholders encouraged to identify likely use cases for this powerful new capability.
On the training side, the low-hanging fruit is any experience that is prohibitively expensive or dangerous to recreate. In terms of field operations, agencies can be looking for areas where more immersive data visualization can improve job performance and decision-making. Situations like these are ripe for XR enhancements. For advancing situational awareness, agencies can explore which information should be brought together in new, visual ways to make the data more actionable or digestible.
It’s helpful to think of this process in a three-year timeline. This scope allows agencies to map out how their missions can be transformed both in the near- and long-term, while still allowing flexibility to integrate not-yet-realized technologies beyond the three-year mark.
Iteration is key here. Rather than revamp an entire work process, it may make sense to carve off some small piece, to look for an area that is most readily fixable. It helps, too, to find areas where progress is most easily measurable, where metrics show that key procedures aren’t being met, or where there are high-value assets at risk. Rich data builds the business case for future XR investments.
2. Build out teams
Establish a Center of Excellence with requisite skillsets.
Agencies should begin investing in people: They can start to build the teams they will need to support widespread XR adoption. One first step is establishing a Center of Excellence (CoE) that can lead XR strategy, development, and program management for the agency.
This group can bring together the business and IT side to explore how this emerging technology can be applied for the agency. Like with any new technology, people need to understand the realm of the possible to reimagine how they execute their agency’s mission.
The CoE will need traditional IT skills such as infrastructure, network, and application developers, as well as systems engineers and project managers, who understand how these skills can be adapted for XR development and deployment specifically.
However, XR also requires new skills in addition to IT, including:
Creative storytellers to plan out the immersive experience in an engaging and logical way
Graphic 3D artists to design the virtual environments, objects, and features
3D game engine developers to cohesively bring the storytelling and artistry to life
User experience designers to customize the XR experiences to the individual
To build these teams, like with every other new technology, there’s an element of upskilling your people. The CoE can begin now to socialize with employees what the XR landscape looks like and how it can benefit the mission.
Agencies may need to recruit outside of their traditional pipelines. They can look to the video game industry as both a source of inspiration and as a potential talent pool; bringing XR to life will require the storytelling skills and experience with 3D engines such as Unity and Unreal Engine that have long been prevalent in this field.
3. Ensure a strong foundation
Implement XR technology architecture and infrastructure.
Agencies need the appropriate technology architecture and infrastructure to support XR adoption.
Cloud, edge computing, and 5G networking will all play a key role in enabling XR. Widespread, successful XR deployments will require future-ready compute capabilities and network infrastructure, with the ability to support high-performance, high-definition operations both within the traditional perimeter and at the edge.
For example, leaders need to assess the bandwidth impacts of these devices on networks and may need to consider 5G implementations where WiFi is not viable but high bandwidth and low latency is still needed for a high-quality XR experience. When data processing needs surpass network capacity, agencies can still provide a high-fidelity experience by offloading compute intensive programs to the edge. Ultimately, understanding how an agency’s technology architecture can intersect to provide an impactful XR experience, regardless of bandwidth, location, or use case, will be key to long-term success.
In terms of hardware, agencies can assess if their XR implementations can be viewed in 2D on existing screens, or if they want to invest in 3D experiences that require more specific devices, such as headsets. For those that are exploring headsets and goggles, it’s important to understand that the available options are rapidly evolving. These should not be viewed as long-term or permanent investments, but rather as an opportunity to take advantage of the best of the market at this point in time.
Lastly, XR leaders must understand how the technology will integrate within an agency’s existing cybersecurity framework. Now is the time to put in place the foundational elements, best practices, and security policies so that as new and better hardware and software emerges, IT will be able to upgrade seamlessly and securely.
4. Integrate human-centered design
Take a human-centered approach to XR deployment
Critical to the success of XR is the use of human-centered design as a means of ensuring XR experiences are engaging and effective. This requires an understanding of users’ needs, and a program in place to train workers how to interact with and interpret XR technology and data. As deployments scale, agencies must ensure they are equipping users with the appropriate knowledge and skills to make the most of their investment.
For example, the user experience for XR can differ dramatically compared to what people are used to from web or mobile interactions. Agencies should plan to onboard capabilities incrementally when it comes to XR, particularly for more immersive experiences, to avoid sensory overload.
It is important that agencies bring together the right skills to create the content and experiences – including experts in spatial cognition understanding, 3D experiences, designers, and the subject matter at hand.
Co-creating the XR experience with its users across all levels of the agency early and often will be critical not only for a successful outcome but also to increase confidence in the usability and value of XR overall.
Even with the right expertise creating and deploying XR, though, it is equally important to have the right analytics and measurements in place to validate the outcomes.
5. Monitor and iterate
Track, assess, and improve performance.
XR is an emerging technology with rapidly evolving software and hardware. Given that, agencies should be constantly monitoring and iterating on their XR deployments to ensure they are effectively meeting mission needs and optimizing use of available tools.
Continuous feedback can be a valuable tool for measuring success at the user level. With sensor-enabled devices, it may be possible to collect real-time feedback, which in turn can be leveraged to continually fine-tune the user experience.
In practice this means program leaders should engage with the employees, soldiers, and other end users either via discussion or questionnaire after each XR experience. They need to explore functionality: Could users see the images? Did the flow of the experience make sense? They can also ask about impact: Did this experience help them more effectively complete a task, for example? This human feedback loop can inform future iterations and provide a greater understanding of which technologies or visualizations are most useful in a particular scenario.
Agencies also need to consider results: Did the XR experience fulfill its intended function in terms of either training or mission support? By monitoring metrics including retention of training materials, productivity, speed of decision-making, or quality of customer service interactions, agencies can better understand how the addition of XR is improving mission outcomes.
XR is rapidly advancing and will transform how enterprises operate. Federal agencies should move forward now on this powerful technology or risk being left behind. They have the chance today to move the needle on training, customer service, field operations, knowledge sharing, and situational awareness, with virtual experiences elevating performance across diverse missions.
Given the rapidly evolving XR landscape, agencies should consider teaming with a partner that has a depth of experience in the field, and a track record of delivering needed technologies, particularly at scale. Prototyping and testing technology is one facet of the process, but for a robust federal implementation, scalability and sustainability over the long term will be the hallmarks of success. Agencies will want to team with industry partners who have successfully navigated that territory.
XR will add new dimensions to how the federal government can operate, with enhanced insights and immersive experiences improving how employees can train, execute in their jobs, and interact with customers and each other. And the truth is – XR is becoming increasingly ubiquitous and the convergence of edge and cloud architecture with high-bandwidth, low-latency 5G networks will further accelerate the widespread adoption of XR experiences. Agencies can’t ignore it and should look now at how it can best be integrated in their missions. By adopting this emerging technology earlier on, agencies will be able to experiment with and evolve their usage as new capabilities emerge and XR’s momentum builds.
1 Source: IDC, Worldwide Augmented and Virtual Reality Spending Guide 2The Extended Reality Opportunity Today: Your Employees, Forrester Research, Inc., February 21, 2020