July 15, 2016
How can Mixed Reality help in Learning?
By: Grace T. Cheng

How is intelligence defined?

Experts have decided from ongoing studies that there are generally two main measures related to the quality and rate of a person’s learning—IQ and working memory. IQ tests measure things you might have already learned, similar to how a standardized test might test you on science facts, historical events or chemical compounds. In contrast, a common working memory test might be to remember a sequence of numbers in the reverse order that it was presented to you.

Let’s dig deeper into IQ first. There are two parts: fluid, and crystallized. Fluid IQ is linked to creativity and innovation. If you have really great fluid IQ, you can easily identify patterns, relationships and reasoning in different contexts, as well as extrapolate them to further conclusions using logic. Crystallized IQ on the other hand, represents knowledge gained through age, experience and memorization. A person with solid crystallized IQ would be an excellent anatomy student, or perfect at people’s names at a cocktail party.

Fluid IQ normally pulls information from your short-term memory since you might be figuring things out as you go along. Crystallized IQ pulls from your long-term memory. For times when you might use both short-and long-term memory (think going to the grocery store using a bike rather than driving like you normally do), you might use your fluid IQ more to rethink your habits in a new context.

Working memory on the other hand, is your active memory, which can be thought of as your brain’s temporary buffer (or RAM space) for your fluid and crystallized IQ to dump information in (Alloway, 2011). Many people use the terms working memory and short-term memory interchangeably, although working memory is technically the “framework of processes used for temporarily storing and manipulating information.” (Mastin, 2010) Short term memory is just one of its components.

Working memory sounds really similar to fluid IQ. The difference is this: time (Kaufman, 2014). Put two groups of people in a room, one with great working memory and the other with great fluid IQ. Give them a bunch of tests. If you include a strict time limit on these tests, working memory wins. Without the time pressure, fluid IQ catches up.

What improves learning?

  1. Working memory and cognitive load

    Let’s explain working memory in terms of cognitive load. It is defined as the mental effort used for working memory (Learning Theories, 2016). This load has three parts: 1) Intrinsic; 2) Extraneous; and 3) Germane.

    Intrinsic refers to the inherent difficulty of a new material studied. Imagine trying to teach a field worker the safety procedure at an oil and gas site using just text without any diagrams or maps. The learning material of the safety procedures already carry an intrinsic difficulty, but cramming that amount of information without visuals makes it a lot harder.
    Extraneous refers to the miscellaneous information that takes up mental resources but doesn’t actually help users understand the content. For example, a field worker receives instructions on an application displayed in a centralized location, perhaps a dashboard of metrics on a screen in a particular location for all users. Unnecessary time and effort would be spent running back and forth referring to instructions or filtering out irrelevant data from other users.
    Germane refers to the “effective” cognitive load that is put towards learning the actual material or achieving the goal in mind. In the case of the field worker example mentioned above, the germane load would be the safety content that the field worker needs to learn, rework into the context of his day to day life, and store in his short-term or long-term memory.

    Cognitive experts believe that focusing on working memory is a more accurate predictor of learning than simply measuring IQ. More generally, working memory has been suggested to be the single most important factor in determining intellectual ability (SüB et al., 2002), or the learning potential of a person. With that said, working memory capacity is also a better indicator of task performance (Kane, 2015) and attention control (Klingberg, 2010).

  2. Paying attention

    Paying attention is, of course, a requirement for learning anything. Regardless of whether the attentional capture is explicit (e.g., increased attention when your boss calls you out in the middle of a meeting) or implicit (e.g., dwindling attention due to the unbearable heat in the meeting room), a person’s working memory capacity has a major impact on the quality of his focus and cognitive performance (Conway et. al., 2001). Apart from storing and manipulating information to get things done, working memory also involves filtering out distracting information.

    One study used the “cocktail party phenomenon,” which refers to a situation in which one can attend to only part of a noisy environment, yet highly pertinent stimuli such as one’s own name can suddenly capture attention (Wood and Cowan, 1995). Results demonstrated that this ability is strongly related to working memory capacity (Conway et al., 2001). People who had a more robust working memory capacity held conversation longer and deeper than people without.

  3. Effort and time

    It is no surprise at all that learning is largely determined by how much effort somebody puts into a task, regardless of how “seamless” the learning experience was. A study involved giving two groups of people the same paragraph of text to read. One group was provided with blurred text and the other with clear text. Turns out that people who had the blurred text absorbed more information and did better on their follow-up tests than those who read the clear text simply because they tried a lot harder.

    Now this is really not to say that we should leave all our publications in Curls MT or Comic Sans. We still want frustration-free experiences. The emphasis is that reduced time does not necessarily equate to increased productivity, or retention, or learning. For example, especially in the realms of understanding how to best create training manuals for employees, it is really the quality of the time spent rather than the speed that matters.

    How can Mixed Reality (MR) help in learning?

  1. Frees up working memory for things that matter

    Mixed Reality (MR)

    Since our working memory is limited, we can use MR to limit intrinsic and extraneous loads to “free up more space” in the working memory for germane load, which is encouraged.

    Reduce intrinsic cognitive load
    We can use MR to design the content to be presented in a medium that is best for understanding. For example, we can use 3D visualizations for material with spatial reasoning involved. Think of the luxury of training assembly workers to put machinery together using 3D holograms. We can also design MR content to be broken into smaller segments and allow the user to control the pace of his own learning for a more customized and hands-free experience.
    Reduce extraneous cognitive load
    We can use MR to place content as close as possible to the working area to avoid running back and forth. MR uses image recognition to detect and place content in contextually-relevant places, and SLAM to track where the user is in the real world. There is no more need to filter out unnecessary data and overtax the working memory.

    We can also avoid visual clutter by building on existing mental models. MR uses image recognition technology to recognize hand shapes and their gestures to tie on to specific functionality in the programs. People can use natural, intuitive gestures based on what they already think the system should look like and behave.

  2. Increase focus

    Increase focus

    People with weak working memory have difficulty grabbing and holding on to incoming information (Morin, 2013). For times when long-term learning is not as highly prioritized, MR can simply replace the working memory people need so that they can focus on doing their task. For example, a technician with poor working memory capacity might find it challenging to follow multi-step directions to fix a one-off problem with a particular machinery. He might have trouble keeping in mind what comes next when he is doing the current step. An MR application can be designed to scan his surroundings, identify the broken machinery using image recognition and overlay immersive digital instructions for him to see.

    MR is also, by nature, an immersive environment. Strategic placement of MR content has the potential to sustain the attention of people during a task. Applications designed in immersive environments can encourage monotasking, or deep focus for just one task, simply because all other distractions are blocked off. The act of wearing a whole headset also makes one much more committed to completing a task at hand. For times when multitasking is more appropriate, MR applications can streamline the workflow to reduce the time spent while context switching between tasks.

  3. Hones skills

    Hones your skills

    MR can be used to enhance memory. Many research studies pointed to how drawing and moving enhances remembering (Klemm, 2016). Higher motor excitability leads to increased working memory performance, which is linked to improved fluid IQ. MR’s interface encourages engagement through moving, walking, glancing, gesturing and voicing in a very natural, intuitive way. MR holograms can be placed in the user’s own real world environment, providing the best level of customization and context for learning.

    MR can also be used to help hone a skill. People who aren’t naturally good at spatial reasoning could find the immersive 3D environment in MR extremely helpful. Our exploratory study in the Bay 2 Breakers expo found that 3D LEGOTM instructions were 87.6 percent more helpful than plain old 2D instructions.

A more intelligent workforce
Artificial intelligence for now, knowing the intelligence model of the human mind, helps us customize how we might want to train and influence our employees. Caregivers in hospitals and childcare, perhaps, might want to focus on improving crystallized IQ, which comes with lots of experience, muscle memory and deep understanding about their subject matter at hand. Software engineers, on the other hand, might emphasize fluid IQ. MR can be used in many different contexts with the user’s learning as a priority.

Alloway, T. P. (2011). Is Technology Making Our Brains Lazy? Psychology Today. Retrieved from
Kane, B. (2015). The Science of Analysis Paralysis: How Overthinking Kills Your Productivity & What You Can Do About It. Todoist Blog. Retrieved from
Kaufman, S. B. (2014) Working Memory and Fluid Reasoning: Same or Different? Scientific American. Retrieved from
Klemm, W. R. (2016) One Best Way to Remember Anything. Psychology Today.
Klingberg, T. (2010). Training of Working Memory. Pearson. Retrieved from
Learning Theories (2016). Cognitive Load Theory of Multimedia Learning. Retrieved from
Mastin, L. (2010). Short-term (Working) Memory. The Human Memory. Retrieved from
Morin, A. (2013). 5 Ways Kids Use Working Memory to Learn. Understood.
Williams, R. (2014) Slowing Down Can Increase Productivity and Happiness. Psychology Today.

Topics highlighted


Popular Tags

    More blogs on this topic