VIII. All Fun and Games? The Educational Value of Augmented Reality in Museum Exhibitions

In an age of increasing competition for attention and limited funding, the public exerts great pressure on museums to innovate, often through the introduction of new technologies. Even internally, museum professionals prescribe digital applications as a solution to staying relevant in an age when museums can often seem stuffy, elitist, old, and out of touch with current trends.¹ In this experience economy, museums seek to provide fun and novel opportunities for engagement and often look towards digital technology in order to do so.² One such technology is augmented reality (AR), which consists of supplementing a real environment with virtual objects, and thus is sometimes referred to as “mixed reality.”³ In this paper, I argue that although some museums have utilized AR in interesting ways, most museums are not at the point where they can effectively support AR. Instead, they should focus on low-tech alternatives to engagement. Although AR has great potential, museum professionals will need to think critically about how they incorporate it in exhibitions and the impact it has on visitors’ experiences, learning, and memories.

Defining Augmented Reality

AR was first introduced as a simulation training tool for airline and Air Force pilots during the 1990s, but only recently has become more common in educational settings.⁴ It has rapidly evolved from a bulky computer-dependent system involving a backpack and glasses⁵ to location-aware and context-sensitive AR apps that either track phones or scan and recognize 2D and 3D objects.⁶ The most popular example is the Pokémon Go mobile app that uses location tracking and cameras to impose Japanese cartoon characters on the natural environment as seen through a phone camera lens.⁷ More commonly, people use AR every day when using particular Snapchat or Instagram filters. Additionally, Google’s AR Core and Apple’s AR kit and other software has made it easier and less expensive to integrate AR into mobile experiences.⁸

How Are Museums Using AR?

Museums around the world have been using AR for years to bring their collections to life in various ways. The app ARART, which premiered in 2012 at an exhibition in Sapporo, Japan, brings famous paintings to life. For instance, visitors can see the Mona Lisa wink and move.⁹ The Art Gallery of Ontario, Toronto (AGO) uses the app ReBlink to reimagine paintings in the twenty-first century by adding modern groceries, smartphones, and Starbucks cups to old paintings. At the Florida Museum of Natural History, AR on a wall-sized screen allows visitors to interact with life-sized animated mammoths alongside the rest of their group.¹⁰ At the Kennedy Space Center in Florida, holograms of astronauts bring the space program to life.¹¹ In London, the Tate recently partnered with Facebook to create AR experiences that increase the depth of interpretation for eight paintings. In one, the painting is transformed into a three-dimensional landscape. In another, the artist is brought to life to finish her self-portrait.¹²

One of the best-known examples of AR in a museum is found at The National Museum of Natural History (NMNH), which released an app in 2015 called “Skin & Bones” that superimposes 3D models and animations onto skeletons in a hall that has not changed much since it opened in 1881. These augmentations are meant to highlight the animals’ unique features, drawing visitors’ attention to particularities of their functional anatomy.¹³ However, when I tested the app in October, it felt like an awkward and unhelpful addition to the exhibit, and glitches prevented me from exploring all the features. The app contains a wealth of information, including videos, diagrams, and audio recordings, but almost none of it directly utilizes the skeleton on display. The app could easily be explored at home without the lack of the exhibit having any impact. Many visitors to the Smithsonian are tourists who want to see as many museums as they can in one weekend. As such, time pressure influences what content they choose to engage with in exhibitions, and many do not stay to watch videos.

Skin & Bones illustrates some of the challenges that museums face with AR. While all the information in the app is interesting, there may be too much for a visitor to absorb while they are in the museum trying to complete their visit. Technological difficulties were frustrating and limited the functionality of the app. While the content is interesting, very little of Skin & Bones relies on the objects in the exhibition. For digital technology to make a meaningful impact to the museum experience, it is critical for museums to design the tech to be an integral part of the overall holistic experience rather than just an add on for the sake of providing a digital engagement.¹⁴

Proposed Benefits of AR

AR is often touted as offering increased interaction, participation, personalization, and flexibility, yet many physical interventions can have the same effects. Museums are constantly looking for new ways to add layers of activity that provide successive surprises and discoveries for those who want to explore further.¹⁵ This can be done simply by having drawers to open and panels to flip or by incorporating additional senses with smell jars or audio tours.

One argument in favor of AR uses the idea of “social presence” or the “sense of being in an environment.”¹⁶ The less people feel their experience to be artificial or mediated, the more strongly social presence occurs. For example, an audio guide is more mediated than smoothly integrated AR, so AR is considered more memorable and pleasurable and less distracting because it helps people remain aware of their surroundings. In a similar vein, there is the claim that having a device recognize an object itself without any kind of label or QR code allows visitors to focus more closely on examining the object, thus enhancing interactivity.¹⁷ This kind of seamless integration of information is thought to promote Csikszentmihalyi’s concept of “flow,” a subjective psychological state of control, attention focus, curiosity, and intrinsic interest.¹⁸ However, some studies have found that AR can prohibit the user from developing a relationship with the real world and their surroundings because of the distractions of the simulation.¹⁹ Clearly, more work needs to be done to determine the true benefits of AR.

Can AR Improve Learning Outcomes?

Much of the research on AR has been about how to increase engagement, interest, and usability rather than on what visitors learn and how learning can be improved.²⁰ Museum theorists like Eileen Hooper-Greenhill have worked to redefine learning, especially in informal environments like museums. Museums are able to create unique learning opportunities that focus more on constructivist inquiry-based learning and dialogue rather than passive transmission of information.²¹ Constructivist theories of learning hold that learning depends on people’s prior knowledge, experience, and interests, and that the needs and motivations of visitors should guide the structure of exhibits and programs.²² Hooper-Greenhill expanded the concept of learning beyond memorizing facts to experiences involving knowledge and understanding, utilization of skills, changing values and attitudes, enjoyment, inspiration and creativity, and behavior progression.²³ In addition, John Falk and Lynn Dierking’s holistic contextual model of learning has greatly influenced the museum field’s conception of learning as involving the never-ending integration and interaction of personal, sociocultural, and physical contexts over time in order to make meaning of the world.²⁴ Therefore, the effects of a museum experience on a visitor’s learning can be very hard to measure and might even manifest a considerable time after the museum visit.

Intrinsic motivation, choice, and control have great impacts on learning outcomes. In informal learning environments that encourage free-choice learning, like museums, many pathways and media are provided for visitors to exercise choice and control, thus encouraging lifelong learning.²⁵ Mayer’s theory of multimedia learning states that incorporating emotionally and motivationally appealing design features (e.g. attractive content) can increase cognitive engagement and retain attention.²⁶ As stated before, AR can increase the psychological state of “flow” by enhancing learners’ control over their experience. This leads to positive emotions that improve learning outcomes through their influence on motivational, cognitive, and meta-cognitive processes.²⁷ AR can help personalize a visit by offering variations based on user age, knowledge level, language, interest, time in the museum, and favorite exhibits.²⁸

AR can help visitors visualize invisible and abstract concepts, as well as entities that are spatially and temporally distributed.²⁹ For instance, AR can be used to overlay old photos onto modern landmarks or buildings, helping people see how cities have changed over time.³⁰ Similarly, AR can be used to show visitors how a painting was first sketched out and then changed through different renditions.³¹ At the moment, museums are doing this with photos, but as the theory of social presence argues, the less mediation there is, the more seamless and enjoyable the experience. AR can be used to reveal more about an object without damaging it. For instance, it is no longer considered appropriate to unwrap mummies, but scholars learn what is inside their wrappings with CT scans and other technology. AR can bring those images to life for the public. AR can be particularly effective in science museums for illuminating imperceptible phenomena like magnetic fields. Dynamic visualizations are often more authentic and informative than drawings or other static images. For example, in one study, students at an unnamed museum were able to see how their actions changed a magnetic field and to associate these changes with specific details and aspects of the magnetic field.³² Similarly, AR can bring past contexts into the present, such as providing the literature, poetry and music of the time when an artist was working or short commentaries by curators as audio clips.³³ Museums are already finding ways to do this in low-tech, cheap ways. For instance, in the Smithsonian American Art Museum’s (SAAM) exhibit American Myth and Memory, the introductory text panel provided information about connecting to a curated Spotify playlist so visitors could listen to music related to the artworks as they navigated the gallery. While there are many ways AR could add new types of information to an exhibit, museums must question whether cheaper technology like static images on a tablet can accomplish the same effects before investing considerable resources in an AR app.

Drawbacks to AR

AR can absolutely increase the amount of information that is made available, but museums may need to be wary of providing too much information. An illustrative example of the challenges that opportunities for endless information create comes from the Getty Foundation’s Online Scholarly Catalogue Initiative (OSCI), wherein eight museums developed online scholarly catalogues. They found that digital contains the potential for infinity, but the constraints of time, resources, and attention require limits. The curators had to be able to finish the project at some point, even as they kept open the possibility of additions and revisions, and they did not want to overwhelm the reader with a labyrinth of links and endless scrolling.³⁴ The same issue can be applied to museum exhibitions. A large study conducted by Beverly Serrell in the 1990s found that visitors typically spend less than twenty minutes in an exhibition, regardless of its topic or size.³⁵ Adding extensive layers of information might increase the time visitors spend in exhibitions but more likely will just exhaust them faster. Alternatively, visitors may be in such a hurry that they will not engage with the expensive AR at all.

Some of the potentially negative effects of using AR in exhibitions are due to practical considerations. Clusters of people trying to view an object through their phone screen could cause traffic jams in gallery spaces. Overcrowding leads to obscured lines of sight, which cause augmentations to fail. Inconsistent, poor lighting can cause the AR app to fail at environment recognition, but minimal lighting is typically required for conservation reasons in museums.³⁶ Increased time with each object could mean increased museum fatigue, requiring museums to create more and better seating to combat it.

Other practical concerns prohibit the widespread adoption of AR. Many visitors do not have unlimited data packages. Some are traveling overseas and do not want to use their data. Therefore, museums wishing to support and promote these experiences using visitor devices might have to increase their free WiFi offerings, especially in older buildings, over which structural and financial roadblocks loom. AR apps can be quite draining to phone batteries, so museums might wish to provide charging stations for visitors with abruptly dead phones. Location-based AR experiences rely on GPS, which cannot penetrate through walls or distinguish between floors at the moment, so museums would need to use marker-based or recognition-based AR or deal with visitor frustration when location services fail to work.³⁷ Many smartphones are “limited by factors like processing power, memory and storage, preventing the full use and integration of AR.”³⁸ Even if AR is proven to greatly improve educational initiatives, currently the many practical limitations of the technology will limit its potential and usage.

Continuing Questions about Using AR

The law is still quite vague about the lawful use of AR, which has allowed some unauthorized AR interventions. Who owns the virtual space? What can a museum do if an outside party “trespasses” on that space? There is currently litigation around Pokémon Go that might decide if it is legal for someone to place a virtual object on private property. This virtual violation of private space is already happening in museums. For instance, the unauthorized app MoMAR Gallery remixes Jackson Pollock’s paintings at the Museum of Modern Art in New York beyond recognition or replaces them with “guerrilla artists’ work.”³⁹ To touch upon a complicated subject, such distortions of original works might violate aspects of copyright law, especially moral rights like the right to the integrity of the work, for objects not in the public domain. Both public and private museums will likely take issue with official curation being overlaid by unauthorized images. Such digital interventions do not harm the original paintings or physical exhibition design, but might affect the public’s interaction with an object and the institution. While some might encourage such participation, museums will likely want to create boundaries in terms of explicit content or authorized partnerships in order to protect their brand and ensure coalignment with their mission.

Damjan Pita, the brains behind MoMAR alongside David Lobser, said if art defines our cultural values, then those values are defined by the elite part of society. Apple’s AR kit and Google’s ARCore have made it easier for developers beyond the elite to build and distribute AR apps. Loic Tallon, then Digital Chief at the Metropolitan Museum of Art, was wholly supportive of such “AR invasions,” and said, “If someone is making an AR experience out of the collection, I see it as pure mission fulfillment.”⁴⁰ The Met’s statement of purpose is to collect, study, conserve, and present art in order to “connect people to creativity, knowledge, and ideas,” and it can certainly be argued that AR helps achieve this.⁴¹

The true impact of AR on the experience and educational value of museum visits still needs to be researched. Studies show that AR can help people focus, as well as inspiring curiosity and deeper engagement.⁴² However, it is unclear how much those effects are due to the novelty of the experience and new technology. In addition, studies have focused more on how people are entertained by AR instead of examining AR’s effect on how people learn. Impacts of technology on long-term memory need to be further investigated to determine whether learning through AR is persistent. For years schools have encouraged students to take notes by hand rather than on the computer because involving multiple senses and forcing the brain to synthesize material leads to more effective learning.⁴³

However, it is true that AR presents a new frontier of technology as it can involve multiple senses – audio and visual most successfully. Additionally, by including games or quizzes in AR apps, visitors can play with the content to synthesize it in new ways. As with many technologies, the impacts of AR on cognition and learning will depend on how it is used, the nature of the content. This nuance can be seen with more familiar technology like television and video games. For example, watching “Dora the Explorer” or “Clifford the Big Red Dog” is associated with an increase in vocabulary and expressive language skills in two-year-old children, while exposure to “Teletubbies” is associated with a decrease in both measures.⁴⁴ Action video games can result in improvements in a number of basic attentional, motor, and visual skills including improved short-term memory and ability to switch between tasks.⁴⁵ “Gamification,” or the adoption of game technology and game design methods outside of the games industry, could potentially be brought to AR to create the same positive effects.⁴⁶ The dynamics of gaming are founded on basic human desires involving rewards, status, achievements, self-expression, competitiveness, and altruism and thus are powerfully motivating factors.

Like any technology, museums must be thoughtful about what data they are collecting from the apps that host AR experiences, where they store that data, and what they do with it. At the Great Blacks Wax Museum in Baltimore and in the Tate Gallery in London, an early app let visitors bookmark their favorite exhibits. Based on that information, personalized merchandise – perhaps in the museum gift shop – was recommended.⁴⁷ Whether such a use is ethical is questionable.

AR can boost visitors’ willingness to opt in or even pay for a digital guide when such technologies are new, but it is not certain that these effects are sustainable.⁴⁸ Studies have shown that people spend more time with objects when they have AR applications, but this finding may be due to a novelty effect.⁴⁹ Once the novelty effect wears off, AR might not be seen as such a uniquely engaging and helpful interactive. In one study examining how AR influenced students’ understanding of the historical changes in an architectural structure, a participant’s phone died and the guide had to show print-outs of past images to facilitate comparison with the present structure. This did not hinder the learners’ enjoyment or identification of differences between past and present.⁵⁰ The lesson learned was as long as there is a guide and some learning resources, a low-tech version works just as well.

Lastly, whether AR is a useful technology for people to use in exhibitions or at home is another question. People already question whether phones are used too much in museums, concerned that it ruins the experience.⁵¹ ReBlink, mentioned earlier, was designed to engage rather than distract visitors by getting people to look up, rather than down at their phones. According to the AGO’s Interpretive Planner Shiralee Hudson Hill, 84% of visitors to this exhibition reported feeling engaged with the art because of the app and 39% looked again at the images after using the app.⁵² However, without knowing the baseline statistics, it is unclear if visitors spent more time with the art or if they got an experience that could not be accomplished on their computer at home.

In Conclusion

Although AR has been around since the 1990s, its potential and pitfalls for cultural institutions are only now starting to become visible. Technology should only be used if it is the best or only way to achieve a goal. If there is an alternative method that is just as effective, a low-tech, cheaper version should be used instead. In the end, technology has no positive or negative intrinsic value; it exists through the content it carries.⁵³ Yet while museums cannot afford to wait for technology to reach some imagined perfected state – since technology changes so much, so quickly, there is no such thing – they should be wary of investing considerable time and resources in new gadgets merely for the sake of novelty.

One fear is certainly unfounded, that technology like AR amounts to mere gimmickry and will turn the museum into an amusement park. People had similar fears about radio broadcast technology when it was introduced in 1952 to provide an alternative to docent-guided tours. Audio tours turned out to be one of the most transformative technologies for museums in the twentieth century, but one that is taken for granted today.⁵⁴ Over the past century, museums have dealt with the rise of new technologies by incorporating some of them in helpful and creative ways. Today, we live in a world dominated by digital technology. Smart systems control our homes, we depend on our phones for work, pleasure, health, and communication. Most institutions, from schools and businesses to hospitals and banks, have incorporated digital technology into the ways they are used by constituents. Paola Antonelli, a curator at MoMA said, “We live not in the digital, not in the physical, but in the kind of minestrone that our mind makes of the two.” Museums have an important role in helping people explore and understand the emerging hybrid culture.⁵⁵ Yet even museums cannot predict what the future will bring. Museums must flexibly adapt to a constantly changing world. Part of that means teaching the public about new technologies, but another part involves keeping the public engaged with effective learning methods that generate participation, critical thinking skills, and unique experiences, whether low-tech or high-tech.

---

Notes