XVIAugmented Reality: An Edited View

  • Claire Griffin, The George Washington University

The relationship between museums and their audiences is rapidly evolving with the introduction of new digital landscapes. In this piece, I will be using the term digital landscapes to define any computer-generated element of an exhibition. Digital landscapes allow museum professionals to introduce ideas in creative new ways, either by adding to a pre-existing exhibition or by creating a new mode of exhibition altogether. In general, digital landscapes can be an extremely useful tool for adding additional context, helping the audience visualize, and producing a unique form of engagement, however, choosing the right digital platform to expand an idea can be a difficult decision. When making these considerations, it is important to choose a digital landscape that seamlessly integrates into the exhibition itself. With this framework in mind, museums should consider using augmented reality to create their digital landscapes, due to its flexibility in use and its potential for creating rich, complex histories.

Why augmented reality?

Augmented reality (AR) is the use of digital technology to superimpose a digital rendering on top of an image in real life, often using a smartphone camera. There are a lot of advantages to choosing to produce a digitally integrated exhibition with augmented reality, as opposed to a technology like virtual reality. Virtual reality (VR) uses a computer-generated simulation to depict or represent a three-dimensional environment, creating a totally immersive experience, often using specialized headgear or other equipment. Unlike VR, augmented reality is not a totally immersive experience. Instead, augmented reality enables the viewer to compare the real-world object alongside an augmented version of it, layering the digital information over the physical. Compared to virtual reality, augmented reality is relatively more affordable and accessible. While one might still need access to a smart device to use augmented reality, for virtual reality a participant needs a specialized headset in addition to much more complex and expensive software.1 Essentially, augmented reality allows an exhibition team to manipulate the digital landscape of an exhibit without requiring a whole digital world to be designed. As Matt Power and colleagues put it, “Though other technologies may perform the same function, rescaling in Augmented Reality systems provides the user a clear representation of spatial and temporal concepts as well as the extra advantage of contextualizing the relationship between the virtual object and the real-world environment.”2 Other digital technologies, like virtual reality, just do not have the same advantages of being able to layer information on top of an already existing exhibit, making it generally less useful and accessible for the field as a whole.

Augmented reality can therefore be an important technology in the creation of digital experiential learning environments. Experiential learning is the process by which an individual learns by actively participating in an experience.3 Museums have a long history of creating experiential learning environments, such as lift-up panels or other hands-on experiences. While these techniques are useful, they are often limited to what can physically be done in the space. Physical techniques are also nearly impossible to personalize for the guest, as the environment remains the same from person to person. By creating a digital landscape, the audience has the opportunity to customize how they see and interpret the environment of the exhibition. Not only does this help to customize the experience to an individual’s needs, but it allows the participant to become a part of the experience, assisting with the learning process. While this can be done through a variety of different digital technologies, augmented reality has some clear advantages.

A study by Moorhouse and colleagues found that schoolchildren in a museum setting that had the opportunity to use augmented reality found an “increased engagement with the learning environment, evoked curiosity, personal achievement, and motivation to continue learning with AR,”4. Moreover, in another experiment done by Peter Sommerauer and Oliver Müller in 2014 on a mathematics exhibit, wherein participants were tested on their knowledge prior to the experience and again at the end, participants did significantly better on the questions related to the augmented parts of the exhibition. Of the 101 participants, they found that “62 participants gained more on questions related to augmented exhibits, 20 participants gained more on questions related to non-augmented exhibits, and 19 participants showed no difference.”5 This is just a small fraction of the research done on the interplay between augmented reality and education, the vast majority showing that when used appropriately, augmented reality is a powerful and helpful tool for the learning process. By using this educational framework as a backing, augmented reality easily integrates into the museum setting as a form of experiential learning.

When augmented reality works

When designing an augmented reality experience, there are some lessons that can be learned from recent experiments in the field. For example, in the augmented reality system Mobile Augmented Reality Touring System (M.A.R.T.S) for Bayonne’s Basque Museum, the team developed a comprehensive augmented reality system to test its usefulness and engagement in an art exhibition compared to other systems, like labels and audio-tours. They found that visual assets were very helpful to the users, pointing out elements they might have missed otherwise in the artifacts. However, the team also implemented a “virtual human,” which was a digital avatar that gave narration similar to what you might find in an audio tour. Many visitors found this distracting and felt that they would have preferred audio tours in these instances.6 In this case, the successful elements pulled from the reference material to either point something specific out or manipulate the original artifact in order to give it more context. When a visual augmented reality element was added without expanding on the context of the original artifacts or source materials it interrupted the engagement of the user.

Augmented reality is also a powerful tool when working in less-than-pristine environments outside of the museum. In 2018, the National Portrait Gallery in London collaborated with several other institutions across the city to display art in the streets with their Art of London Augmented Gallery exhibit. In this case, curators designed a walking exhibit in the middle of London that presented rectangular outlines with a QR code. When passing bystanders took note and used the QR code, the rectangle was filled using augmented reality with a portrait of a famous Londoner.7 This use of augmented reality worked well for a variety of reasons. First, the National Portrait Gallery was enabled to engage with a variety of people they might not usually interact with, along with promoting the physical museums where these objects live. Secondly, they are protecting the physical artifact while developing high-quality renderings that allowed users to see the artwork outside of a museum context. Ultimately, this exhibition makes classic art accessible to a broader audience, and hopefully, encourages interest in people who might not have taken interest otherwise.

In a similar vein, in 2018 artist Alex Mayhew produced an augmented reality exhibition for the Art Gallery in Ontario called ReBlink. In this exhibition, he took classic pieces from the Canadian and European collections and reimagined them in the 21st century.8 In several instances of self-portraits, he added cell phones, making them into selfies. In another case, he added industrial smoke columns in the background of a scenic farm landscape. The big idea behind this digital exhibition was to see how different and similar these historical figures are to us. The exhibition is an interesting example of the use of AR in a museum context. When interviewed by the Financial Post via email on launch day, Mayhew spoke in-depth about why using augmented reality to manipulate the viewers' context was important to him. Most museum guests will only observe a work of art for 16 seconds on average. To encourage his audience, in particular the younger generation, to slow down and fully absorb the pieces they were seeing, his use of augmented reality transformed the pieces, making them more relatable to a modern audience.9 This approach allows the artwork to be reframed in a modern context by bringing it into the digital era, while also helping the audience to relate to the artwork more, promoting a general interest in the stories of peoples past. Like many of the other exhibitions we have talked about above, this exhibition is also included in general admission, allowing more accessibility to digital technologies than what might be possible with something like virtual reality, which often comes with an additional charge or fee.

The integration of augmented reality is not just a phenomenon we are seeing in art museums; it is being integrated across the entire field. In cultural and heritage centers, augmented reality can be used in unique ways. For example, augmented reality can show a historical site through a variety of different time periods. This can help the audience to develop a stronger sense of place for a historical site, which can lead to more interest in local heritage.10 Science centers and natural history museums are also trying to integrate augmented reality into their exhibitions, to varying degrees of success. Because of the inherently visual nature of AR, art and culture Museums have a natural path to using augmented reality seamlessly. For artifacts like bone, lithics, and ceramics, other ways of approaching augmented reality that focus on reconstruction or movement are perhaps the most powerful tools. For example, the Smithsonian National Museum for Natural History’s Bone Hall has an interactive augmented reality app called “Skin and Bone”.11 This app allows participants to see the skeletons side-by-side with a reconstructed version of themselves. This helps the participant see how the actual articulation of the skeleton creates the structure for these species. In addition, there have been several exhibitions attempted in science centers, however, because science centers tend to be hands-on, there can be concerns that the augmented reality systems takes away from the physical interaction element. As Steven Snyder and Karen Elinich wrote in 2010, “The gear, essentially, becomes the experience. And, the gear effectively forces the learner to learn alone – in contrast to the social learning goals associated with most science exhibits.”12 There is a point in which the digital augmented reality experience becomes the exhibit, not just a tool for it. When designing digitally integrated installations, a tool like augmented reality should be used to help facilitate the learning process across the exhibition, not become the main source of focus for the exhibition. Often, this can leave the exhibition feeling more like entertainment than it is educational.

Entertainment is not the antithesis of education, and exhibitions can strike a balance between the two. One exhibition in the National Museum of Singapore in conjunction with teamLab does this exponentially well. The ‘Story of the Forest’ is a digital art installation that features flora and fauna native to Singapore. A rotunda and hallway have been converted with massive digital monitors that encompass the space. As the viewer steps closer to the wall, different animals will come forward to interact. With the use of a smartphone app, participants use augmented reality to record the plants and animals they are interacting with on the monitors and to receive further information about that species.13 This exhibition is visually stunning while providing regional context with further information. This balance is done well, and it allows the participants to choose their own level of engagement without disrupting the whole.

How we can expand

As these digitally integrated exhibitions can be hard to plan and develop, one route for digitally integrating a museum might be to add context to a pre-existing exhibition. This usage can also be applied through an activist lens. For museums founded in colonial American history, this could mean breaking down the stories of “great men” and augmenting contextual information that helps depict their whole history, including practices like enslaving people. For sites of historical protest or of political unrest, augmented reality could be implemented to superimpose archival footage on top of these historical sites. This use might provide a deeper understanding for a wider variety of audiences, furthering the goal of making museums an inclusive and inviting place for everyone within our communities. The more we can include voices and perspectives typically left out of historical narratives, we will not only be engaging different, underrepresented parts of our communities, but we will allow those narratives to become authoritative actors in their own histories. Augmented reality should be a useful tool in this context, as it allows for multiple perspectives to be superimposed in one setting. As history continues to deviate from the traditional linear narrative, tools that allow multiple sets of information to be displayed will not only help with developing future exhibitions but also with revisiting current exhibits with a new lens. Without further research, the interplay between augmented reality and curatorial activism is relatively unknown but should be researched moving forward.

Of course, while augmented reality can be used to expand the context of new and pre-existing exhibits, it does require that participants opt-in to the experience. Without the information being readily available, participants may overlook the presence of a digital landscape within an exhibition, or they might not have access to the technology required to view the expand content. When designing augmented elements for an exhibition, the design team should be especially sensitive to the in-exhibition marketing done for the experience. With clear signage and instructions, along with several reminders throughout the exhibit, the audience has a higher chance of being able to engage with the digital landscape. In addition, when adopting a digital landscape for an exhibition, museums should provide access to the technology required for participants who would not have access otherwise. By providing access to tablets or charging stations, along with providing the wireless connection necessary to run these digital landscapes, we can provide greater access while continuing to further our dive into digital landscapes.

Ultimately, augmented reality can enhance the learning experience by providing the user with an experiential addition to the exhibition. However, when applied unsuccessfully, augmented reality can distract from the exhibition and leave the user feeling like they lost part of the experience. Because augmented reality is so context dependent, the question remains; when should augmented reality be applied? In general, augmented reality is most useful when applied to visual mediums, either through manipulation or reconstruction, and when additional information is being conveyed, in a non-disruptive way. Augmented reality can also be incredibly useful when working in unique or experimental conditions, i.e., public streets. When deciding whether to develop an augmented reality asset in an exhibition, the design team should consider the purpose of its usage: Does it add additional context? Does it support the message of your exhibition? Does it engage the audience in a meaningful way? If the use of augmented reality meets these criteria, it is likely that it will be received well, and will help to guide guests in a more meaningful way.

Notes

  1. Coates, C. 2020. “How Museums Are Using Augmented Reality.” Museumnext. https://www.museumnext.com/article/how-museums-are-using-augmented-reality/. ↩︎

  2. Bower, Matt, Cathie Howe, Nerida McCredie, Austin Robinson, and David Grover. 2014. “Augmented Reality in Education - Cases, Places and Potentials.” Educational Media International 51 (1): 1–15. https://doi.org/10.1080/09523987.2014.889400. ↩︎

  3. Morellato, Massimo. 2014. “Digital Competence in Tourism Education: Cooperative-Experiential Learning.” Journal of Teaching in Travel and Tourism 14 (2): 184–209. https://doi.org/10.1080/15313220.2014.907959. ↩︎

  4. Moorhouse, Natasha, M. Claudia tom Dieck, and Timothy Jung. 2019. “An Experiential View to Children Learning in Museums with Augmented Reality.” Museum Management and Curatorship 34 (4): 402–18. https://doi.org/10.1080/09647775.2019.1578991. ↩︎

  5. Sommerauer, Peter, and Oliver Müller. 2014. “Augmented Reality in Informal Learning Environments: A Field Experiment in a Mathematics Exhibition.” Computers and Education 79 (2014): 59–68. https://doi.org/10.1016/j.compedu.2014.07.013. ↩︎

  6. Ghouaiel, Nehla, Samir Garbaya, Jean-Marc Cieutat, and Jean-Pierre Jessel. 2017. “Mobile Augmented Reality in Museums : Towards Enhancing Visitor’s Learning Experience.” International Journal of Virtual Reality 17 (1): 21–31. https://doi.org/10.20870/ijvr.2017.17.1.2885. ↩︎

  7. National Portrait Gallery. 2018. “Art of London Augmented Gallery - National Portrait Gallery.” London.https://www.npg.org.uk/visit/art-of-london-augmented-gallery. ↩︎

  8. “ReBlink | Art Gallery of Ontario.” 2018. Art Gallery of Ontario. https://ago.ca/exhibitions/reblink. ↩︎

  9. Sapieha, Chad. 2017. “New ReBlink Exhibit at Toronto’s AGO Takes Augmented Reality beyond Video Games.” Financial Post. https://financialpost.com/technology/gaming/new-reblink-exhibit-at-the-ago-takes-augmented-reality-beyond-video-games. ↩︎

  10. Chang, Yu-Lien, Huei-Tse Hou, Chao-Yang Pan, Yao-Ting Sung, and Kuo-En Chang. 2015. “Apply an Augmented Reality in a Mobile Guidance to Increase Sense of Place for Heritage Places.” Journal of Educational Technology & Society Educational Technology & Society 18 (182): 166–78. http://www.jstor.org/stable/jeductechsoci.18.2.166. ↩︎

  11. “Smithsonian National Museum of Natural History.” 2015. https://naturalhistory.si.edu/exhibits/bone-hall. ↩︎

  12. Snyder, Steven L., and Karen J. Elinich. 2010. “Augmented Reality for Interpretive and Experiential Learning,” 87–92. https://doi.org/10.14236/ewic/eva2010.15. ↩︎

  13. teamLab. 2016. “Story of the Forest | TeamLab / チームラボ.” Seol. https://www.teamlab.art/jp/w/story-of-the-forest/. ↩︎