I. Empathy + Technology: Digital Testimony in Holocaust Museums

In the 21st century museum, technology seems to reign supreme as a tool intended to enhance the visitors experience. Museums are looking to mobile applications, virtual reality, augmented reality, and countless other forms of technology to create a more immersive experience. Bruce Wyman attributes this to a “trend towards personal interactivity.” ¹ Visitors want to take a more active role in the learning environment. But often, it seems that technology is being used more as a shiny object to get people through the door, as much as a tool to assist in opportunities for learning. All museums must be careful to utilize new technology only when it enhances learning, not simply because it is new. Holocaust museums have to get over the additional hurtle of utilizing technology that elicits empathy in their visitors. The interactive oral history experience, Dimensions in Testimony, which offers visitors the opportunity to “ask” questions of a lifelike digital avatar of a Holocaust survivor, is an ideal case study to unpack the complexities of this issue.

Brief History of Oral History

Before exploring the specifics of this new format of testimony, it is helpful to give a brief history of oral history. The perception of oral history as a source in academia has greatly shifted over the years. The scholar Steven High has written extensively on the subject, revealing the criticism, skepticism, and overall hesitation by historians in seeing the value of oral histories as a legitimate source. Much of this uneasiness comes from the claim that unlike much historical documentation, oral history is not objective. It is susceptible to what High refers to as the “problem” of memory. ² Recall can -at any point- be skewed, but as time passes, events and experiences in life can affect how memories are recalled. Of course, the same argument could be made for other historical documents, which can often be just as subjective. After all, as Churchill famously said, history is written by the victors.

Oral history’s highly subjective nature may just be its greatest strength. It has the potential to reveal something much deeper, much more human than many other kinds of documentation. Yes, the passing of time allows memory to create errors, but it also allows the individual to create perspective on their lived experience. ³ Furthermore, seeing and hearing a person’s testimony- every hesitation, intonation, and body movement can tell us things the written word cannot.

High makes an important distinction between testimony and life story. Testimonies are eye-witness accounts of an event or moment in time, whereas life stories, in widening the lens out on a person’s whole life, “finds meaning in the context of a life lived.”⁴ What happened before and after the event is important in better understanding the event itself and how and why the person remembers it as they do. Oral histories related to the Holocaust are typically recorded in this life story format. Interviews with survivors, witnesses, and liberators do not begin in 1933 or end in 1945, but rather entail their whole life lived up until the moment of recording. So, by High’s definition, this technology should really be called ‘Dimensions in Life Stories.’ However, the ‘lingua franca’ in the world of Holocaust scholarship is to use testimony, so I will do so for the remainder of this piece.

One organization that has been a leader in recording and preserving oral histories on a monumental scale is the Shoah Foundation (SFI) at the University of Southern California. After directing Schindler’s List, Steven Spielberg felt the need to have a permanent record of the lives of survivors of the Holocaust. Since he founded the Shoah Foundation in 1994, the organization has amassed more than 55,000 video testimonies from 65 countries, recorded in 43 languages.⁵ Though initially founded to record survivors of the Holocaust, SFI has expanded their work and recorded thousands of testimonies of survivors of other genocides.

Dimensions in Testimony

In 2010, Heather Maio approached SFI with the desire to create an interactive exhibition where people could simply walk up and talk to a Holocaust survivor. Maio is the director of Conscience Display, a company that creates exhibitions around Holocaust survivors. SFI began a partnership with Maio and the USC Institute for Creative Technologies that led to the development of Dimensions in Testimony.

A tech team, content team, and exhibition team worked together to create an interactive exhibit that would make Holocaust testimony more accessible to museum visitors. DIT began as a research and development project, only recording its first testimony some four years after its founding. According to SFI, an advisory committee of three Holocaust survivors also played a role in the process. ⁶

The average length of a standard testimony in the SFI collection is 2.5 hours.⁷ The survivor usually sits for anywhere from 2-8 hours, typically in their home, and answers about 200 questions. DIT is a much more significant undertaking. To create what SFI Director Stephen Smith dubbed the “interactive video biography,” the team had to build a recording space comprised of 116 cameras and over 6,000 LED lights that envelope the survivor. An average DIT testimony takes 15 hours and the survivor is asked to answer between one and two thousand questions. ⁸

Though it is recorded with 360 camera technology, early formats of DIT were displayed on a standard 2-D screen. SFI says they used this technology so that these recordings could be displayed on other future technologies- including those not yet available. ⁹ Some of the latest versions of DIT are being displayed in a hologram-like format.

The final product has thus looked different in its many versions to date, but it essentially boils down to a video that responds to human voice commands. Rather than watching a traditional linear testimony from start to finish, the testimony is broken up into short clips that are unlocked by the user’s questions and active participation. A visitor can speak a question into a microphone placed across from the display, prompting a prerecorded response from the virtual survivor. Using a natural-language-processing software, visitors can ask questions in any way they would naturally speak, similar to Apple’s Siri or Amazon’s Alexa.¹⁰ The software is constantly learning and improving based on visitor language use. In beta testing at various Holocaust institutions around the U.S., DIT was accompanied by docents who went through three days of training to help facilitate user interaction.¹¹ Today, docents still assist visitors in using the technology to varying degrees.

Who Will Use This Tech?

As stated above, most standard-format testimonies are between two and eight hours long, so it would be rare to find one in its entirety in an exhibition. Curators and museum staff know that visitors will not sit (or more likely stand) for six hours straight to hear these stories. In the past, curators have created much shorter clips of testimony and brought them into the exhibitions in order for visitors to make contact with this source material.

The permanent exhibition at USHMM offers visitors two opportunities to hear testimony. First, audio of Auschwitz survivor testimony plays on a loop in a glass room adjacent to an exhibit on concentration camps. Second, at the end of the permanent exhibition, 79 minutes of video testimony excerpts from multiple survivors and liberators plays in a small theater. Narration between clips further contextualizes the stories being told.

In my two years working and giving tours at USHMM, the testimony theater was always the place visitors wanted to stop for the longest amount of time. Yes, it could have something to do with the fact that after walking through three floors of concrete exhibition space, the seat was a nice break, but I also believe that after all the photographs, documents, text panels, and artifacts, the human connection made the greatest impact.

Yet, visitor interaction with recorded testimony usually begins and ends here. Typically, researchers and scholars are the ones watching the full testimonies. They are often only accessible in a museum’s library or resource center, not the main floor or exhibition spaces. So, it seems that DIT has the potential to make testimony more accessible to the average museum visitor, both by being interactive and broken down into shorter, more accessible clips, visitors can hear part of a survivor’s testimony. It still will not allow the average visitor to experience a full testimony, as that would require the user to ask the digital survivor all 2,000 questions. However, in making it easier for the visitor to experience this testimony, DIT risks dramatically shifting agency away from the survivor, to the user.

Survivor Agency and Decontextualization

In 2017, SFI’s annual international conference was centered around digital approaches to genocide studies. One of the panelists, Dr. Noah Shenker of Monash University, expressed deep concerns about DIT as it relates to survivor agency. “The experience,” Shenker said, “now focused on the user – the agency of the survivor was moved to user-driven imperatives.”¹² He explained that the survivor no longer speaks to listeners from start to finish. Instead, the user must ask questions to trigger sporadic narratives. Often times, that user comes in with agency that is decontextualized.¹³ In a standard video testimony, a trained interviewer has researched the individual’s story and asks questions to help guide a conversation that contextualizes the survivor’s life. With DIT, users would not automatically come prepared with that context and may not know what questions to ask in order to learn that survivors’ story.

The Illinois Holocaust Museum and Education Center (IHMEC) is among the first institutions to make DIT a permanent feature of their exhibition, set in a theater-like space. In an attempt to provide context to their users, IHMEC begins the experience with a seven-minute documentary about the Holocaust survivor that will be featured. During Howard Reich’s visit for the Chicago Tribune, the virtual survivor presented was Fritzie Fritzshall. In this opening video Fritzshall speaks about her family’s arrest, the harsh conditions of the ghetto, the terrifying boxcar ride and the experiences of the concentration camps. At the end of the video, the house lights come up, and the hologram Holocaust survivor appears out of thin air with the opening line, “I have so much to tell you. So please, ask me questions.”¹⁴ Although the opening documentary may provide some context, interactions with DIT will vary immensely in a way that allows for the possibility of multiple narratives. While this has potential benefits for users, and aligns with movements in museum storytelling towards nonlinear and personalized experiences, it deprivileges the survivor’s capacity to define their own story.

Furthermore, as I watched videos of people using DIT for my research, something began to feel oddly familiar to me in a way I couldn’t quite place. Then, one visitor began each of her questions by stating the survivors name: “Pinchas, when were you born?” “Pinchas, do you remember liberation?” “Pinchas, do you have a family today?”¹⁵ It hit me, she was talking to this technology -this Holocaust survivor- like it was a home smartspeaker. A question is asked, there is a brief delay, then an answer is generated. In Wyman’s Digital Storytelling in Museums, he speaks about how, because the iPhone made touch-screen technology so commonplace, museums can assume visitors would walk up to a screen and automatically know to use two fingers to zoom in and out without being instructed.¹⁶ This may just be how visitors will approach DIT. With our daily use of Siri and Alexa, will it feel natural to verbally ask a piece of technology a question and expect to receive a human-like answer, even when that question is about ghettos and concentration camps?

At the same SFI conference, a scholar in attendance asked about the “vulnerability of this virtual agent.”¹⁷ Often times, people like to play with Siri or Alexa by asking them trick or even inappropriate questions. Like Siri and Alexa, the DIT survivor does not have the agency to defend themselves. Dir. Smith had two interesting responses to this criticism. First, he said that in all the testing at conferences, in museums, and at tech shows, the only people who have tried to fool the system are “historians and techies.”¹⁸ He went on to explain that in all their testing, they had not seen any misuse by teenagers. He attributed that respect to the setting of the museum and the structured environment the students were in. Second, Smith said he consulted with survivors extensively on the possibility of being asked inappropriate questions through the system. Survivors told him that they put themselves in front of the public eye every day in real life and run that same risk.¹⁹

Visitor Interaction

So, what has actual visitor interaction looked like? In a blogpost for SFI, former USHMM staff member Elissa Frankle reflected on the experience of running the team that beta tested DIT at the museum in 2016. Frankle wrote that going into it, the staff assumed all visitors would want to ask a question. In reality, visitors who sat in the back and listened told evaluators they felt they had just as powerful an experience as those actively participating.²⁰

This earlier version of DIT was displayed in 2-D on a flat screen. Frankle attributed visitors’ comfort with this technology to its similarities to Skype or FaceTime. “The idea of asking questions of a face displayed on a flat screen, and having them answered in real time, is pretty natural for a number of our visitors.”²¹ That comfort may have allowed the content to triumph over the content. But what of the new holograms being used today? Is that technology also able to fade into the background or is a distraction, a game, a form of entertainment and wonder?

In another article published by SFI in 2016, the institution touted the sophistication of the technology by pointing out how surprised one teenager was when the virtual survivor gave a proper greeting response after the teen “sarcastically asked ‘whazup?’”²² Herein lies part of my unease. Would that same teen have been so sarcastic and silly if she were face-to-face with a real Holocaust survivor? Would she ask ‘whazup’ and then follow up with ‘so, did any of your family survivor the Auschwitz?’ Once again, the virtual agent does not have the ability to defend itself.

There is a trend among museums towards both non-linear storytelling and visitor agency. I will be among the first to step up and advocate for giving the voice of the visitor more importance in the museum space. However, one key does not unlock every door. There will still be times where the visitor’s role must be more passive, as a listener or observer. For Holocaust testimony, survivor agency and the capacity to shape their own narrative is more important that the agency of the user. Dimensions in Testimony fit both these trends, but they do so at the expense of survivor agency that should remain unaltered.

The ways in which we communicate with one another is changing because of technology. When not face-to-face, some barriers are dropped and people feel more comfortable saying things they may not necessarily say in person. Whether over text or on Twitter, not speaking to the person directly seems to change how people interact. As we saw with this teenager, this seems to be the case with our ‘virtual survivors’ as well. Is it better to just listen to a standard oral history recording rather than allow for the potential of people interacting with one in an inappropriate matter? Does it diminish the horror survivors went through when people can say whatever they want to this virtual survivor without consequence?

While technology breaks down some barriers in communication, it also allows us to build up others. Steven Cohen notes how as humans, we are psychologically equipped to protect ourselves.²³ Whether watching a full oral history or just clips put together by curators for an exhibition, if visitors choose to watch these films, they must face difficult subjects. In theory, a visitor could ask dozens of questions to the ‘virtual survivor’ and not once broach the topic of the trauma they experienced. In her work From Empathetic Understanding to Engaged Witnessing: Encountering Trauma in the Holocaust Classroom Liora Gubkin stresses that it is important for those choosing to learn about the Holocaust to recognize that “pain is a critical element of the other’s experience,” the ‘other’ in this case being survivors.²⁴ This technology can allow visitors to skirt around the most difficult subjects, which silences the pain these survivors experienced and defeats the whole purpose of testimony.

There’s a diagnosis for everything these days, isn’t there? Don’t believe me? Well I can even put a name to my uneasiness interacting with DIT. In 1970, Japanese roboticist Masahiro Mori coined the term uncanny valley, referring to a mental uneasiness triggered by a robot or virtual character with human characteristics. “A viewer’s familiarity drops sharply into the uncanny valley once the artificial figure tries but fails to mimic a realistic human.”²⁵ Karl MacDorman, a robotics researcher at Indiana University is conducting tests to understand how the uncanny valley influences emotional empathy during an interaction with a virtual character. Volunteers in the test will talk with either real actors or their digital doubles. MacDorman predicts that the uncanniness will “interfere with participants’ normal empathetic response within this scenario.”²⁶ The technology is DIT is not yet flawless, so its glitches may create a similar interference in creating empathy.

Museum Embrace

Despite all the potential issues I have laid before you, museums still seem to be embracing DIT. As newer, more interactive technology continues to be the hot trend, museums need to figure out how technology enhances the learning experience. For Holocaust museums, they – in theory – have the added task of figuring out the relationship between empathy and technology. It is clear that the intended use of DIT is for the museum space. So, it would seemingly be no surprise to see an overwhelming embrace of this technology in Holocaust Museums around the country. But I am skeptical about the motivation of museums in their adoption of this shiny new object.

After years of beta testing, DIT now exists in a more permanent form, debuting permanently in two major holocaust institutions. The Illinois Holocaust Museum & Education Center (IHMEC) became the first institution to permanently exhibit these survivor testimonies, followed soon after by the newly renovated Dallas Holocaust Museum & Human Rights Center.²⁷ Both institutions have gone all in, investing heavily by building special state-of-the-art theaters in order to display the experience in a more three-dimensional form.²⁸ How heavily? According to the Dallas Morning News, the museum put down a cool $2.5 million to build their theater for DIT.²⁹

In order to see the success of their investment in this new educational tool, these museums are trying their best to spread the word to potential future users. On a recent trip to Chicago, I found myself in the back of an Uber, stuck in traffic on the highway. As we crept along, we came upon a massive three-story high billboard that read, “What can a hologram tell you about the Holocaust? Experience the first interactive 3D exhibit of its kind” with the words “World Premiere” slapped on the upper right corner. Similarly, on their website, the Dallas museum tries to attract visitors this way, “It’s real time. It’s groundbreaking. Interact with virtual Survivors in a specially designed space, where high-definition holographic interview recordings paired with voice recognition technology enable these incredible people to respond to questions from the audience, inviting one-on-one ‘conversation.’”³⁰ Both museums use language to sell the tech, not the important or meaningful lessons that visitors can take away as a result of its use. Is DIT a technology that enhances the learning experience or is it merely a way of keeping Holocaust museums competitive, trendy, and relevant in the museum world?

Timing: Record Now, Display Later

From the technological perspective, DIT is coming into the museum world at just the right time. Wyman notes that the time of the voice of authority speaking to the public is dwindling away. Instead, he writes, there is a desire for a “multi-faceted experience that invites conversation and interaction with visitors.”³¹ Visitors want to have their voice be a part of the museum experience, and Dimensions in Testimony allows them to do so. But from a content perspective, I do not believe now is the right time for this technology to be exhibited.

One of the greatest privileges of my time working at USHMM was being able to work alongside the survivor volunteers, Holocaust survivors who come to the museum each and every day and give of their time, speaking to visitors, translating archival documents, and sharing their stories. The further we get from the events of the Holocaust, the fewer survivors there are left to speak first-hand about their experience. That was a major motivation behind the DIT project. But just because they have been recorded, does not mean they have to be displayed right away.

There are living Holocaust survivors coming into the museum every day to tell their story face-to-face with visitors, and while we still have the immense privilege to hear from them firsthand, we must not sideline them for some shiny object. Think about it, if a tech-obsessed middle schooler had a choice between sitting for an hour listening to a speaker or spending a few minutes with flashy new tech, which do you think they would choose?

New technology has the potential to greatly enhance the ways museums teach, reach, and engage with their visitors. But museums run the risk of falling into the trap many in society have with wanting the newest and sleekest tech. Technology for technology’s sake rarely succeeds in creating a meaningful, lasting learning environment for the visitor. Dimensions in Testimony does have the potential to create this enhanced learning experience. Its development may be a result of the shift in visitor desire for greater interactivity; or the ‘chunking down’ of testimony could simply be what Martin Bazley and Helen Graham diagnosed as a “symptom of an attention deficit society.³² If nothing else, DIT can be a cautionary tale to Holocaust museums as they continue to invest time, money, and resources into innovation.

An Alternate Path

Before I finish, I’d like to tell you about a different kind of experience that was similarly non-linear and personalized for users, but involved a much more shared sense of agency and contextualization. In 2017, USHMM brought in The Portal, a traveling digital experience. This large, repurposed shipping container housed a bench, screen, and projector. Between 10:00am and 2:00pm each day, visitors could video conference with Syrians in refugee camps across Europe.³³ This exhibit was at the museum during the height of the ‘migrant crisis’ in Europe, when images of children’s lifeless bodies washed up on the shores of Greece flashed across the screen of most news networks. Its placement in the museum could not have come at a more relevant time.

My first experience with The Portal occurred during a donor tour I was leading. My visitors sat at the bench, I behind them. We listened as three young Syrian men introduced themselves in broken English, then slowly and naturally an incredibly moving conversation unfolded. At one point, one of the visitors in D.C. asked whether they wanted to make a home in Berlin (where they were currently) or wanted to continue to America. “No sir,” one of them said, “Damascus is our home. We just want to go back there, back to our normal lives.” The visitor broke down, and told the Syrian men that after surviving the Holocaust, all his mother wanted to do was move back to Germany, her homeland.

This experience has stayed with me. Suddenly a face, a name, and a connection to the crisis was made. This is an exceptional example of a way to achieve empathy through technology. It was never about the technology itself, but the empathetic understanding Gubkin speaks of, the way the technology allowed us to sympathize with the other in real time.³⁴ Steven Cohen believes testimony has the potential to make the greatest impact when it is “directly relevant to key dimensions of [our] own lives.”³⁵ For the visitor that day, the testimony of these Syrian refugees could not have been more relevant to his personal family experience. The technology faded into the background and succeeded to make a lasting impact.

Notes