XII. Examining Biases in GLAM Data and Metadata

I owe an incredible debt of gratitude to Patrick Bond for assisting me with the technical aspects of this paper. Additionally, I believe it is important to situate my own perspective and background as a cis-het white woman from a fair amount of privilege as it is only by acknowledging our own biases that we can begin to look beyond them.

Within the world of Galleries, Libraries, Archives, and Museums there are seemingly endless individual objects and each of these objects need to be classified, categorized, and tagged. Constructing these classifications, categories, and tags creates order out of what would otherwise be chaos, gives objects physical and temporal context, and allows for relationships between objects to be seen. However, creating the standards that govern these classifications, categories, and tags is highly subjective and can reflect the biases of the people or culture that is creating them.¹ Many GLAM institutions have made their collections and the data associated with their collections open access so it is possible for external personnel to evaluate what biases may exist within the metadata themselves and also what biases may exist in individual institutions and across the GLAM community by looking at data across institutions. The purpose of this paper is to examine how bias can be revealed in GLAM data and metadata.

The inspiration for this paper came from a distillation of many sources with the following four forming the primary impetus; First being the writings of Michell Caswell² and her presentation at the 2019 iPRES³conference in Amsterdam, in which she discusses using feminist standpoint epistemologies⁴ to evaluate archival appraisal along with the importance of acknowledging internal bias. The second being the work done to decolonize collections data for indigenous collections.⁵ The third source is the work of Fiona Cameron, specifically her paper with Helena Robinson on “Digital Knowledgescapes”.⁶ The final piece solidifying this subject came from one of the proposed topics listed for the upcoming 2020 Gender and Sexuality in Information Studies Colloquium⁷: “How do search algorithms, metadata standards, and user interfaces challenge or reinforce white supremacy, heteronormative patriarchy, and ableism?” To fully answer that question would be beyond the scope of this format; thus, it was decided to focus on language usage in metadata to breach the surface of the topic.

What is Metadata?

Prior to exploring the impact of metadata, the first step is to lay out a general definition for the subject, and then identify the types of metadata that are typically used in GLAM institutions. The concept of Metadata was not created with the advent of the digital age. The purest definition of the word is “data about data” and at essence, is used to ‘tag’ one piece of information (or object, or person) with identifying information. The usage of metadata dates to the library of Alexandria (ca. 3rd century BCE- 48 CE), where the librarians would physically place tags on the scrolls with author, title, and subject.⁸ That format developed into the card catalogue. Names, dates, and locations written on the back of photo prints constitute metadata. With the rise of computers, standards of digital metadata began to take shape.⁹ In the age of the internet, metadata are produced by most every action on the internet automatically. For instance, every email sent contains a variety of data that are not seen in standard user interfaces and are added at different points along its journey to the recipient.¹⁰ Therefore, it can be said that digital metadata can be basically broken down into two types-those which are algorithmically (human by proxy) created, and those which are created by a user (human).

Museum metadata are almost entirely created by human activity, though there are current projects that are working towards an automated process.¹¹ There are also an array of metadata standards and of controlled vocabularies used in GLAM institutions some of which are specialized for certain collections.¹² However, within museums, metadata can be broadly divided up into five basic groups:

Table 1. Adapted from “Introduction to Metadata” Edited by Murtha Baca¹³

How metadata can contain and show bias

The most easily seen way that metadata can perpetuate bias is through a use of language (controlled vocabulary) that is biased, especially in subject headings and tagging. One example of this is a project at the University of Alberta whose purpose is “ To investigate, define and propose a plan of action for how the University of Alberta Libraries can more accurately, appropriately, and respectfully represent Indigenous peoples and contexts through our descriptive metadata practices.”¹⁴ A project that they are using to look at their own metadata is the Manitoba Archival Information Network’s (MAIN) project to change Library of Congress Subject Headings (LCSH) in cases where the current subject headings (tags) are insensitive to the Indigenous Peoples in Manitoba and add additional subject headings that are more specific and reflective of the Indigenous culture.¹⁵ The vast majority of the changes consisted of taking every instance of ‘Indian’ and replacing it with “Indigenous”:

Screen shot of the changes tab in the MAIN spreadsheet.

fullscreen

The subject heading issues surrounding Indigenous collections are fairly blatant however, there are other situations where the issue is not as cut and dry as changing the language but instead about how to give the desired weight to different objects that have the same tag. One such circumstance is another project out of the University of Manitoba, The Digital Archives and Marginalized Communities Project (DAMC)¹⁶ which is a project that is developing three different but interrelated archives. One of these archives in development is the Sex Work Database (SWD). In their paper “Tagging for activist ends and strategic ephemerality: creating the Sex Work Database as an activist digital archive” Shawna Ferris and Danielle Allard discuss the community-produced controlled vocabulary and how they “are developing a tagging system that is designed to draw records into conversation with one another in ways that privilege sex worker activist messaging”.¹⁷ The example they discuss is for the tag “prostitute” which when searched or clicked on would bring results not only from the mainstream media, which tends to dehumanize sex workers¹⁸, but also give equal or higher weight to works from the point of view of sex workers.¹⁹

What all of these projects call into focus is the need for community involvement when creating controlled vocabularies especially when creating a new collection or housing a collection that was obtained unethically in the past. I believe that especially now that collections data are becoming more open access with increased public interaction with those data, GLAM institutions must shed the concept of being the ‘ultimate authority’ and understand that objects are polysemic.²⁰

Examination of Digital GLAM data

Metadata can also show bias at an individual institutional or even at the broader GLAM community level when looked at in the aggregate and as part of my research for this paper I wanted to do a comparative study of descriptive data across several museums to see if any bias was detectable. However, due to time constraints and the steep learning curve a project such as this would require it was not feasible. Instead, I performed a study of the Tate object file to look for the trends in the acquisition of art by women. Prior to getting to that result I am going to introduce and describe a few of the other data sets I looked at and their file formats.

There are two excellent sources I am aware of for locating open access data from museums; one is the “Survey of GLAM open access policy” Google spreadsheet by Douglas McCarthy and Dr. Andrea Wallace CC BY 4.0, 2018-present²¹ and another is the Museum APIs page on the Museums and the machine-processable world wiki maintained by Mia Ridge.²² The data I looked at primarily consisted of two formats; CSV (comma separated values) and JSON (JAVA Script Object Notation). While CSV files can be used with several applications, they are primarily used with spreadsheets. When imported into an application like Microsoft Excel, they typically result in simple spreadsheets. In the case of GLAM data, these files are normally bounded to one set of subjects such as artists, artworks, authors, documents, etc…The columns are the metadata shared by all the objects.

An example would be the SherlockNet data set developed by Luda Zhao, Brian Do, and Karen Wang for their project for the British Library Labs Competition: “Using Convolutional Neural Networks to Explore Over 400 Years of Book Illustrations”.²³ In the most simple of terms they trained a machine in the tags they wanted it to use and then had it tag and caption the 1 million images in the British Library 1M Collection which consists of scans of book illustrations from 1500 to 1900. The CSV file (sherlocknet_tags_verbose) that came out of this project has the following columns of data: image_idx; flickr_id; tag; size; scannnumber; imageorder; title; author; vol; pubplace; sysnum; date; filename. An example of one of the image’s data set in an inverted format for ease of reading is:

Table 2. Example of the data from the SherlockNet CSV file

For this format to be efficient the metadata tends to be limited to attributes that all the objects in the data file share. Metadata elements which only a belong to a subset of the objects under consideration result in lots of empty cells and cumbersome spreadsheets. Consequently, the simple structure can be limiting. However, spreadsheets tend to be well understood and for most non-technical people, these data are easy to work with. In the case of this object all of the tags generated by sherlocknet can be seen on its flicker page.²⁴

JSON files have a richer and more complex structure. The objects are described in JAVA Script which allows each object to have attributes (metadata) not shared by similar objects while avoiding the inefficiencies of the spreadsheet’s many blank cells. However, while this format is human readable, the entries tend to be very lengthy and are only easily interpreted through the use of a customized application. For example, consider the spreadsheet style data (CSV) for a given artwork. For this example, I am using the Tate object file as it was the one I spent the most time with.

Table 3. Example of the data from the Tate CSV object file

Now consider the corresponding JSON entry:

{

"acno": "A00003",
"acquisitionYear": 1922,
"all_artists": "Robert Blake",
"catalogueGroup": {},
"classification": "on paper, unique",
"contributorCount": 1,
"contributors": [
 {
  "birthYear": 1762,
  "date": "1762\u20131787",
  "displayOrder": 1,
  "fc": "Robert Blake",
  "gender": "Male",
  "id": 38,
  "mda": "Blake, Robert",
  "role": "artist",
  "startLetter": "B"
 }
],
 "creditLine": "Presented by Mrs John Richmond    1922",
 "dateRange": {
 "endYear": 1785,
 "startYear": 1785,
 "text": "?c.1785"
},
 "dateText": "?c.1785",
 "depth": "",
 "dimensions": "support: 343 x 467 mm",
 "foreignTitle": null,
 "groupTitle": null,
 "height": "467",
 "id": 1037,
 "inscription": null,
 "medium": "Graphite on paper. Verso: graphite on paper",
 "movementCount": 0,
 "subjectCount": 5,
 "subjects": {
 "children": [
  {
   "children": [
    {
     "children": [
      {
       "id": 1050,
       "name": "arm/arms raised"
      },
     {
      "id": 270,
      "name": "standing"
     }
    ],
    "id": 92,
    "name": "actions: postures and motions"
   },
   {
    "children": [
     {
      "id": 799,
      "name": "group"
     }
    ],
    "id": 97,
    "name": "groups"
    },
    {
     "children": [
      {
       "id": 195,
       "name": "man"
      }
     ],
     "id": 95,
     "name": "adults"
     }
    ],
    "id": 91,
    "name": "people"
   },
   {
    "children": [
     {
      "children": [
       {
        "id": 12297,
        "name": "preaching"
       }
      ],
      "id": 120,
      "name": "religious"
     }
    ],
    "id": 116,
    "name": "work and occupations"
   }
  ],
  "id": 1,
  "name": "subject"
 },
 "thumbnailCopyright": null,
 "thumbnailUrl": "http://www.tate.org.uk/art/images/work/A/A00/A00003_8.jpg",
 "title": "The Preaching of Warning. Verso: An Old Man Enthroned between Two Groups of Figures, by ?William Blake",
 "title": "The Preaching of Warning. Verso: An Old Man Enthroned Between Two Groups of Figures, by ?William Blake",
 "units": "mm",
 "url": "http://www.tate.org.uk/art/artworks/blake-the-preaching-of-warning-verso-an-old-man-enthroned-between-two-groups-of-figures-by-a00003",
 "width": "343"
}

As you can see, this is far more difficult to read and not as easily understood. However, while it contains the same metadata elements that appear in the columns of the CSV spreadsheets, it also contains additional “descriptive” elements. In this case there are also fields that describe actions associated with the artwork with values such as: “actions: postures and motions”; “hand/hands raised”; “standing”; “looking / watching”.

Maintenance of JSON files can be incredibly time consuming. For example, the Tate JSON files had 64,819 changed files with 253,856 additions and 54,402 deletions.

Research Project based in the Tate’s data

As I stated above, rather than do a comparative study of metadata over several institutions I made an examination of the acquisition trends at the Tate in terms of Gender. This required the combining of the two CSV data sets available from the Tate on GitHub.²⁵ The object file has the columns as described in Table 3 and thus does not have a column for Artist gender so I inserted a column for gender and used the Tate’s Artist file to assign gender to the artists in the object file. I did not attempt to confirm any of the data nor did I attempt to clean any of it, thus any artist that did not have a gender assigned in the Tate’s file did not have one assigned in the object file. I did find it interesting to note that while all the artists in the object file were present in the artist file a very few of the artists listed in the artist file were not in the object file. All this does is remind one of how messy data can be. It should also be noted that none of the files in the Tate GitHub have been updated since October of 2014.

Some basic statistical notes on the Tate collection according to this file are; the total number of artworks listed is 69,201; that only 1.1% of the collection does not have an artist with a gender assigned (very few of these were individuals and most were a variation on “British School”); 95% of the collection is by male artists; and only 3.9% of the collection is by women artists. However, I am not the first person to examine this object file and it was noted by Florian Kräutli in his Doctoral thesis “Exploring Digital Collections Through Timeline Visualizations”²⁶ that art by Joseph Mallord William Turner constitutes 57% (56.9%) of the works at the Tate.²⁷ Removing his data leaves the collection at 88.5% male, 9.5% female, and 2% without assigned gender. While either way these data are examined shows that works by female artists do not make up even 10% of the collection, what I thought would be more interesting consider is if there had been an increase in the acquisitions of art by women. I hypothesized that though the data here show a bias at the collection level toward males, presumably white males, there would be an observable historical trend toward increasing the number of women artists. Once I ran the numbers, I was able to confirm this hypothesis as you can see in Graph 1 below.

Percentage of Acquisitions by the Tate of art by female artists for 25 year periods

fullscreen

Table 4. Data in Graph 1

According to the data in the object file, the year of the Tate’s first acquisition was 1823. The first object created by a female artist in the file is listed as being acquisitioned in 1868³⁰, and the second not until 1890. However, the data and graphic above does show a marked increase in the percentage of objects created by female artists especially in the last 25 years. From the original acquisitions to the last 25 year period, the percentage of objects created by female artist increase from 0.000% to 13.732%.³¹

Conclusions

The finding from the Tate is not at all surprising. In April 2019 a study was released based on the collections data of 18 major art museums in the United States. It found that over all of those institutions “85% of the artists are white and 87% of them are men”.³² This study is part of the larger recent discussion surrounding gender bias in art institutions³³; The Baltimore Museum of Art recently announced that it will only buy art from female artists for their permanent collection in 2020³⁴; and there is software in development by Mujeres en las Artes Visuales, a Spanish Advocacy group, that will allow Museums to examine and quantify gender and racial bias within their collections.³⁵ However, all of these efforts are looking at the world through the lens of heteronormativity. If the Tate’s nomenclature only includes the binary male/female or else leaving the field blank, that nomenclature is biased, as is any institution that includes gender as a metadata field and does not include an expanded vocabulary.³⁶

Looking at heteronormative bias brings up a fairly large set of issues associated with finding bias in collections data. The data that would allow for determining bias against artists who are non-binary, or are not heterosexual, or even are ethically non-monogamous (which is also part of heteronormative patriarchal bias)³⁷ is not data that was recorded in artist files in the past and is not always information that people are willing to share with a broad audience. So, then the first question is: should GLAM institutions collect this information from living creators and make it part of the data record? And if they do, how should that data be stored? Should that data be open access, or should it only be for institutional use and shared as statistical information? And really, if the bias doesn’t already show in the data, should the data be made to show it? As Frances Lloyd-Baynes says in her article “Documenting Diversity: How should museums identify art and artists?”:

“Isn’t pinning down someone’s gender/ race/ ethnicity/ sexual orientation playing to the same old biases we’re trying to divest from our practice? Yes, quite possibly. But as we collectively make the transition from unconsciously biased to an open, non-racist/-sexist/-genderist society, it does help us identify the problem and work to balance the scales.” ³⁸

Notes