Data Science Studies Meeting – Wednesday, January 23rd 2:30-3:30 pm @ Data Science Studio
Cooking Data with Care: The Role of Contextual Inquiry in Large-Scale Quantitative Research
For January’s meeting, we’ll be joined by R. Stuart Geiger, staff ethnographer at the Berkeley Institute for Data Science (BIDS), who will present a talk called, “Cooking Data with Care: The Role of Contextual Inquiry in Large-Scale Quantitative Research.” This event is being held in conjunction with the QUAL Initiative.
How do we bring human-centered perspectives and cultural contexts to data-intensive, highly-automated algorithmic ways of knowing? Qualitative and quantitative methods are often imagined as orthogonal ways of answering questions in fundamentally different, incommensurable ways. In this talk, I argue that there is often substantial qualitative contextual inquiry and expertise deployed in quantitative methods. Such insights are crucial to “cooking data with care,” as Geoff Bowker advocated. These situated knowledges are typically implicitly leveraged as analysts make key decisions about what data to use, when data needs to be calibrated, how to transform and merge data for a given purpose, how to reduce dimensionality to work with complex datasets, which variables to use as proxies, and how to interpret the validity of results. To illustrate, I share experiences with a recent publication in which we directly integrated ethnographic and qualitative methods to reproduce, extend, and ultimately contest the interpretations of a previous large-scale computational study that claimed to discover substantial levels of conflict between automated bots in Wikipedia. Our mixed-methods approach is an integrative, iterative synthesis, not a coordinated pluralism: there is a column in a dataframe that would not exist without ethnography, as well as thick descriptions of cases that were found by sampling from the extreme ends of a statistical distribution. Through my experiences in this project, I discuss several lessons learned and future directions, which touch on topics including open science, reproducibility, and data ethics.
Following Stuart Geiger’s Data Science Studies talk, Data Science Studies and the QUAL Initiative will be convening a small group for a workshop-style discussion exploring how to incorporate qualitative methods into data science curricula and how to advance mixed-method research collaborations in data science. This event is by invitation, but we have a couple slots still available, so email Anissa Tanweer at firstname.lastname@example.org if you’re interested in the topic.