What are our class questions?

• How do discrimination and injustice manifest in computing systems?

• Where does this come from?

• How can we measure it?

• What can we do about it?

• What does it imply for the fundamental ways we think about computing?

  We’re going to go beyond computing literature, and engage with readings that perhaps even question the entire enterprise.

How are we going to answer them?

• Read and discuss papers

  The biggest, most constant activity throughout the semester will be to read and engage with research papers.

  I’ve already posted the reading list for the first 6 weeks, so you can get started right away.

  • Alternate formats: sometimes the ‘paper’ will be a talk
  • Week 3: watch FAT* livestreams (and work on W4 reading)
  • Seminar talk

• Practice

  There are two small assignments to give you hands-on experience. I encourage you to try to reproduce portions of other papers that we read throughout the semester.

• Do research

  A major component of this class is an original research project. I strongly recommend that you select a research project that complements your primary research, although it should not be the same.

Let’s talk about reading

• Reading takes skill and practice

• Naive reading is likely to be ineffective

  Don’t just sit down with the paper and read it front to back, making sure you understand each thing before moving forward.

• Read with purpose - one order:

  • Abstract
  • Introduction
  • Section headings and conclusion
  • Go back and fill in details

• Focus on answering questions about the paper

  • What are its claimed contributions?
  • What do we know after the research that we did not know before?
  • What is the normative background?
  • What is the key insight or principle of its methods or constructs?

    • What do metrics mean?

      What does it mean — in human terms — when this metric says something is fair or unfair?

  • What question does it seek to answer?
  • What are the implications of answering this question?
  • How does it answer that question?
  • What are the implications of its answer?
  • Is it credible?
  • What limitations are there to the answer?
    • Generalizability (applicability to other topics / problems / data sets)
    • Methodological weaknesses
    • Data weaknesses
  • What might we do next?

Distinctions

Note: a paper may not be clear about these!

• What are the claims?
• What is the data?
• What is the argument?
• What are the assumptions?
  • Some will be stated
  • Some will not!
• What is the normative grounding?
  • What harms are trying to be measured or prevented?
  • What is the ethical basis for identifying & assessing harms?
• Right, wrong, and contestable answers
  • There is a lot of contestible space in this topic without ‘right’ or ‘wrong’ answers
    • What ethical framework(s) should we use?
    • How do we define and assess harm?
    • How do we balance different tradeoffs?
  • There are ‘right’ and ‘wrong’ answers
    • The paper did something
    • The paper claims something
    • Under this set of definitions and metrics, the data show X
    • This is a disparate impact, or disparate treatment
    • The paper still may be right, wrong, or neither in its claims, but the fact of what it claims we can know
  • Contestability and social construction do not mean anything goes
• Disagreement
  • We will read papers you disagree with
  • We will read papers I disagree with