Collaborative AI Paper & Audit

Assignment Overview


Many current discussions of AI writing tools critique the ways language models—systems designed to predict which words are typically used with which other words—are being marketed and used as knowledge or information-retrieval systems—i.e. systems we can use to find and convey reliable facts. We even use an evocative, metaphorical name for AI models’ inventions: hallucination. What makes such hallucinations most concerning is that they are not always obviously or patently false, but instead blend snippets of truth, snippets of half truth, and outright falsehood, often employing generic features that make the mix seem reliable. What makes this difficult to really interrogate is that many of these systems are corporate “black boxes” where the details of their algorithms and training data are deliberately obscured.

In many cases, it is neither the technology or data itself that is the “black box,” so much as it is the corporate structures that prevent scholars or users from interrogating the technologies or their underlying data. It is difficult to write a media archeology of an AI like ChatGPT without direct access to its training data or systems. But even the most closely-guarded systems can be studied. Increasingly researchers interested in algorithmic justice have turned to approaches such as “algorithmic auditing”, “reverse engineering”, or “probing” to interrogate closed systems, seeking to understand a program and its training data through iterative uses or inquiries that test its boundaries and assumptions. On-the-ground investigation also helps illuminate systems, such as Time Magazine’s expose in January 2023 exposing OpenAI’s use of low-wage Kenyan workers to help filter toxic content from ChatGPT.

To explore such nuances, this assignment will unfold in two parts:

1. Generate an AI Essay

In the first step of this assignment, you will impersonate a high school student trying to use AI to cheat on their five-paragraph essay assignment. You will first choose one of the literary-historical topics we have studied in our class that you are interested in exploring more deeply. This could be a literary work we read together—a short story or our novella—or a historical topic we discussed—e.g. Luddism, automata. You will then use an AI writing tool such as ChatGPT to generate a critical essay on some facet of your topic. We will brainstorm ideas in class, but think of the ways you narrow down an essay topic in other classes by focusing on particular theme, motif, or critical framework for evaluation. I would not expect you to generate a perfect essay in a single pass—this assignment will likely require you to iterate and experiment with prompts until you create something that reads like a stereotypical five-paragraph essay. Here are a few key guidelines to keep in mind:

  1. First, your essay should center a claim of some kind—you should guide the AI away from simply reporting encyclopedic facts and towards writing an essay that is more interpretive.
  2. Second, your essay must include citations. I am not saying these must be viable citations (see part #2 of the assignment), but you should prompt the AI to quote and cite sources.
  3. As much as possible, you want your essay to look and read like a plausible essay. Our goal is not to generate obvious hallucinations, but essays that would seem passable to a general reader.

2. Critical Annotation

We will generate a rubric for stage two together, but in brief, the second stage of this assignment will ask you to undertake a critical evaluation and annotation of your AI-generated essay. We will evaluate our essays across several metrics, including the quality of the writing, the rhetorical structure of the argument, and the reliability of the facts it cites. While the precise details of our rubric are not yet determined, we will work towards continuous scales rather than binary evaluation. In other words, one particular citation might be 100% accurate and verifiable, while another might be in line with a critic’s work but not a literal quote—i.e. “somewhat reliable”—while yet another might be an outright fabrication—e.g. this cited author does not actually exist. The goal of our annotation is to tease out the nuances of mapping a language model onto an information retrieval task: in what ways does it perform well and where precisely are the points of failure?

Note that your annotation will almost certainly require research, particularly if you are uncertain about the veracity of the model’s claims or evidence.

3. Reflection

Finally, you will draw on your prompt engineering and annotation to write a critical reflection of approximately 1500 words that use your experiences to theorize about the language model itself, and its applications to scholarly or critical writing. Essentially, you are using the iteration and probing of this assignment to interrogate the “black box” system. This assignment asked you to begin probing the boundaries and assumptions of language models through comparison of different inputs and outputs.

Concretely, your reflection might explore:

© 2023 Ryan Cordell   •  Powered by Soopr   •  Theme  Moonwalk