Right before my school was shut down two weeks ago I assigned my AP World History students to write the 2015 World History AP exam DBQ, which is about the flu pandemic of 1918-1919. (At the time the coronavirus was still just a major current events story, not a full blown global crisis.) As I’ve been working with them (virtually) on it, I’ve had more time than ever to reflect on the problematic nature of many College Board free response questions. Too many AP exam questions are unclear about what they want students to do, intellectually. The College Board knows they want students to demonstrate historical thinking, but like most history teachers (including Gary and me before we argued our way to the Four Question Method), they’re not clear in their own minds about precisely what that looks like. Because they lack good epistemological categories for their questions, the questions are often unclear. The result is that students have two intellectual tasks to complete if they are to score well: first they have to figure out what the question is asking them to do, then they have to succeed at doing it. I think that students, teachers, and exam graders would all be better off with clearer questions — then everyone involved could teach, practice, and assess historical thinking directly. Let me use the 2015 DBQ to show you what I mean.
“Analyze” Means “Identify”
The 2015 prompt tells students to use nine documents to “analyze responses to the spread of influenza in the early twentieth century.” What does “analyze” mean in this context? At our workshops we tell teachers that if you’re not sure what question you’re asking, take a look at an exemplar answer. If a good answer is a narrative, you’re asking a Question One; if a good answer is describing someone’s thinking, you’re asking a Question Two, and so on. Because the verb “analyze” itself doesn’t give us any guidance about what kind of historical thinking this prompt requires of students, we need to turn to the exemplars provided in the scoring guide to see what the question is really asking. The first exemplar earned a perfect score, in part because “The thesis of this essay is found in consecutive sentences in the introduction where the student identifies three responses [to the flu pandemic]” (emphasis added). This tells us that the College Board is asking a Question One. Students need to read all nine documents, then identify different responses to the flu pandemic that appear in the documents. This may seem prosaic, but in fact answering Question One well requires considerable skill at reading and categorizing, which the scorers acknowledge: this student gets points for understanding all nine documents, using evidence from all nine documents, and effectively grouping the documents into three categories of responses in three separate paragraphs identified with clear topic sentences. All of these points are awarded for a narrative intellectual task: reading the documents to tell us “What Happened?” during the influenza pandemic of 1918-1919.
“Analyze” Means “Interpret”
The top-scoring exemplar essay also gets points for doing more than identifying responses to the pandemic. The student earned points for noting the different points of view of the authors of two of the documents, suggesting that they might not be entirely accurate, and for noting that one document is a recollection of events from fifty years earlier. This is not terribly deep thinking, but it does go beyond the surface meaning of the documents and shades over into what we at 4QM call interpretation, or Question Two thinking. In this case, “analyze responses to the spread of influenza” means, “interpret some of these documents about responses to the spread of influenza.”
Re-Writing The Prompt
There is a missed opportunity here: if the prompt were re-written to clearly define the two types of historical thinking demonstrated in the exemplar, more test-takers would have the opportunity to demonstrate their skills as historians. I’d revise the 2015 DBQ to be a straight up Question One, Question Two essay, like this:
“Use the documents below to describe at least two different responses to the influenza pandemic of 1918-1919 [Q1]. For each response you describe, interpret at least two documents to show what the people who took those responses were thinking when they did so [Q2]. Essays that are well organized and well written will earn top marks.”
Gary likes to say that the verb “analyze” is usually a sign that the question writer doesn’t know what they mean to ask, and this prompt is a perfect example. In the context of the AP exam students faced with “analyze” need to first solve the mystery of what the question is asking before they can work on their answers. I’m in favor of taking the mystery out of exam questions. Too much College Board “rigor” is obtained by forcing students to guess at what will actually be graded. Let’s make the questions clear so that we can grade students’ history skills instead.
J.B.