“History is not my thing … I am going to be honest, I still don't like history. You can't make me take it again.” This assertion is from a student who just completed an introductory U.S. history course at a large public university in California, thereby clearing a significant early hurdle to earning a bachelor's degree. From all appearances, the student checked off a burdensome graduation requirement with little intellectual gain to show for the effort. We might be tempted to decry the course, the instructor, or some broader failure in higher education.
But let's keep listening to our supposedly history-averse student. Speaking in an anonymous focus group one month after final exams, the student described habits of thought typical of expert historians. In her words, she learned to appreciate “primary sources that may disagree with each other, but they are talking about the same exact thing.” She acknowledged learning to build arguments by comparing multiple types of sources: “I feel like that secondary source comes in, talks about [the primary source], and it more clearly states whether or not my original assumption was correct.” Our dyspeptic student also described moments of historical empathy and how important they were to her. Recalling a class activity on 1930s Mexican American deportations, she commented, “Like a certain number of people were sent—yeah, I understand that number, but now it's like I am hearing stories of people going over there, what their experience was like, and that hits me.” Not least, she developed a sense of why our subject matters, arguing that “American history is important. It is our country. It is the one we live in. We have to understand the history of it in order to become functional society members of it.” All this from a student who claims a frank antipathy toward history.
Aside from the jabs to our disciplinary pride, everything about this student's reflections indicates a successful learning experience. Her strategy for checking an interpretation's validity and her sense of civic connection through historical insight certainly meet important “learning outcomes” for an introductory U.S. history course. Her awareness of the past's importance for the present also reflects one of our deepest professional purposes in teaching history, especially to nonmajors.
For most who hold power within higher education today, none of this history-specific learning matters. As many public universities currently define the term, student success has no connection to actual student learning. In the California State University (Csu) system, where we teach, and across the country, administrators define success in readily quantifiable “data-driven” terms: faster time to graduation; higher retention rates; and reduced DFW grades, an acronym that combines D and F grades with students who withdraw (earning a W) from a course. At its most compelling and urgent, this administrative vision of student success seeks to eliminate equity gaps—that is, unequal educational experiences and outcomes appearing to correlate with students' race or ethnicity, Pell Grant eligibility status, or parental level of education. Yet even these laudable goals are worryingly unmoored from what our students actually learn.1
In the Csu system of twenty-three campuses enrolling nearly half a million students and awarding more than one hundred thousand degrees per year, large-scale “success” initiatives now command growing institutional attention, while the university's traditional educational mission recedes from view. In 2016 the Csu Chancellor's Office announced Graduation Initiative 2025, an ambitious program calling for “actively leveraging data” to increase graduation rates and eliminate opportunity and achievement gaps. For this campaign, the university generated a vast online data warehouse, known as the Csu Student Success Dashboard. With a few clicks faculty, administrators, and trustees can dive into the dashboard to view student grades and aggregated demographic data for every course on every campus going back nearly a decade. Under headings such as “In Which Courses Do They Struggle?” and “Are There Equity Gaps?,” the dashboard quickly generates colorful charts comparing course-specific grade point averages (Gpas) and DFW grade rates for an array of student backgrounds and statuses. Such dashboards, along with “predictive analytics” of seemingly infinite variety, are now pervasive features of academic life at public universities.2
The rise of big data relegates educational goals to a minor nuisance. Rather than encouraging professors to examine our bailiwick—what and how students learn—the Csu Student Success Dashboard invites department chairs and faculty to dwell simplistically on our year-to-year DFW grade percentages plotted along an x-y axis. We become brokers in a scholarly stock exchange. Are we trending up or down? Will another department's rival course outperform ours? Will campus advising centers use dashboard DFW grade data to steer students toward departments with higher pass rates? More fundamentally, faculty in the Csu system worry that actual learning has been “suppressed” by a focus on what can be easily measured in the age of student success. The Csu Academic Senate, a faculty-led body with limited powers, described in 2018 what big data fails to capture: “When, for instance, a student has struggled with a threshold concept in a discipline or with a foundational academic skill, and then ‘gets it,’ has a moment, clears a hurdle in their intellectual lives, that real student success does not get recorded.” As this report urges, “the challenge for faculty is to begin to make visible this deeper vision of student success to those outside our classrooms, and to encourage the institution to understand its vital place in the university experience.”3
Despite our keen sense of big data's shortcomings, student success dashboards should demand our attention. They reveal long-standing patterns of educational inequity in introductory U.S. history courses. When over 25 percent of U.S. college students receive grades of D, F, or W in introductory U.S. history, we should feel the weight of this social cost. When we learn that these rates are higher for historically oppressed groups such as Latinx and black students, we need to investigate our complicity in such failure and inequity. Unfortunately, the institutional data available to us hardly constitute solutions, and such data might well obscure more than they reveal about students' actual learning experiences.4
Our article is an effort to meet these challenges. In the era of big data, we still insist on broader and harder-to-quantify measures of success, even (or especially) in lower-division introductory courses enrolling first-year and second-year students. For us, success in these classes involves more than a passing grade. Success also means ensuring that all our students understand history as interpretation supported by evidence, that they develop stronger academic skills, and that they go forward with a sense of how the past informs their lives as active participants in a larger political community.
Working in a university system that privileges “data-driven decision making,” we decided to collect our own data to understand our specific students and our own learning goals. For our core evidence, we analyzed precourse and postcourse surveys from nearly four hundred students enrolled in required introductory U.S. history classes at five different Csu campuses, and we compared survey responses with students' grades in those courses. We also assessed our students' interpretive and analytical skills and conducted semistructured focus-group interviews with twelve students, to answer vexed questions about how to define, measure, and predict student success in introductory U.S. history courses. In this pursuit, we worked with education consultants, taught ourselves survey research and statistical analysis, and even added a coauthor with expertise in education and statistics.5
Our findings dispel common assumptions held by many faculty—ourselves included—to explain why some students pass our courses while others struggle. Our data suggest that scholars must grapple with racial disparities as they investigate variations in student grades. At the same time, our data show that attitudinal and cognitive factors are equally important. Most fundamentally, students who shift their perspective to understand history as interpretation rather than memorization earn significantly higher grades in our classes. A skeptic might treat this finding as a tautology, but such a dismissal would be short-sighted. As education reformers across the country legitimately target the social and budgetary costs of high failure rates in introductory U.S. history classes, our research suggests that teachers who make the practice of historical interpretation transparent and understandable for students can make a difference not only in what we most want—to improve students' historical thinking skills—but also in the new metrics of student success.
In the following sections, we introduce our students and the opportunities, joys, and challenges of teaching in the nation's largest university system. We next describe the teaching principles that we adopted in redesigning our courses around historical thinking skills, particularly as we adapted the excellent scholarship on history teaching to the specific contexts of our large and diverse public university classrooms. A third section presents our survey process and data analysis. Using multivariate regression, we explain how some of our initial assumptions (for example, that student demographics and economic concerns would closely correlate with student grades) proved less statistically valid than another correlation—the tighter connection between students' grades and their attitude toward historical study. Finally, we examine students' reflections, which help explain the link between historical thinking skills and students' persistence in introductory history courses that they might otherwise have failed. In these ways, our essay seeks to rise to the myriad challenges presented by a data-driven educational environment while advancing the core principles of the historian's discipline.
Reassuringly, we argue that history faculty can make a dent in DFW grade rates and reduce equity gaps in introductory U.S. history courses by more fully understanding the students in their classrooms and by recalibrating students' attitudes toward our discipline. Successful students, such as the one who opened our essay, do not need to “like” history, but they do tend to recognize the work of interpretation at the heart of historical study. In the age of student success, we can make a significant difference by knowing our students beyond big data's relentless graphs and tables and by helping students gain a stronger sense of our discipline. These are encouraging signals in an otherwise-unyielding algorithmic world.
Hope and Struggle in the People's University
Students in the Csu system belong to groups historically excluded from higher education, with nearly half defined as underrepresented minorities (Urms). In the introductory U.S. history classes we surveyed in fall 2017, approximately 41 percent of students identified as Hispanic or Latino; students of Asian-Pacific Islander origins represented about 20 percent; white students made up 17 percent of our sample; and African Americans were 4 percent. Students who identified with more than one ethnic or racial background made up slightly over 11 percent of our sample. Earning a college degree is unfamiliar territory to most of these students. In our survey sample, 44 percent of students identified as “first generation,” meaning that their parents did not attend college. When we include students whose parents started college but did not finish, a remarkable 71 percent of enrolled students might be considered first generation.6
These numbers help explain why many in the state proudly refer to the Csu system as “the people's university.” Wealthy or poor, documented or undocumented, Californians can often find a robust, full-service university close to home. The twenty-three Csu campuses offer Californians a path to economic opportunity and a quality education, though it bears emphasizing (and repeating to policy makers) that the system suffers chronic shortfalls given its role as an engine of upward mobility. The solutions that we outline depend not just on professors learning proper modes of pedagogy. Faculty also need the security and institutional resources to do so.7
In line with the Csu system's grand civic purpose, undergraduates must complete two courses in U.S. history and government to graduate. As part of this American Institutions requirement, our introductory U.S. history courses stand as high-stakes rites of passage. If students do not pass, they cannot graduate. In light of these stakes, the dashboard figures stunned our team of historians. Between 2011 and 2017, almost seventy thousand students took introductory U.S. history on the five Csu campuses involved in this research project, and nearly twelve thousand (roughly 17 percent) received grades of D, F, or W. Across the whole Csu system, the rate was even higher, with one in every five students falling into the DFW category. To our chagrin, administrators began to label our classes as low-success “bottlenecks.” Big data revealed patterns that we had not seen—or even looked for—in our individual courses evaluations and assessments. Perhaps we did not know as much about our students as we had thought.8
Our introductory courses also presented equity gaps, particularly for Urm students. For example, at Fresno State, 10,873 students took the first half of introductory U.S. history between 2011 and 2017. Non-Urm students earned, on average, a 2.68 Gpa, and Urm students a 2.44 Gpa, creating a 0.24 achievement gap. At the Los Angeles campus, the Gpa gap was 0.32. First-generation students (only some of whom were Urm) experienced somewhat more equal outcomes. At the East Bay campus, the Gpa gap between first-generation and non-first-generation students was 0.12. The gap at the San José campus was 0.14. These gaps appear to be generally in line with other general education courses for the social sciences. Over time, and across multiple courses, such gaps have a troubling cumulative impact. For example, the six-year graduation rate for Urm students stood about 10.6 percentage points below that of non-Urm students for the last measurable cohort of Csu graduates, which began in 2012.9
Moreover, DFW grades, especially if earned early in a college career, can entirely derail a student's progress to graduation. The Csu system has not identified system-wide “predictor” classes for graduation, as some colleges and universities have done, but introductory U.S. history classes generate red flag warnings that we should heed. According to Csu campus data spanning 2011 through 2017, just under half of the DFW grades given in our introductory U.S. history courses were earned by students who struggled only in those history courses. These students earned grades of C or higher in all their other courses that term. Even so, a recent study by the John N. Gardner Institute for Excellence in Undergraduate Education suggests that students such as these are more likely to drop out of college completely. To locate those “many thousands failed,” a good place to start is the Csu system.10
The sheer scale of Csu campuses may exacerbate academic struggles, especially for students who are not sure if they belong. Many of our students experience a simultaneous crush of anonymity and fear in introductory history classrooms. In a focus-group interview, one East Bay student explained, “when I first stepped into Meiklejohn Hall, I felt overwhelmed because it is very big…. I was like, how do 120 people fit in here?” A Fresno State student offered this direct admission: “You come to college and you're like—you're just kind of scared.” Activities that faculty often see as fun and energizing can leave students feeling “super nervous” with hearts “pounding.” Now add to these cortisol-inducing settings the sheer newness and mysterious trappings of academia to many underserved, underrepresented students. That Ph.D. in history, or “Dr. Chilton, historian,” must surely seem like inscrutable—and deeply hierarchical—code language to any number of new students. Moreover, high enrollments in lower-division classes, which range from fifty to two hundred undergraduates among our research team, can easily undercut best efforts in supporting and knowing students who are unfamiliar with (or not entirely convinced about the value of) the rites and passages of academia.11
Very often living at home, our undergraduates balance competing responsibilities outside school. Finding time for “extracurriculars” that might foster a sense of belonging in college can be difficult. The burden of financing college presents another inescapable reality. Pell Grant recipients made up about 41 percent of our surveyed students, and 81 percent reported feeling worried about making ends meet. Students' financial concerns also place constraints on their time. On average, students taking our introductory U.S. history classes worked fifteen hours per week, and 20 percent worked thirty or more hours each week. Family obligations pose further challenges. Our students report spending twenty hours per week, on average, caring for family members, often grandparents or younger siblings. Also consider California's affordable-housing crisis and pronounced income inequality. Although we did not directly ask about housing and food in our surveys, a 2018 study found that 11 percent of Csu students had experienced homelessness at least once in the prior twelve months, and as many as 42 percent expressed some level of food insecurity. We can surmise that finding a place to sleep, or money to buy food, is often foremost on many of our students' minds.12
Our classrooms thus reflect the “real world,” not a disconnect from it. We can all see and feel the disinvestment in public education in recent decades—for instance, when we gather in classrooms that feel like a form of time travel to the 1960s or to some future apocalypse. Stained and dangling ceiling tiles in Los Angeles, air so stifling that a student passed out in San José, and rodent infestations at East Bay give learning and teaching in the Golden State an aspect of derring-do, and not incidentally, an object lesson on the rise (or at least the fall) of the New Deal liberal order. Yet a focus on structural inequalities should not lead us toward a deficit model in thinking about our students. Although most do not take part in a more traditional residential college experience, they bring to class tremendous knowledge drawn from their immersion in diverse communities and from their success in overcoming obstacles to reach college.13
Course Design for Historical Thinking among Nonmajors
We were good teachers, if our course evaluations stood for anything. Students spoke of our “enthusiasm,” our “caring,” and our expertise—often in that order. We also “knew” we were good teachers because we entered the classroom with a love of primary sources. To our lights, primary sources inherently engaged students by inviting them into a rich world of investigation. Something was amiss, however, as we learned from administrators' data showing high failure rates and unequal outcomes in our introductory courses. For one of its early “student success” projects, the Csu Chancellor's Office in 2015 offered faculty a precious one-course reduction in teaching duties in exchange for one week sequestered in a windowless hotel conference room learning about “course redesign with technology.” We had no special interest in technology, but we took the bait. In time, we formed a collegial bond and found a handful of creative Csu administrators willing to help us investigate our students' learning. As a faculty cohort, we worked to change what we could control: our course design. We found no silver bullet, no single intervention to disrupt long-standing patterns. Our teaching for nonmajors instead derives from extensive practice and tinkering, conversations with colleagues, and close reading of scholarship on teaching and learning. Like all good teachers, we continue to adjust even now.14
We started with “backward course design”—that is, we abandoned our preconceived list of topics we “needed” to teach. We began instead with our broad goals for the course and worked backward to select appropriate activities and content. Experience had taught us that reliance on textbooks and attempts at comprehensive coverage in a “survey” of U.S. history stifled historical thinking and deprived students of the chance to explore the past as historians do. Our course designs, therefore, focused on cultivating cognitive shifts that help students see history as inquiry rather than memorization. To achieve shifts in historical thinking, we pursued a set of core methods inspired by earlier teachers but adapted for today's college students. These methods—each elaborated below—include frequent skills-building and low-stakes assessments, collaborative and peer-based learning, and careful practice connecting history to the present.15
Good pedagogical intentions, of course, are not enough. We needed to know if we could coax nonmajors into challenging interpretive work, from primary-source analysis to questioning conventional narratives. Through the years, occasional student course evaluations would include complaints that we were not teaching “real” history. We usually ignored these comments. When redesigning our courses, these dismissive evaluations suddenly seemed more salient. We realized that we needed to dive into the scholarship of teaching and learning to figure out how students viewed history.
Scholars of history learning describe a “cognitive bottleneck” facing students who view history as a fixed narrative of unchanging and readily found facts. Research in the last two decades shows that shifting students' “epistemic cognition” about a discipline demands much conscious instructional work. Helping students understand the procedures necessary for historical interpretation requires repeated assessment and reinforcement throughout a course. Teaching historical thinking also demands a refined sense of precisely where students struggle and stumble. Although history majors are hardly immune to “epistemic” misapprehensions, this task is especially challenging with nonmajors in introductory courses.16
Understanding our students is fundamental to this process. Most students have little academic experience with historical interpretation and investigation. We therefore do students a disservice if we artificially separate content acquisition from active learning. Only by consistently practicing interpretive skills can students succeed with coursework that demands analysis rather than recitation. We discovered that students often become lost, overloaded with content from lectures and textbooks, leaving them unable to see how primary sources can yield understanding of the past. Another major revelation came by grasping what scholarship increasingly confirms: lecturing at length leaves all students poorly served, with disproportionate costs for first-generation and Urm students.17
In this project, we have investigated the expectations, fears, and even well-intentioned hopes that pull students off course and lead to frustration, if not wrath, as they log in to online grade books and see their scores decline. To our students, low grades can appear arbitrary and opaque, with good reason. Even “simple” assignments such as a three-page essay can obscure rather than illuminate the skills and interpretive work we wanted students to practice. All too frequently, both students and faculty muddled through learning and grading—with failures and frustrations at both ends. But with a more evidence-based understanding of our students' attitudes and dispositions, we have worked to “challenge and shift ways of knowing about the past.”18
To understand the expectations and beliefs that students bring to our classes, we took time on the first day of class to survey nearly four hundred students at five Csu campuses. We asked them to rank their level of agreement with statements relating to history and student life, and we provided open space for students to answer two questions: “Besides a passing grade, what do you hope to gain from this course?” and “What do you think your biggest challenge will be in this class?” These precourse surveys proved revelatory. They dispelled our own cognitive assumptions about students—that they are poised, ready, and excited from day one to jump into the heady work of historical analysis. We did not anticipate, for instance, that nearly two-thirds of our enrolled students would agree with a statement that declared, “Doing well in a history course is mostly a matter of memorizing facts and dates.” Just 14 percent disagreed, and the remainder were not sure. Meanwhile, only 7 percent felt confident in their ability to “analyze complex evidence from the past,” and a mere 9 percent felt ready for “evaluating historical arguments.” These responses suggest students' widespread uncertainty about key disciplinary phrases that we historians use all the time in an unreflective way, as though everyone knows what we mean by “analyzing” evidence and “evaluating” arguments.
As jarring as it may seem, we need to come to grips with the likelihood that the vast majority of our students start class without the foggiest notion of what we professors mean when we describe our discipline. Indeed, a far greater number expressed concern about how to do the intellectual work of … rote memorization. In presurvey comments, students wrote that they hoped “not to forget the material, like I have in past classes.” They wanted to “actually remember” what they have learned, “even after the semester is over.” One student emphasized the problem of retention in a tellingly redundant way: “I hope to gain the skill of remembering facts from history naturally rather than memorizing them from the top of my head.” Here, in a nutshell, is the “cognitive bottleneck” history teachers face in reframing for students the intellectual work we expect them to do in our classes. It may seem banal to state, but the first step in teaching students historical complexity may be the simple task of discerning where they really are.19
We also found obstacles related to student motivation. Many of our students remembered “boring” K–12 history classes. One appeared discouraged from the start, remarking, “Honestly, I don't know what I hope to gain from this class.” Others wished to feel “a little more enthusiasm towards history” and “to actually engage” with the material. One student wanted “to start actually liking it,” and another hoped for “a better understanding of why I need history in my life.”
Our first opportunity to convey to students a different vision of history comes with our syllabi. Our syllabi range from colorful and conversational to pragmatic and contractual, but they all invite students into an investigatory enterprise—and consciously, many times over. We frame analysis of “historical evidence” and exploration of “questions” and “problems” as the central work of our classes. We can thus preserve flexibility for our individual expertise and temperaments while still sharing critical components of our course design model. Our assignments run the gamut from more traditional essays to documentary filmmaking and Reacting to the Past simulations. In all cases, we shed artificial divisions between supposedly lower-level content acquisition and deeper-level “analysis.” This imperative leaves no component of our courses untouched. Our advice on taking notes, for instance, stresses the importance of linking basic facts to larger interpretive questions. Requests to silence cell phones provide a chance to remind students of the need for higher-level thinking and collective knowledge building in the classroom.20
Knowing that cognitive shifts require deliberate and sustained practice, we design our courses so that students engage in interpretation with every single class activity. Expectations of mindless cramming can be difficult to dislodge. These assumptions can also generate resistance to modes of teaching that do not provide students with one true answer. In response, we emphasize the scaffolding approach, in which we introduce students to different skills in a progression that helps them build toward historical thinking. In our first sessions, we model for students how to read primary sources as historical artifacts. At this same early stage we also teach how to read secondary sources to find and evaluate an author's argument. Short but frequent assignments let us see within a week or two how students are learning. These so-called formative assessments might involve short reading quizzes or informal source annotations. Because students need to practice academic writing skills, we all provide short paragraph–writing assignments that lead to larger, thesis-driven essays. Formative assessments pose risks for faculty, especially the danger of an avalanche of grading. Without a squad of Ph.D. students to provide grading labor, we use peer editing, online quizzes, and other measures to balance low-stakes assignments with manageable grading time.21
Community building provides another pillar of our courses. Learning in formalized teams rather than in haphazard break-out groups can help students feel less anonymous on large, commuter campuses, especially when they have little time for leisure on campus. To be candid, team-based learning also helps us deal with a heavy grading workload without depriving students of active-learning opportunities. We build community in both old-fashioned face-to-face ways and through collaborative digital platforms. We bring peer mentors (advanced history majors or M.A. students), when available, into the classroom to support team activities. To foster listening and debate without distraction, we discourage the use of phones, tablets, and computers during many class meetings. At the same time, we embrace digital technology's ability to visualize students' patterns of thinking. Online tools such as Google Docs, Qualtrics, and Kahoot let us share and analyze students' thinking more quickly and on a larger scale than traditional “reporting-out” sessions. Moreover, online collaborations outside class help connect our large number of commuter students with each other.22
Team-based learning can promote the development of historical thinking skills in ways small and large. Given the “unnatural” habits required to practice history as interpretation and to see the past as different than the present, students must develop a collective acceptance of these elusive concepts to succeed. To this end, classmates make powerful teachers who can help their comrades develop the skills and mind-sets needed to thrive in history classes. Through collaboration and discussions, students learn how to extract meaning from difficult primary sources better than through individual confrontations with frustrating texts. For example, one of us provides a ship captain's logs from the early Atlantic slave trade. Students read the source together, decoding slave-trading language to recover Africans' own lived experiences. Teams first work on a single-sentence interpretation of a source. They then compare their work with the efforts of other teams and finally read the argument of leading scholars writing on that same source. In this way, our classrooms resemble the communities that scholars themselves create to work through challenging material. Another professor assigns jigsaw-style collective essays where each student writes a paragraph that their team assembles into a coherent single essay. Students might receive five different sources on anti-Chinese nativism and then work in class to draft a collective thesis explaining the intersection of economic and cultural anxieties in nativist politics.23
Learning with an emphasis on historical interpretation, academic skill building, and deliberate community consumes significant amounts of students' time and focus. Because time is arguably the most precious resource in any course, we proudly abandon the pretense of “coverage” in our history content. As some historians have stressed for over a century, students learn more about history when we slow down and let students explore a smaller number of topics in greater detail. Some of us build our course around a single overarching question, such as this from the Los Angeles campus: “For those Americans struggling to achieve equality and prosperity, what have been the pros and cons of working within the system, compared to pursuing change by working outside the system?” Others select a handful of thematic “pivot” moments for students to explore in richer detail. By and large, our solution has been to select content that connects to students' experiences living in diverse racial and ethnic communities and to emphasize history from the bottom up.24
We also embrace connections to the present. Of course, we recognize the dangers of presentism and the search for timeless but ahistorical moral truths. Scholarship on historical thinking warns against students (and teachers) who offer simplistic parallels that ignore differences between historical contexts and the present. In fact, much of this literature frames present-day discussions as something to avoid in the history classroom. In contrast, many students enter our classrooms hungry for civic engagement and for an understanding of present issues. In one typical survey response, a student wanted to “learn from … successes and mistakes” in the past and “to apply that knowledge towards today's politics, history, and world news.” Some wanted to understand “how recent events connects with past events.” Others hoped the class could help them “stand up for my own opinion confidently in political debates.” Despite students' fixation on rote memorization in their earlier history classes, 84 percent of our surveyed students felt that previous history classes helped them understand current events. Not surprisingly, many wanted to explore similar connections in our classes.25
Students' interest in connecting past and present stemmed at times from a desire to make sense of their community or racial or ethnic identities. “I've always liked history,” claimed one student, “because my race, the Hmong race, it's like nothing's really written, it's all oral tradition, so it's like I'm losing it slowly as I become more Americanized. It's like I've got to find my past if I'm going to understand myself.” Another student remarked, “My dad's black and my mom's white…. Anything having to do with, like, slavery or any of that kind of stuff is always super interesting … if these things in history wouldn't have happened, my family wouldn't be my family today.” It is worth remembering, then, that a good number of students enter history classrooms with deeply personal or civic goals tied to the present.26
We do not see any categorical reason to thwart these students' hopes, especially now, when historians are in frequent demand to contextualize strains within today's civic institutions. Moreover, ties to current affairs advance the civic mission encoded in the Csu American Institutions requirement. So how do we help students move beyond clichés about history repeating itself? We find the best solution is repeated practice. We create multiple opportunities for students to find and evaluate connections between past and present. One professor teaching the first half of the U.S. history sequence concludes primary-source exercises by having students translate their source into everyday speech on Google Drive, followed by an in-class discussion on how the document shapes life in the United States today. A reading of John Winthrop's sermon, “A Model of Christian Charity,” for instance, ends with a discussion of what Puritan ideas of community mean in today's more racially and ethnically diverse society. With a Reacting to the Past simulation of New York's 1788 constitutional ratification debate, one teacher asks students to consider the influence of wealth on politics and how economic inequality affects faith in democratic representation now. Another faculty member asks students to compare voting rights challenges in 1965 to current contests over electoral exclusion.27
Though time-consuming, letting students engage in repeated attempts to connect past and present can also promote classrooms where students feel they belong. The more students engage in dialogue with each other over tricky questions of interpretation, the more they practice the kind of civil, evidence-based exchanges required by a functioning democracy. Further, when our classrooms hold these discussions, they promote peer communities that can help students persevere through the hardships of college. Collaborative discussions on current issues that students experience in their daily lives can also lessen cultural gaps between students of color and a largely white faculty, creating classrooms where faculty learn from students. An embrace of the present in history teaching promises a more horizontal distribution of knowledge and authority.28
Our emphasis on connecting past and present and on promoting students' interpretive and civic voices means that even “basic” introductory history courses cannot be prepackaged. These course designs rely on faculty expertise and passions, whether from a full professor or a recent master's degree recipient on the adjunct circuit. A sense of craft matters greatly here. Students need teachers who can model how to read between the lines of sources and who can adjust historical topics to respond to the specific communities in which their students live. That said, for all our individual specialities and curating of key primary sources, we cannot simply stand alone as faculty in the face of big data in a large university system. To disrupt patterns of historic inequity, working singly against long-standing forces is not enough. We need to work at scale, looking for the patterns and practices we share and testing those practices by consistently and carefully listening to our students. This is the most effective way for us to make significant change of value for our students, our administrators, and, not least, our discipline.
New Methods for Measuring Historical Thinking
With our redesigned courses completed by 2016, we eagerly awaited results. By the standards of the dashboard, our new courses mostly succeeded. Our DFW grade rates, which had previously hovered around the national average of 25 percent, fell to between 8 and 11 percent. We were heartened but also eager to learn more. We thus began a large-scale research project to explore the factors that contribute to success and failure among our students. Like the dashboard creators, we wanted to explore student grades in relation to demographic variables, but we wanted to do so in tandem with student attitudes toward academic life and the study of history. In other words, we wanted what the dashboard could not provide: a sense of the intellectual and educational life hidden behind all those one-dimensional DFW grade charts. We thus created a survey to gather data about students' demographic backgrounds and concerns about balancing school, work, and family. We also asked students to agree or disagree with statements about academic habits and historical study. During the start and end of fall 2017 classes, 374 students from five campuses voluntarily completed both precourse and postcourse surveys during class time with an easy-to-use digital survey tool (Qualtrics). We then matched student responses with their corresponding course grades and rendered the data anonymous.29
Our survey analysis confirmed some of our assumptions but surprised us in other ways. Importantly, paired t-tests revealed that students generally showed significant improvements in historical thinking (p < .001) and civic disposition (p = .009) from the beginning to the end of the course. This shift suggests that students increased not only their ability to think like historians but also their perception of the relevance of studying history. Demographic variables did help explain some grade variation, but variables measuring student attitudes toward history explained similar amounts of variance. The key nondemographic variables were students' change in historical thinking from the start to the end of the course and, independently, students' understanding of historical thinking at the end of the course. These nondemographic variables measured growth and suggest that students who started the course feeling unsure about what the study of history entailed would score four percentage points higher if they left the course feeling measurably more confident about their historical thinking skills. When we controlled statistically for the influence of demographic and socioeconomic variables, we saw that all students scored a grade almost five percentage points higher than their peers' scores if they held a more “expert” understanding of historical thinking. As faculty, we find these results clarifying and empowering. The more we design our classes to promote a vision of history as interpretation, the more we are able to move all students toward greater success.30
We analyzed four types of potential predictors of student grades: demographic variables, subjective concerns, changes in student thinking about history, and postcourse student thinking about history. Demographic variables included gender, generation, language, and race/ethnicity. Subjective concerns included students' self-reported level of concern about paying for college and balancing school with work, family, social life, and social media.
The concept of “expert shift,” borrowed from scholars in science education, helped us explore student attitudes toward historical thinking. Five of our questions asked about attitudes to history and self-reported confidence in the ability to read and analyze historical sources. For instance, we queried students about their agreement with statements such as “Doing well in a history course is mostly about memorizing facts and dates” and “Historians should come to the same conclusion when examining evidence from the past.” We rated students who agreed with these statements as more “novice” historical thinkers, while students who disagreed were rated as more “expert.” By aggregating five statements such as these, we arrived at a measure of the scale of the “expert shift” that students experience in our redesigned courses—along a continuum from perceiving history as a discipline based on the retelling of memorized “facts” to a vision of history based on interpreting complex evidence and evaluating historical arguments.31
With this survey data in hand, we now had a way to understand students as complex individuals. But to understand the variables weighing most heavily on student grades, we needed to be able to disaggregate the data, or look at each of these factors in turn. Thus, we conducted a stepwise multiple regression analysis of the data to determine which of the potential predictors of student grades were the most significant when we controlled for the influence of the other variables. This allowed us to recognize the influence of multiple variables on student lives, while understanding which were the most predictive of their grades. For example, when we statistically controlled for the effects of other blocks of variables, we could see that some of the variability in grades among students within the same racial or ethnic group could be explained through their differing levels of historical thinking. For example, among all Latinx students, levels of financial concern or parents' educational status were less significant predictors of grades than whether these students held “novice” or “expert” views of historical thinking.32
Our findings therefore agreed with, and differed from, the Csu dashboard in important ways. The dashboard presents the view that student grades correlate most strongly with demographic factors such as ethnicity, gender, and parents' education levels. To an extent, this proved true. Students in our small-sample-size group that included Black/African American, American Indian, and Pacific Islander students, had, on aggregate, lower grades, while students who identified as Asian had higher grades. But note that these “groups” of students are highly heterogeneous. Our analysis also found that gender, generational status, and language were not significant predictors. Further, entering all the demographic variables into the first step of the model (located in the first column of Table 1) explained only 2.5 percent of all the variation we saw in course grades. In sum, when we controlled for the effects of demographic variables, we found just a few significant differences. Using only the demographic variables available to faculty on the dashboard means that most of the variation in student grades remains unexplained.33
This table shows the changes in R2 (the coefficient of determination, or how much of the variation in grades can be explained by each set of variables) with each block of predictors (demographic variables, subjective concerns, changes in thinking, and postcourse thinking) on students' grades. B represents the unstandardized regression coefficient, SE B stands for the standard error of that coefficient, and ß stands for the standardized coefficient. The significant coefficients for each model are also available in this table. For a full explanation of how these results were calculated using a stepwise multiple regression method and for full tables of results, see the online appendix for this article, https://jah.oah.org/projects/beyond-big-data.
This table shows the changes in R2 (the coefficient of determination, or how much of the variation in grades can be explained by each set of variables) with each block of predictors (demographic variables, subjective concerns, changes in thinking, and postcourse thinking) on students' grades. B represents the unstandardized regression coefficient, SE B stands for the standard error of that coefficient, and ß stands for the standardized coefficient. The significant coefficients for each model are also available in this table. For a full explanation of how these results were calculated using a stepwise multiple regression method and for full tables of results, see the online appendix for this article, https://jah.oah.org/projects/beyond-big-data.
We initially expected economic concerns to be a significant predictor of student success in our courses. We know from our surveys that some students face significant financial hardships and work many hours while also pursuing full-time studies. In all, our students reported working an average of 15.17 hours per week (but with a very large standard deviation of 14.28). Yet when we controlled for demographic variables, students' concerns about finances and balancing school-work-life responsibilities could not predict student grade outcomes. For example, a white student working a high number of hours per week might feel confident balancing work and school. Conversely, an Asian American student may work only a small number of hours a week but feel that this negatively affects their ability to keep up in school. Without the ability to relate these beliefs and identities to each other, the dashboard—the high point of our administration's focus on systematic data—gives short shrift to our students' complexity.
Once our data analysis showed that student financial concerns had little predictive power (after controlling for demographic variables), we removed that data from our model and turned our attention to the shifts in thinking that students experience in our courses. For this final step, we entered postcourse levels of historical thinking, civic disposition, and mind-set into our regression model. The full model that included all predictor variables further refined our story. This full analysis revealed that demographic variables were not as predictive once we took into account students' changes in historical thinking (see the coefficients [B] in the full model in the third column of Table 1). The model revealed that expert-like shifts in historical thinking were significantly associated with a higher grade in the course. Remarkably, students who improved their historical thinking by at least one step between the beginning and end of the course were predicted to earn a grade that would be 4 percentage points higher (for example, an 82 versus a 78) relative to someone who showed no improvement in historical thinking. Students who scored a one-step-higher interval of historical thinking than others at the conclusion of the course were predicted to earn a grade that would be 4.8 percentage points higher. This result, however, may reflect some students who arrived in the course with strong skills and left with the same. Given the small difference between these variables (less than 1 percent difference in grades), we see the greatest significance in the impact of transforming students' attitudes toward history.
We can measure student success solely through grades, but a more meaningful measure is the shift students undergo during the course. This may be the shift from a novice thinker who finds history boring to one with emerging skills in historical interpretation, or from an emerging historian to one who finds the confidence to engage in their own historical interpretation. According to our data findings, this shift will help both kinds of students achieve a higher grade in the course. As teachers, we cannot change students' level of historical thinking when they start our course, but we can alter the way we teach by listening to what students say about history when they enter our classroom.
Students Define Success
For all the complexity in our data set, numbers alone could not explain how our particular interventions moved students toward better historical thinking. Student comments, collected as part of our survey and in subsequent focus-group interviews, give voice to our quantitative findings. In their own words, students were able to describe their shifts in historical thinking and civic confidence and what this meant for their experience in the course. They emerged from our classes as more complex thinkers. They understood the study of history to be an act of interpretation rather than rote memorization, even if they remained unfamiliar with our disciplinary language. They also appeared more confident in their ability to do the work of historians, to see connections between the past and the present, to identify change over time, and to make evidence-based arguments.34
When given the chance to speak freely on the topic, students emphasized the development of these interpretive skills, particularly analyzing and establishing the credibility of primary sources, and determining the causes and consequences of events. One focus-group student described this approach as “education” rather than “schoolwork,” by which he meant that these courses “teach you the steps to do something rather than just say, ‘Here's the information.’” Others recounted how learning from primary sources, which they called “smaller stories” or “actual evidence,” taught them to think critically about credibility, authorship, and potential bias. Although students did not use these disciplinary terms, in their own way they were describing how they had arrived at a more complex understanding of historical interpretation through the close reading of sources.
After completing the course, students also noted having moved beyond the notion that studying history was “just memorizing facts” and taking a test. Instead they described their experience as “putting ideas together and kind of using that to engage all the senses.” In the focus-group interviews, several students were critical of learning names and dates precisely because they saw greater value in developing interpretive skills and seeing connections between the past and the present. One student admitted, “I am not a history person, so I don't give a crap when the date of a certain war was,” because “understanding why and how things happened in the past” was more important for making sense of today's world. Though we are heartened by this (somewhat profane) interest in the “why and how” of the past, we lament how the student still associated being a “history person” with rote memorization. Such persistent views of our discipline, shared by the student we profiled in our introduction, suggest just how entrenched these beliefs are, even among those who otherwise see value in studying the past.
Students often attributed their shift in attitudes and reasons for doing well in the course to specific teaching methods in our course redesigns. At the end of the semester, for instance, students still viewed reading as one of their biggest challenges, but they also spoke of how common active learning practices such as team-based learning, peer mentoring, and/or discussion sections provided a smaller, dependable community of learners. They valued classroom environments where they could personally connect with other students, allowing them to “work together, like collaboratively,” and “hear from [their] peers” who might offer “different ways to learn” or understand the material. They were also able to connect these interactions with their success in the class. As one student put it, “it's a little uncomfortable at first, but then when you get to your tests and you actually know things.” Focus-group students were honest that these peer relationships were mostly important through the duration of the course and the semester, but some reported making lasting friendships or keeping in touch with other students afterward.
Students also credited scaffolded assignments as important to negotiating the challenges of college. Using small assessments to build to a larger assignment not only helped develop and refine their interpretive skills but it also alleviated some of the stresses of college life, especially common among first-generation students. One student appreciated that the professor “breaks it down … unlike other classes I have where they just are like, okay you have a big project at the end.” Another student recalled, “It wasn't like, oh my God we have a project due, we have to get it all done. It was more like, okay we already had to submit the storyboard last week, we already had to submit the script last week.”
Students also emerged from our courses feeling more civically confident. Repeated efforts to connect past and present helped them see how previous events shape contemporary issues and their own lives. Some expressed being empowered by the knowledge gained in our courses. In our postsurvey, nearly two-thirds of our students believed the course better prepared them to participate in political debates. One student described the course as learning “the fundamentals [of] how this nation came to be,” leaving students “more likely to engage in a political action or vote.” Another spoke of being able to “better understand … when somebody talks about issues,” having studied the history of partisan debates over federal and state government power. The course helped another student think more seriously about the current debate over building a wall along the southern border. “You wouldn't be able to understand [this issue] fully,” the student argued, “unless you knew the full history of the United States.”
Making these connections proved especially meaningful to several focus-group participants who spoke of carrying on these conversations with family and friends. At Fresno State University, students recalled how in-class debates over the memory of slavery and the future of Confederate monuments continued outside the classroom. Others stressed that learning different perspectives on the past, instead of one narrative story, helped them be more confident talking about contemporary issues with their peers, because “you're able to argue against it with facts and not just emotion.” There were some students who came to see certain contemporary issues in a different light after having completed the course. One student talked about finding a new perspective on the current debate over athletes kneeling for the national anthem after learning of the importance of dissent and protest in the nation's founding. Others expressed feeling more empathetic and aware of inequalities in American society today as a result of having taken this course. Before learning the history of redlining, for example, one San José State University student felt “confused” and “couldn't appreciate the circumstances that some people were in.” But “now knowing the reason behind why something is the way it is today allows me to kind of be upset.” Such awareness “allows for positive change,” the student argued, for it was only by learning how “we got here [that] we can sort of start seeking to address” problems in society today.
By the end of the semester, students were better able to see how building historical thinking skills affected their success in the course. They understood that studying the past was an act of interpretation, requiring reading sources critically and drawing evidence-based conclusions. Developing these skills not only made class more enjoyable, they argued, but it also helped them understand how the past shapes their lives today. This shift in attitude not only likely led to a better grade in the course but also fulfilled the core mission of California's American Institutions requirement. Students left our classrooms believing that the knowledge and skills they had acquired would better “enable them to contribute to [their] society as responsible and constructive citizens.”35
Reckoning with Data while Seeing our Students
In an age of student success dashboards, faculty cannot merely play the critic from the sidelines, or, worse, ignore the data presented in these new digital spaces. Big data such as the Csu Student Success Dashboard might not explain much about student learning, and it should not stand as a measure of faculty teaching effectiveness. The dashboard does, however, reveal historical patterns. With the aid of such data, we believe it is time to reckon more fully with the introductory U.S. history course in its historical context. Most of all, we need to understand how these lower-division courses influence students' ability to earn a bachelor's degree and how they contribute to an unequal college experience. In light of equity gaps in our classrooms, we need to examine our sense of ourselves as neutral participants who merely report how students perform.
We worry that the dashboard might exacerbate assumptions—that all students are struggling, facing stubborn challenges outside the classroom. When we meet one-on-one with students who are anxious about assignments, we frequently learn that outside work is a stressor, that family obligations impose on their time and emotional well-being, and that bouts of self-doubt hinder the submission of assignments. We do not wish to discount these student feelings and experiences; our students truly do need support from advisers in finding the right mix of classes, and some personal crises do require students to withdraw from a class. At the same time, our research for this essay shows that these conversations misattribute blame for student struggles to forces outside of our control. Anecdotal office-hours impressions distract us from powerful solutions already in our control: our historians' repertoire of teaching skills.36
Our efforts to understand student success suggest that a good number of key challenges lie with us faculty—particularly that we assume too much about student understandings of our field. History faculty have not fully grasped how ensconced certain ideas about our discipline are in students' minds. This is true even as many K–12 educators have moved well into inquiry-driven models of history education (particularly in California). A growing fleet of student success administrators might propose varying strategies to reduce DFW grade rates and equity gaps, but we history professors already have powerful tools at hand in our syllabi and course designs. We have the duty and ability to effect change. We just need to make our own “expert shifts.”
Historians also have a duty to demonstrate “success” beyond student grades. Certainly, good assessments, designed for lower-division history classrooms, will help. But, frankly, based on our experience in the Csu system, even the best assessments will not help us preserve the place of U.S. history in general education programs if we cannot show that our teaching moves all demographic groups toward academic success. We offer here a solution. By collecting our own data, we show that growth in historical thinking (what we care about as historians) actually moves the needle on the measures of student success that our administrators must account for. Teaching our disciplinary insights and our methods for reasoning with evidence can increase students' success and intellectual growth. We also need to match this lesson to our specific students. Here is a simple takeaway from our research: get to know your students, especially in large introductory classes. In our online appendix we offer models for how to listen to students.37
We have also learned that we cannot stand on rigid disciplinary tradition and eschew connections to the present in our classrooms. Presentism can impede rigorous historical analysis, but, in introductory history classes, a professorial bias against presentism also forecloses civic debates that students want and need. Our own surveys tried to capture shifts in “civic dispositions,” and our evidence hints at some gains in students' civic capacities and confidence. Future research by historians can do more to measure students' civic mind-sets before and after introductory U.S. history courses. New research can also explore how students think about the “relevance” of historical understanding for the present.
Further, we need more discussion on the hard work of generating professional “mind-set” shifts. Just like our students, we need community to find our own expert shifts as teachers. In our case, we needed collegial conversation and friendship, plus considerable time to create new assignments and survey tools. Our Ph.D. training taught us to work on our own, but the rise of the data-driven university requires a collective response. It takes a team to develop new teaching mind-sets and skills, to negotiate campus institutional review boards for human-subjects research, to build statistical databases suitable for multivariate regression analysis (difficult!), and to run focus-groups interviews (fun!). As history departments take stock of student success in their introductory U.S. history classes or programs, we strongly advocate doing this work in collegial teams, using face-to-face and online conversations. If administrators are serious about student success, they also must provide support and incentives for these collaborations. Grappling with big data by oneself is a lonely and likely futile pursuit. We must do this work together, taking professional pride in the distinctive ways that we historians can foster forms of “success” that are genuinely meaningful both to students and historians.
Notes
We have many debts to acknowledge, starting with our California State University (Csu) colleagues Laura Guardino, Catherine Haras, Linda Ivey, Cherstin Lyon, Birte Pfleger, Yvette Saavedra, Carole Srole, and Wanda Washington. The Csu Course Redesign with Technology program, especially Kathy Fernandez, Brett Christie, and Gerry Hanley, sustained our faculty collaborations. Our focus-group leaders, Leslie Corona, Chiqutara Kendall, Brooks Moyer, and Emily Rivas, gathered excellent qualitative evidence. For advice on our survey design, we thank Erika Kato at the Center for Evaluation and Educational Effectiveness at Csu Long Beach and Mark Smith and Sam Wineburg at the Stanford History Education Group. For insightful readings and editorial work, we are grateful for Lendol Calder, Tim Herbert, Robert Johnston, Amy Kittelstrom, Leah Shopkow, and Laura Westhoff.
Footnotes
California State University (Csu) is part of a nationwide trend to improve graduation rates and eliminate equity gaps, especially in the thirty-five states that have performance-based funding formulas. See Paul Fain, “Collaborating on Completion,” Inside Higher Ed, Feb. 21, 2018, https://www.insidehighered.com/news/2018/02/21/public-universities-band-together-completion-rates-and-achievement-gaps.
The California State University 2018 Fact Book (Long Beach, 2018), 2, https://www2.calstate.edu/csu-system/about-the-csu/facts-about-the-csu/documents/facts2018.pdf; “Student Success at the California State University Reaches All-Time Highs,” California State University, https://www2.calstate.edu/csu-system/news/Pages/Student-Success-at-the-California-State-University-Reaches-All-time-Highs.aspx; “Graduation Initiative 2025,” ibid., https://www2.calstate.edu/csu-system/about-the-csu/budget/2017-18-support-budget/supplemental-documentation/Pages/graduation-initiative-2025.aspx; Jill Barshay and Sasha Aslanian, “Under a Watchful Eye,” Apm reports, Aug. 6, 2019, https://www.apmreports.org/story/2019/08/06/college-data-tracking-students-graduation.
Academic Affairs Committee, Academic Senate, California State University, “White Paper on Student Success,” May 2018, pp. 1, 3, California State University, https://www2.calstate.edu/csu-system/faculty-staff/academic-senate/Documents/reports/ASCSU_White_Paper_on_Student_Success.pdf.
Andrew K. Koch, “Many Thousands Failed: A Wakeup Call to History Educators,” May 2017, Perspectives on History, https://www.historians.org/publications-and-directories/perspectives-on-history/may-2017/many-thousands-failed-a-wakeup-call-to-history-educators.
On the data-driven decision making, see “2019–20 Operating Budget,” Nov. 14, 2018, p. 6, California State University, https://www2.calstate.edu/csu-system/about-the-csu/budget/2019-20-operating-budget/Documents/2019-20-Operating-Budget%20Book.pdf. Faculty involved in this study teach at distinctive Csu campuses representing regions as diverse as north San Diego County (San Marcos), the Inland Empire (San Bernardino), urban Los Angeles, the San Joaquin Valley (Fresno), the San Francisco Bay Area (East Bay), and Silicon Valley (San José). San Bernardino and San Marcos students contributed to our precourse and postcourse surveys only, while Los Angeles students participated only in the focus-group interviews. Faculty from all six campuses contributed deeply to the design and implementation of this research project.
The Asian category is especially broad, and the ethnic and national origins of Asian students differ by campus. For example, Fresno State University's Asian students tend to be Hmong American, who make up 6% of the student population. See Yoshiko Takahashi and Alex Nottbohm, “The Role of Institutional Data: A Focus on Hmong American Student Success,” Jan. 2019, p. 4, Fresno State University, http://www.fresnostate.edu/academics/documents/The%20Role%20of%20Institutional%20Data%20A%20Focus%20on%20Hmong%20American%20Student%20Success.pdf. Our course data resembles the Csu population as a whole. Across all Csu campuses, Hispanic/Latino students represent 41.6% of the student body; whites make up 22.6%; Asians 16.5%; African Americans 4.1%; and American Indians 0.2%. The broad Csu student population does not allow for mixed-race or mixed-identity classifications. “How Are Our Student Populations Changing?,” n.d., available through the Csu Student Success Dashboard. Csu limits access to dashboard data to users inside the university system. One-third of all Csu undergraduates are the first in their family to attend college. California State University 2018 Fact Book, 10.
Donald R. Gerth, The People's University: A History of the California State University (Berkeley, 2010). Though the Csu system does not track the status of students enrolled under Deferred Action for Childhood Arrivals (Daca), estimates suggest there are as many as 8,000 to 10,000 full-time Daca students across the 23 campuses. Jill Barshay, “Counting Daca students,” Hechinger Report, https://hechingerreport.org/counting-daca-students/. In 1987, state funding accounted for at least 70% of revenue received by Csu campuses. By 2012, that share had fallen to under 40% at most Csu campuses. “25 Years of Declining State Support for Public Colleges,” Chronicle of Higher Education, March 3, 2014, https://www.chronicle.com/interactives/statesupport.
Requirements in United States History, Constitution and American Ideals, Cal. Code Regs. tit. 5, sec. 40404. In 2019 a general education task force proposed eliminating the long-standing historical emphasis, sparking organized opposition from Csu historians and advocates for the humanities. For media coverage, see Colleen Flaherty, “Study of American Democracy on Cal State Chopping Block,” Inside Higher Ed, Feb. 27, 2019, https://www.insidehighered.com/news/2019/02/27/faculty-proposal-would-split-us-history-and-ideals-gen-ed-sequence-across-cal-state. On course bottlenecks, see “Mission of Course Redesign with Technology,” Course Redesign with Technology, http://courseredesign.csuprojects.org/wp/mission.
The Csu system defines underrepresented minorities as Black/African American, Hispanic/Latino, and American Indian. Nonunderrepresented minorities encompass everyone else, including students reporting two or more races or ethnicities in their backgrounds. “Redefining Historically Underserved Students in the Csu: Moving beyond Race and Economic Status to Close Equity Gaps,” n.d., p. 3, California State University, http://www.dashboard.csuprojects.org/rethinkingthegap/Historically-Underserved-Student-Factor-Model.pdf. At the Los Angeles campus, for example, a comparable political science course carried an average 0.35 grade point average (Gpa) gap, and Introduction to Psychology stood at 0.41 over the same period. Unfortunately, the dashboard does not let us explore multiple demographic variables together, so we cannot compare non-Urm first-generation students to Urm first-generation students. “Are There Equity Gaps?,” n.d., available through the Csu Student Success Dashboard.
“Which Early Academic Behaviors Help Most with Closing the Gap?,” n.d., available through the Csu Student Success Dashboard. On predictor analytics, see Joseph B. Treaster, “Will You Graduate? Ask Big Data,” New York Times, Feb. 2, 2017, https://nyti.ms/2jYBTAJ. We received institutional data from Csu campuses at East Bay and Fresno. It is worth noting that approximately half of the students who earn DFW grades in our history courses earned similar grades in one or more other classes in that same term. Koch, “Many Thousands Failed.”
On academic codes, see Gerald Graff, Clueless in Academe: How Schooling Obscures the Life of the Mind (New Haven, 2003).
In the Csu system, 49% of students receive Pell Grants. California State University 2018 Fact Book, 10. Rashida Crutchfield and Jennifer Maguire, “Study of Student Basic Needs,” Jan. 2018, California State University Basic Needs Initiative, https://www2.calstate.edu/impact-of-the-csu/student-success/basic-needs-initiative/Documents/BasicNeedsStudy_phaseII_withAccessibilityComments.pdf.
On cultural wealth in the classroom, see Tara J. Yosso, “Whose Culture Has Capital? A Critical Race Theory Discussion of Community Cultural Wealth,” Race Ethnicity and Education, 8 (March 2005), 69–91. Thanks to Cindy Mata at the University of California, Los Angeles, History and Geography Project for calling this work to our attention. College professors also have much to learn from K–12 education experts about inclusive teaching. See, for instance, “Access and Equity,” in History–Social Science Framework for California Public Schools: Kindergarten through Grade 12 (Sacramento, 2017), 509–46, https://www.cde.ca.gov/ci/hs/cf/documents/hssfwchapter20.pdf.
Particularly important works that influenced our team were Sam Wineburg, Historical Thinking and Other Unnatural Acts: Charting the Future of Teaching the Past (Philadelphia, 2001); Lendol Calder, “Uncoverage: Toward a Signature Pedagogy for the History Survey,” Journal of American History, 92 (March 2006), 1358–70; and Bruce VanSledright, Assessing Historical Thinking and Understanding: Innovative Designs for New Standards (New York, 2013). We later learned that our improvised method of collegial exchange and “empirical tinkering” enjoys its own place in educational research. See Anne K. Morris and James Hiebert, “Creating Shared Instructional Products: An Alternative Approach to Improving Teaching,” Educational Researcher, 40 (Jan.–Feb. 2011), 5–14.
On backward course design, see Tom Angelo, “Designing Subjects for Learning: Practical Research-Based Principles and Guidelines,” in University Teaching in Focus: A Learning-Centered Approach, ed. Lynne Hunt and Denise Chalmers (New York, 2013), 93–111. For our template for applying this design method to history classes, see the online appendix for this article, https://jah.oah.org/projects/beyond-big-data. Calder, “Uncoverage”; Joel M. Sipress and David J. Voelker, “The End of the History Survey Course: The Rise and Fall of the Coverage Model,” Journal of American History, 97 (March 2011), 1050–66.
Arlene Díaz et al., “The History Learning Project: A Department ‘Decodes’ Its Students,” Journal of American History, 94 (March 2008), 1211–24. On cognitive or epistemic shifts, see Liliana Maggioni, Bruce VanSledright, and Patricia A. Alexander, “Walking on the Borders: A Measure of Epistemic Cognition in History,” Journal of Experimental Education, 77 (Spring 2009), 187–213. Peter Lee and Denis Shemilt, “A Scaffold, Not a Cage: Progression and Progression Models in History,” Teaching History, 113 (Dec. 2003), 13–23; and Bruce VanSledright, The Challenge of Rethinking History Education: On Practices, Theories, and Policy (New York, 2010). On the difficulties in shifting even preservice teachers' views of history, see Kimberly Reddy, “Challenging Preservice Teacher Beliefs about the Past: The Influence of a Course Designed to Shift Ways of Knowing about History Teaching and Learning” (Ph.D. diss., University of Maryland, College Park, 2013). Kadriye Ercikan and Peter Seixas, “Issues in Designing Assessments of Historical Thinking,” Theory into Practice, 54 (no. 3, 2015), 255–62. See also Kadriye Ercikan and Peter Seixas, eds., New Directions in Assessing Historical Thinking (New York, 2015). For an excellent study of high school history teaching, see Abby Reisman, “Entering the Historical Problem Space: Whole-Class Text-Based Discussion in History Class,” Teachers College Record, 117 (Feb. 2015), 1–44. For a more general discussion of the importance of modeling and practicing key skills, see Susan A. Ambrose et al., How Learning Works: Seven Research-Based Principles for Smart Teaching (San Francisco, 2010), 10–39. On helping history majors learn the discipline, see Laura M. Westhoff, “Historiographic Mapping: Toward a Signature Pedagogy for the Methods Course,” Journal of American History, 98 (March 2012), 1114–26. Only 6% of our surveyed students identified as arts or humanities majors.
On the limits of lecture-based teaching compared to more “active” or “structured” courses, see Carl Wieman, Improving How Universities Teach Science: Lessons from the Science Education Initiative (Cambridge, Mass., 2017); Scott Freeman et al., “Active Learning Increases Student Performance in Science, Engineering, and Mathematics,” Proceedings of the National Academy of Science, 111 (June 2014), 8410–15; Louis Deslauriers, Ellen Schelew, and Carl Wieman, “Improved Learning in a Large-Enrollment Physics Class,” Science, 332 (May 2011), 862−64; and Katie E. Amaral et al., “Web-Enhanced General Chemistry Increases Student Completion Rates, Success, and Satisfaction,” Journal of Chemical Education, 90 (March 2013), 296–302. On the particular value of active learning for first-generation students and students of color, see Sarah L. Eddy and Kelly A. Hogan, “Getting under the Hood: How and for Whom Does Increasing Course Structure Work?,” CBE—Life Sciences Education, 13 (Fall 2014), 453–68. On benefits for women and students with low grade point averages, see David Gross, et al., “Increased Preclass Preparation Underlies Student Outcome Improvement in the Flipped Classroom,” ibid., 14 (Winter 2015), 1–8.
On the value of concise but frequent assessments for teaching historical thinking skills, see Sam Wineburg, Mark Smith, and Joel Breakstone, “What Is Learned in College History Classes?,” Journal of American History, 104 (March 2018), 984–91; and VanSledright, Assessing Historical Thinking and Understanding. Reddy, “Challenging Preservice Teacher Beliefs about the Past,” 20.
The classic work on bottlenecks in historical thinking is Wineburg, Historical Thinking and Other Unnatural Acts.
Reacting to the Past simulations provide structured games rooted in specific historical settings. See Mark C. Carnes, Minds on Fire: How Role-Immersion Games Transform College (Cambridge, Mass., 2014). On the artificial distinction between content and analysis, see Sam Wineburg and Jack Schneider, “Was Bloom's Taxonomy Pointed in the Wrong Direction?,” Phi Delta Kappan, 91 (Dec. 2009–Jan. 2010), 56–61.
Ji Son and James Stigler, “Practicing Connections: A New Design Framework for Developing Courseware in Rich Domains,” Proceedings of E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (Las Vegas, 2018), 862–67. Bruce A. Lesh, “Why Won't You Just Tell Us the Answer?”: Teaching Historical Thinking in Grades 7–12 (Portland, Maine, 2011). Carole Srole, “Scaffolding Research Skills in a Nonresearch Class,” Perspectives on History, 36 (Jan. 1998), https://www.historians.org/publications-and-directories/perspectives-on-history/january-1998/scaffolding-research-skills-in-a-nonresearch-class. On scaffolding research skills themselves, we largely follow the recommendations of the American Historical Association project, Tuning the History Discipline in the United States. See “Aha History Tuning Project: 2016 History Discipline Core,” American Historical Association, https://www.historians.org/teaching-and-learning/tuning-the-history-discipline/2016-history-discipline-core. For sample activities, see the online appendix for this article, https://jah.oah.org/projects/beyond-big-data.
In a study of Csu students who took team-based learning versions of the U.S. survey course, students singled out group work as the most “useful or enjoyable” feature of the course. See Carole Srole, Christopher Endy, and Birte Pfleger, “Active Learning in History Survey Courses: The Value of In-Class Peer Mentoring,” History Teacher, 51 (Dec. 2017), 89–102. On the peer-mentoring program for introductory U.S. history at the Los Angeles campus, see ibid. Scholarship on laptops in the classroom offers no consensus, to date, on best practices. Worse, these studies usually equate learning with lecture notes and short-term recall tests. More useful for us would be studies exploring how laptops affect student interactions with each other during team activities, or if digital screens affect how students read primary sources. On lecture notes, contrast Pam A. Mueller and Daniel M. Oppenheimer, “The Pen Is Mightier than the Keyboard: Advantages of Longhand over Laptop Note Taking,” Psychological Science, 25 (April 2014), 1159–68; and Kayla Morehead, John Dunlosky, and Katherine A. Rawson, “How Much Mightier Is the Pen than the Keyboard for Note-Taking? A Replication and Extension of Mueller and Oppenheimer (2014),” Educational Psychology Review, 111 (Feb. 2019), 1–28.
For examples and discussion of in-class group activities, see the online appendix for this article, https://jah.oah.org/projects/beyond-big-data. Wineburg, Historical Thinking and Other Unnatural Acts.
Sipress and Voelker, “End of the History Survey Course.” Gary J. Kornblith and Carol Lasser, “‘Pivoting the Center’”: Diverse Surveys of American History,” Journal of American History, 93 (March 2007), 1163–64.
For important warnings against simplistic presentism, see David Lowenthal, “The Timeless Past: Some Anglo-American Historical Preconceptions,” Journal of American History, 75 (March 1989), 1263–80; Wineburg, Historical Thinking and Other Unnatural Acts; Reisman, “Entering the Historical Problem Space”; and Laura M. Westhoff, “Lost in Translation: The Use of Primary Sources in Teaching History,” in The Teaching American History Project: Lessons for History Educators and Historians, ed. Rachel G. Ragland and Kelly A. Woestman (New York, 2009), 62–77. For advice on connecting past and present, see Keith C. Barton and Linda S. Levstick, Teaching History for the Common Good (Mahwah, 2004), 69–90. Many Csu campuses have strong and pioneering ethnic studies programs that engage with current affairs, and it is possible that this context shapes Csu students' expectations for history courses.
Our diverse Csu classrooms point to the need for more research on history students' race and ethnicity. In Indiana University's History Learning Project, for example, researchers describe classrooms that appear to be racially homogeneous. They worry about students (presumed to be white) “withdrawing from the intellectual process” when “faced with a stark moral choice between their families and the stories told by a history professor.” Faculty at Indiana University teaching Latinx history found that their students could “engage with the history of the dominant Anglo culture,” but only when they were able “to separate themselves from moral guilt about actions of the past.” Neither argument allows for the kinds of classrooms we teach in, where white students are a statistical minority. Our students, with diverse family histories and less personal investment in a glorifying narrative of white America, might also encounter different obstacles to engaged learning. Díaz et al., “History Learning Project,” 1216.
In one of our in-class student assessments for this project, a professor asked students to explain the relevance of the Gettysburg Address today. Approximately half of the students still provided vague statements. For similar frustrations, see Nancy Quam-Wickham, “Reimagining the Introductory U.S. History Course,” History Teacher, 49 (Aug. 2016), 528–31. John Winthrop, A Model of Christian Charity (New York, 1830).
National Task Force on Civic Learning and Democratic Engagement, A Crucible Moment: College Learning and Democracy's Future (Washington, 2012). Vincent Tinto, “Classrooms as Communities: Exploring the Educational Character of Student Persistence,” Journal of Higher Education, 68 (Nov.–Dec. 1997), 599–623. Unlike our Csu student body, the history faculty involved in this research are mostly white, with college-educated parents. These differences matter. A recent study shows that white faculty and administrators in the Csu system often imagine their students in a state of persistent crisis. White educators frequently assume the role of antiracist “benefactors” while failing to interrogate their own positions of racial privilege. See Demerris Immel-Brooks and Susan B. Murray, “Color-Blind Contradictions and Black/White Binaries: White Academics Upholding Whiteness,” Humboldt Journal of Social Relations, 39 (May 2017), 315–33, esp. 324–26.
For national averages, see Koch, “Many Thousands Failed.” These results resembled those of Csu historians involved in the Csu Course Redesign with Technology program in 2015–2016. Fourteen of the 19 sections of introductory U.S. history taught after the redesign reported a decline in DFW grades, with five sections showing no change. Each of the 19 faculty members who redesigned a U.S. history survey course created an “e-Portfolio” of results. See “e-Portfolio Showcase,” Course Redesign with Technology, http://courseredesign.csuprojects.org/wp/eportfolios#history. This research received support from the Csu Chancellor's Office Course Redesign with Technology program. Although this program was envisioned as one for individual faculty, administrators supported our efforts to work as a team and devise a larger-scale survey across multiple campuses. Our survey was based on a survey of student attitudes developed by the Los Angeles campus historians Christopher Endy, Carole Srole, and Birte Pfleger. We refined the questions and survey response options with assistance from Erika Kato at the Center for Evaluation and Educational Effectiveness at Csu, Long Beach, and Mark Smith at the Stanford History Education Group. Each faculty member obtained institutional review board permissions from their campus to conduct research on student subjects, including matching survey responses to grades. For the full precourse and postcourse survey instruments, see the the online appendix for this article, https://jah.oah.org/projects/beyond-big-data.
We define measurable change as a one-step shift on a Likert scale of agreement or disagreement with survey questions. For example, a one-step shift takes place when a student changes a response from “Disagree” to “Neutral” or from “Strongly Disagree” to “Disagree” between the precourse and postcourse surveys.
W. K. Adams et al., “New Instrument for Measuring Student Beliefs about Physics and Learning Physics: The Colorado Learning Attitudes about Science Survey,” Physical Review Special Topics—Physics Education Research, 2 (Jan. 2006), 1–14. For a more detailed explanation of the “expert shift” method, see online appendix for this article, https://jah.oah.org/projects/beyond-big-data.
First we examined lower-order relationships among variables that had the potential to affect grades. Then we created more complex models to take all the variables that could predict grades into account at once (multiple regression). Next, we performed a stepwise multiple regression analysis of this data set to control for demographic variables to learn what other factors might be significant in predicting student performance in our classes. In a stepwise regression, we enter blocks of conceptually similar variables in a meaningful order, each subsequent step controlling for all the variables in prior steps. So, the simplest model would attempt to predict variation in grades using only demographic variables alone. Next, we would examine any further benefit of predicting variation using students' subjective concerns, controlling for demographic variables. The next step would examine the role of changes in thinking, controlling for demographics and concerns. Finally, we can see the effect of postcourse thinking, controlling for demographics, concerns, and changes in thinking. For a complete description of the analysis and data, please see the online appendix for this article, https://jah.oah.org/projects/beyond-big-data.
A limit of our study is that we did not have enough students in certain racial/ethnic groups to run meaningful statistical analyses. To keep these students in our analysis, we had to cluster them into one umbrella category that we labeled “small-sample-size groups.” This category included students who were less than 5% of our sample pool: Black/African American, American Indian, Alaska Native, Native Hawaiian, other Pacific Islander, “other,” and “decline to state.” As with our small-sample-size group, the “Asian” demographic category includes members of fairly different ethnic minorities with different levels of privilege. Our finding that gender, generational status, and language were not significant predictors contrasts with the John N. Gardner Institute for Excellence in Undergraduate Education survey of introductory U.S. history courses, which found that male students, first-generation students, and Pell Grant recipients had a higher average rate of repeatable grades. Our study did not find this relationship but used a different kind of analysis with a smaller sample size. As we had access to course grades, our analysis considered the impact of multiple variables on quantitative (A, B, C) grade measures rather than a single variable on a categorical (pass/fail) measure. Koch, “Many Thousands Failed.” For a full explanation of the stepwise multiple regression method and results, see the online appendix for this article, https://jah.oah.org/projects/beyond-big-data.
The precourse and postcourse surveys of students contained qualitative questions that students answered with short comments. A total of 374 students completed both surveys, though not every student wrote comments. For a list of the questions, see the online appendix for this article, https://jah.oah.org/projects/beyond-big-data. To collect more in-depth responses, we used semistructured focus groups to interview twelve students across four campuses. Graduate student moderators used a protocol of questions designed to draw out their experiences in the course. For the protocol, see ibid. Though the sample is small, the interviews provide an invaluable source of information on how some students shifted their mind-set toward history and were analyzed as such. John Lofland, Analyzing Social Settings: A Guide to Qualitative Observation and Analysis (Belmont, 2006).
Requirements in United States History, Constitution and American Ideals, Cal. Code Regs. tit. 5, sec. 40404.
Immel-Brooks and Murray, “Color-Blind Contradictions and Black/White Binaries.”
Anne Hyde, “Five Reasons History Professors Suck at Assessment,” Journal of American History, 102 (March 2016), 1104–7. Online appendix for this article, https://jah.oah.org/projects/beyond-big-data.
