Stalled at the Gate: Addressing Student Failure in a “Gateway” Course

Higher education is increasingly data-driven. Institutions now collect and use vast amounts of data about student characteristics and behavior to manage recruiting and enrollment, schedule facilities and human resources, organize degree and course offerings, and track student completion. Until recently, such data have been utilized largely to guide institutional or program-level decisions. For example, the Center for Community College Student Engagement’s massive data sets, drawn from tens of thousands of students across the country, have been used by a number of institutions to implement a variety of student engagement and retention programs, including accelerated and self-paced developmental courses, first-year experiences and learning communities, and structured academic pathways. Utilizing student data at the classroom level to design and teach individual courses is not yet a common practice, although that is beginning to change (Stewart, 2017), with projects such as the Civitas Explore platform and Instructure’s Student Insight Engine, both of which are now being piloted at USU. In trying to understand the failure-to-complete problem in HIST 1700, I had access to very limited data. But the data I did have was central to gaining insight into student performance and to imagining new pedagogical strategies for success.Failure-to-complete rates in “gateway” courses are a serious concern in open-enrollment institutions like the former College of Eastern Utah, where many students arrive with low college readiness skills.^[2] In 2010, 22% of all grades at CEU were D or F grades. At Utah State University, only 12% of grades in undergraduate courses fell in this range. Remediation, as most higher education scholars agree, can be problematic. It is expensive for institutions to provide college preparatory courses, which are also costly for students, who find themselves forced to spend time and money on classes that do not produce credits toward graduation. The problem was particularly pronounced at CEU because high D-F rates in courses such as MATH 1030 (42%), ENGL 1010 (22%), and HIST 1700 (26%) contrasted sharply with exceedingly high grades in almost all other courses. In 17% of the courses offered at CEU from 2008-2010, all students received a grade of an A; in 42% of the courses, more than half of the final grades were As.^[3] The impact of this grade disparity between required “gateway” courses and other classes on student expectations and satisfaction is unknown; but at an institution facing a dramatic decline in enrollment (as CEU was in 2010) a serious look at grade distribution and failure-to-complete rates seemed warranted.It was in this context that I set out to determine why so many students were not succeeding in HIST 1700. At my request, the CEU administration provided data on the 495 students who had taken the sections of 1700 I taught between 2008 and 2010. The provided data included overall GPAs for each student at the time they took HIST 1700, their ACT scores (when available^[4]), and their grades in ENGL 1010 and whichever math course they had taken. From my own course grade books, I compiled assignment completion rates and assessment scores for all students as they progressed through the 16-week course. In addition to examining these data, I conducted CLOZE reading tests in the spring 2010 sections of the course (82 students took the test) and ran a Flesch-Kincaid analysis on 148 student essays. Although I had no data on the age, race, class, or ethnicity of the students enrolled in HIST 1700, casual observation suggested that they were representative of CEU’s student body, which was (and is still today) overwhelmingly white and young. In 2010, 82% of CEUs students enrolled at the Price campus were white.^[5] Unlike many open-enrollment community colleges, CEU enrolled mostly traditional-age students, with 75% being under the age of 25 and a significant number (through Utah’s concurrent enrollment program) being under the age of 18.^[6] By contrast, 17% of CEU’s students were over the age of 30, which meant that each section of HIST 1700 had a few nontraditional students intermixed with a large group of very young students. Thirty-five percent of the students at CEU had Pell Grants, suggesting that a significant number of students in HIST 1700 came from economically distressed homes. Fifty-two percent of the students in HIST 1700 were women, a reflection of the fact that CEU was one of only two USHE institutions that enrolled more women than men.^[7]

The students enrolled in HIST 1700 had basic college skill levels consistent with expectations at an open-enrollment community college. The average ACT composite score for students at CEU in 2009 was 20, and this was true for the students in HIST 1700. The average reading score was 21. The ACT’s 2009 College Readiness Benchmark score for predicting a 75% chance of earning a grade of C or higher in college social science courses was 21 on the reading test. Sixty-three percent of the students enrolled in HIST 1700 met or exceeded this benchmark. This would suggest a reasonable expectation that a significant number of students enrolled in HIST 1700 should have been able to pass, and indeed they did. But success in the course did not clearly correlate with ACT scores. Students who received an A in the course all had ACT scores at or above the readiness benchmark of 21. But some students whose ACT scores met the benchmark failed the class, and others who did not meet the benchmark received B and C grades. Although ACT scores were not predictive of individual student success, I found it valuable to know that the students in my course generally had modest college-level skills –particularly in reading—and that a significant number (36%) scored well below what the ACT identified as college ready, having reading scores that ranged from 12 to 19.This fact was supported by the CLOZE tests I administered to 82 students during the spring of 2010. The results of these tests indicated that students in HIST 1700 had an average eighth-grade reading level, with a number of them ranking at the sixth-grade level, and none scoring above the 12th grade level. On the Flesch-Kincaid scale, the written work submitted by students was also, on average, at the eighth-grade level. Only one essay reached the 12th grade level, while 18 were written at or below the sixth-grade level. The textbook used in the course was written at the 12th grade level on the Flesch-Kincaid scale, which is standard for commercial, college-level history textbooks. These results helped explain the anecdotal evidence I had from conservations with students and course evaluations that many students were struggling with the textbook. My experience through nearly 20 years of teaching the U. S. history survey is that students rarely “like” a textbook. Clearly, the problem in HIST 1700 went beyond the usual student displeasure with assigned reading.

A disconnect between the textbook and student skill levels may have been one factor in the high failure rate in the course. But data on overall GPAs for the students in HIST 1700 and on their success rates in other “gateway” courses suggested that I needed to take a deeper look. The average college GPA for students at the time they took HIST 1700 was 2.68. A breakdown of GPAs clearly correlated with success in the course—those students who received an A in the course had, on average, much higher GPAs (3.5) than those who failed (average GPA of 1.7). Of the students who received failing grades in HIST 1700, 13% also failed ENGL 1010, and 29% failed MATH 1030 or MATH 1050.^[8] All of this seemed to indicate there was a cohort of poorly performing students, whose failure extended beyond the specific conditions or content of HIST 1700.

Did these failing students have any identifiable characteristics or behaviors that might help explain their failure to thrive? The data available to me limited any conclusions about class, age, or race beyond the broad ones mentioned above, but my course grade books revealed two important characteristics. First, a majority (55%) of the students who failed HIST 1700 were men. Conversely, women accounted for 60% of the A grades in the course. This seems consistent with recent scholarship on the declining success of men in college (Levin, 2006; Lopez & Gonzalez-Barrera; 2015; Nolan, 2015; Voyer, 2014).

The second distinguishing characteristic of those who failed HIST 1700 was the lack of what I call “work discipline.” Students who failed did so because they simply did not complete the required coursework. Students were required to complete a variety of tasks in the course, including a group project with a classroom presentation, short weekly research and writing projects, exams, and a final essay paper. There was a total of 22 graded activities divided across four course units. Students earning an A in the course consistently completed 98%-100% of the graded tasks during each unit. Students earning an F completed, on average, fewer than half of the graded tasks, with the completion rate declining during each unit. Failing students completed 73% of the tasks during the first unit, but only 42% during the final unit. No student who completed all 22 tasks failed the course. Among the students who failed the course, the average grade for the work they did submit was a D+ (D- on exams and C on assignments). If these students had completed a greater percentage of the course requirements, many would have been able to pass.

On the basis of the analysis described above, I began to consider what possible changes to the course’s content, structure, and pedagogy might help improve student success. Two issues in particular seemed most amenable to remediation: the lack of student engagement and the disparity between reading skill and the level of instructional materials. To address the issue of instructional materials, I decided to supplement the textbook with a set of additional tools designed to assist students in accessing and digesting the material. I rejected the idea of adopting a textbook written at a lower reading level because those texts, designed for the middle-school market, did not have content suitable for a college-level course.^[9] I prepared a series of illustrated study guides for each chapter of the textbook that included a graphical outline of the content and highlighted key vocabulary, names, and events.One advantage of these study guides was that individual students could utilize them according to their own needs. Some students might rely on the study guides heavily, while others might not need them at all. Separately, some students might find the guides most helpful early in the semester, but as the semester progressed, might develop the ability to read chapters without as much support from the guides. All study guides were posted on a course website so students could access them at any time.^[10] I took the further step of editing the publisher-provided test bank to rephrase question language and vocabulary to a reading level that seemed more accessible. I did the same for all of my lectures and course instructional materials, giving particular attention to providing more time in my classroom presentations to define vocabulary, as well as to explain the meanings of words and concepts that I now realized were not as commonly understood as I had once assumed.

Finally, I added a simple statement in my course introduction explaining that students should expect to encounter new words and concepts over the semester. I told students that the textbook is a tough read, but that there are tools to help. I also encouraged them that persistence and patience will pay off. In Student Engagement Techniques, Elizabeth Barkley (2009) notes that student failure is most acute when there is a disconnect between expectations and skill levels. By adjusting expectations to a realistic level, along with reassurances that help is available at any time they feel the need for it, I believe students of reading level are better prepared to use the textbook.

Addressing the issue of student engagement proved more difficult than adopting strategies to help students bridge the gap between reading capacities and instructional materials. The failure of so many students to do the work required in the course seemed likely due, in part, to factors beyond the content and organization of this specific class. Educational scholars and boots-on-the-ground instructors are all too aware that a myriad of circumstances, from personal psychology to socio-cultural conditions, influence the performance of college students. As instructors, we all appreciate the need to address individual student problems humanely and realistically. But a 26% failure-to-complete rate struck me as requiring more than the usual complement of flexible due dates, make-up assignments, and extra credit projects. Were there any changes to the course content and instructional approaches that could help improve student engagement?

A detailed discussion of the various pedagogical changes I considered are beyond the scope of this brief essay, but the two principal ones that I adopted were a switch in the thematic focus of the course and replacing the weekly and groups assignments with three active-learning projects. For most general education courses, there is ongoing discussion within professional disciplines about how best to teach basic surveys. This is true in history. Some historians prefer a broad chronological overview, providing a brief look at the main events in the long expanse of U. S. history. In recent years, many historians have adopted a thematic approach, choosing to focus on specific issues, such as social history. This choice is for topical depth, rather than comprehensive chronological coverage. I decided to reorganize HIST 1700 from a chronological approach to a topical one, focusing the entire course on the changing concept of freedom through four key episodes from the eighteenth through twentieth centuries.^[11] A strong theme that is as relevant today as in the past, coupled with an in-depth look at several of the most dramatic eras in American history, seemed more likely to capture the attention of students than 16 weeks of “this happened, and then this happened.” This choice is, frankly, a trade-off—much that is important in U. S. history goes unmentioned in the course, but the hope is that what is covered is compelling and will better sustain student interest.

The second pedagogical change I made was to introduce two role-playing activities and a multimedia research and essay project. In one assignment, for example, students were tasked to be either a Northerner or Southerner during the Civil War and to compose a series of imaginary letters describing their experiences. In another, students took a virtual tour of New York City during the Gilded Age and wrote an essay describing what immigrants might have seen when they arrived in the city. These assignments replaced the short weekly essays and group projects previously required, thus significantly reducing the number of graded assignments. These three assignments were organized around separate websites, which provide students all the needed information, documents, and instructions. The websites were designed to be visually stimulating and to allow students to interact with materials according to their own schedules. These assignments were made available to students at the beginning of the semester. Although there were specified due dates for each assignment, students had many weeks in which to prepare and complete the projects.

As I began to implement these changes, an even larger transformation took place. In July of 2010, Utah State University took over the College of Eastern Utah. The institution, renamed Utah State University Eastern, continued to operate as an open-enrollment institution, primarily offering associates degrees and technical certificates, but integrating its curriculum with USU’s main and regional campus system. USU Eastern no longer offered HIST 1700, replacing it with USU 1300. As the sole instructor for USU 1300 at USU Eastern, I taught the course exactly like the former HIST 1700, using the same curriculum and textbook, but with the changes in assignments and study guides I devised following the analysis described above. In the years since the merger, 351 students have taken my sections of USU 1300. It is possible, therefore, to make some assessment of the impact my analysis and reorganization has had.

The data show a very different grade curve from the previous, pre-merger record.

The biggest change has been a shift of the curve from C to B. The percentage of students receiving an A in the course has remained the same (13.3% and 13.6%). Significantly, the percentage of students failing the course has declined from 23% to 16%; the decline in D grades has been more modest, from 10% to 8%. This is heartening, but it is worth noting that failure is still directly related to a lack of engagement: as before, no student who completed all the requirements failed the course. My changes seem to have been most helpful in slightly improving the grade of C students. This improvement is largely the result of scores on the active-learning assignments. Exam grades have not significantly improved (the average remains a C, increasing from 72% to 76%), despite the addition of the textbook study guides.

The shift in the grade curve for HIST 1700/USU 1300 needs to be placed in the context of a development that had nothing to do with the changes to the course that I implemented. In the years since Utah State assumed control of the old College of Eastern Utah, enrollments have continued to decline.^[12] As a reflection of this decline in the USU Eastern student population, enrollment in my sections of USU 1300 has also dropped. Prior to the merger, enrollment in HIST 1700 averaged 99 students per semester. Post-merger enrollment in USU 1300 has averaged only 41 USU Eastern students per semester. However, since the merger, the course has been open to students throughout the USU system via IVC, and these non-USU Eastern students now account for 18% of the course enrollment. I do not have access to the same kind of data that I did for the pre-merger students, but USHE data for the general USU population suggests that these students are likely to have higher ACT scores. At this time, there appears to be no significant difference in the performance of the two student populations in USU 1300, but the sample size is too small to draw any significant conclusions.

As of this writing, the USU history department has decided to reinstate HIST 1700. Beginning with fall semester 2017, USU 1300 will no longer be offered at USU Eastern. My HIST 1700 course format will remain much the same: a thematic approach, four-unit structure with assessment based on active-learning projects, final essay, and four exams. One significant change will be the introduction of a new format for the textbook. Although the course will use the same textbook as in the past, it will now be available to students in digital format through Canvas. Inclusive Access, as this format is termed, makes the textbook available as individual chapters (the instructor can select which chapters to provide in whatever order) from the first day of the semester. Students pay for access to the textbook as a course fee at the time of registration (the fee is returned if a student drops the course by the university’s established class drop schedule). The cost to students is significantly lower than a print version. Coupling access to the textbook with registration ensures that all students have the textbook from the beginning of the semester. In addition, Inclusive Access comes with a publisher-provided online self-quizzing tool called InQuisitive and online chapter guides. These tools, which are much more sophisticated and interactive, will replace the ones I designed.With all the required course materials, including the textbook and active-learning assignments, now available through Canvas, I will have access to a wider range of data on student engagement in the course. I am now also using a cloud-based polling app (REEF from iClicker) that should provide additional data on student attendance and participation. Although I can offer no clear or conclusive data to demonstrate that the changes I implemented resulted in improved student success, I am convinced that the exercise has been worthwhile. Utilizing data to diagnose specific problems in student performance has greatly enhanced my understanding of the challenges I face as an instructor, and I believe I am better equipped to meet those challenges. As I continue to tinker with the course over the coming semesters, I look forward to applying new sources of data to enhance my teaching.