Pairing a bioinformatics-focused course-based undergraduate research experience with specifications grading in an introductory biology classroom

Abstract

Introducing bioinformatics-focused concepts and skills in a biology classroom is difficult, especially in introductory biology classrooms. Course-based Undergraduate Research Experiences (CUREs) facilitate this process, introducing genomics and bioinformatics through authentic research experiences, but the many learning objectives needed in scientific research and communication, foundational biology concepts, and bioinformatics-focused concepts and skills can make the process challenging. Here, the pairing of specifications grading with a bioinformatics-focused CURE developed by the Genomics Education Partnership is described. The study examines how the course structure with specifications grading facilitated scaffolding of writing assignments, group work, and metacognitive activities; and describes the synergies between CUREs and specifications grading. CUREs require mastery of related concepts and skills for working through the research process, utilize common research practices of revision and iteration, and encourage a growth mindset to learning—all of which are heavily incentivized in assessment practices focused on specifications grading.

Introduction

Advancements in the late 20th century in genomic sequencing and computing power substantially increased the amount of DNA and other biological data available for study, making bioinformatic approaches vital to biological research. Despite the importance of bioinformatics, the inclusion of related concepts and skills in the undergraduate curriculum has been slow [1]. Challenges to updating undergraduate biology curricula are numerous, with barriers in curricular expectations, faculty time and expertise, and availability of beginner-friendly tools [2–5]. Big data fields are changing rapidly each year, and highly diverse and technical methodologies exist that are difficult to quickly introduce and retain in the classroom as frequent updates are made to software and data formats. In addition, there is little uniformity or shared learning objectives for training students in this field, though there are growing efforts toward standardization [2, 3, 6, 7]. The inclusion of computing tools in a biology classroom is often intimidating or uninteresting to undergraduate biology students, who typically lack previous exposure to bioinformatics and do not expect to encounter computational tools in a biology classroom [2, 5].

When courses including bioinformatic-focused content are designed, however, there are clear benefits. Students gain the ability to evaluate the many cutting-edge studies using bioinformatics, they develop skills that are directly transferable to many biology-related careers post-graduation, and they better understand how biological research is done in the 21st century. In the last decade, there has been a gradual shift toward a more bioinformatics-focused education for undergraduate biology students. This shift initially began in upper-level elective courses, but increasingly, a bioinformatics approach has been successfully implemented at the introductory level as well [4]. Teaching bioinformatics at the introductory level has many advantages, such as emphasizing the essential nature of bioinformatics in modern fields of biology through its inclusion in the core curriculum. It also provides biology students early exposure to the field, giving those interested in further study more opportunities to seek out and acquire vital skills in related disciplines such as computing and data analytics.

Participation in undergraduate research experiences is shown to be highly correlated to increases in scientific thinking, success in science, technology, engineering, and math (STEM) courses, and interest in pursuing scientific careers [8]. Thus, much funding, time, and resources have been dedicated to supporting undergraduate research [9]. Course-based Undergraduate Research Experiences (CUREs) are undergraduate research experiences designed for the classroom to increase opportunities for students to take part in the research process. Fundamental elements of CUREs include an investigation into a research question of benefit to the scientific community and other stakeholders and practicing iteration that reflects real-life research processes among scientists. Studies have shown that CUREs have real benefits for students, with those participating showing improved odds of earning a STEM degree and continuing in a STEM career, especially if they participated early in their undergraduate career [8, 10].

Past implementation of bioinformatics-focused curriculum has often gone hand in hand with CUREs. Several CUREs described in the literature focus on genomics, including both individual CUREs developed within an institution [11–14], and nationwide collaborations across faculty in multiple institutions, e.g. SEA-PHAGES, GEP, IMG-ACT [15–19]. The Genomics Education Partnership (GEP) is one example of such a CURE [17]. GEP central infrastructure has supported members developing bioinformatics-focused CUREs that are low-cost, accessible, and apply foundational biological principles, particularly the central dogma of molecular biology [18]. Students perform gene annotation using genomic data such as publicly available genome assemblies and RNA-Seq data [18, 20], and online software such as BLAST and the UCSC Genome Browser—two widely used bioinformatic tools [21, 22]. Student gene annotations are published in databases and publications [20] for use in large-scale comparative genomic research. Implementing these research projects successfully at the introductory level is still difficult, however, due to the challenges of teaching fundamental biological concepts, bioinformatic skills, and the research process simultaneously.

Another movement in student learning highlights the potential of alternative assessment systems to shift student focus from grades to feedback. Several grading systems exist—e.g. contract grading, ungrading, and specifications grading—and common elements across all systems include removing grades from most or all assignments, creating opportunities for revision based on feedback, and amplifying student choice in the grade to which they aim [23–26]. For example, in specifications grading, students complete bundles of assignments that correspond to different final letter grades. Completion of assignments is based on pre-determined standards, and student revision to meet standards based on instructor feedback is a core part of the system. Teachers who have implemented one of these grading systems have reported increased student ownership of learning, including deep student engagement with instructor feedback. They have also noted a decrease in student anxiety associated with removing numerical scores and allowing many opportunities for revision [27].

Here, the implementation of specifications grading with a CURE is proposed to facilitate student learning in genomics and bioinformatics due to positive synergy from shared values in CUREs and specifications grading. Below, an example of the co-implementation of a bioinformatics-focused CURE and specifications grading in an introductory biology classroom is shared. First, the introductory biology course and how a bioinformatics-focused CURE associated with the GEP is implemented are described. Then, the specifications grading structure is described, including details on how scaffolded writing assessments, group work, and metacognitive assignments were integrated into this framework. Next, the synergy that exists between implementing a CURE and specifications grading is described, focusing particularly on the shared themes of mastery, revision, iteration, and a growth mindset. Finally, student experience in the course is examined, through describing student accomplishments and the results of a post-course survey asking students to reflect on their experience in this course.

Big data in biology course structure and development

BIOL 199: Introduction to Biological Thinking is the first of three courses in the biology core curriculum at the University of Richmond; it is composed of multiple sections focused on student-directed investigation in an area of biology of the instructor’s choice. The main learning objective across all sections is to develop student skills in scientific research and communication early in their undergraduate career. The course satisfies the “Natural Science” general education requirement and is necessary for eventually taking upper-level biology courses. The flexibility in course design for BIOL 199 created an opportunity to workshop a course focused on genomics and bioinformatics at the introductory biology level. In Fall 2021, the first iteration of this course was developed as a section of BIOL 199 titled Big Data in Biology (BDB), where students learned fundamental biological concepts and skills through the lens of genomics and bioinformatics. As part of this course, students participated in a GEP CURE focused on gene annotation. The course was taught two additional times, in Spring 2022 and Fall 2022, with a class size of 14–15 students per semester. Students were first- or second-year undergraduates with diverse backgrounds and levels of interest in biology. With no prerequisites, students ranged from those who had taken AP/IB Biology to those who had not taken Biology since middle school.

The course met twice a week for 75 min of class and once a week for 170 min of a computational lab. Shared learning objectives across all sections of BIOL 199 focus on training students in the process of performing biological investigation (Table 1). This includes (1) best practices for effective investigation, such as developing hypotheses, carrying out experiments/analyses, and interpreting results; (2) practices for clear and honest scientific communication, such as reviewing previous literature, writing clear and concise narratives on scientific findings, and presenting scientific findings to an audience; and (3) practices for ethical research, such as accurately representing research methods and findings, appropriately referencing source material, and acknowledging other assistance. Along with these shared objectives, the goal of the BDB section was to introduce students to concepts, tools, and questions in the field of genomics and bioinformatics, such as how genomic sequencing works, the history and availability of online databases, common bioinformatic tools for comparative genomics, and the utility of remote computing and Linux (Table 1, bolded). Fall 2022 course materials are available at https://github.com/MYangLab/BDB_F22.

Table 1.

Major learning objectives in BIOL 199: Big Data in Biology (BDB) and associated sub-objectives

Major learning objective	Associated sub-objectives
A. Appreciate the impact of big data in biology and understand how the rise of big data in biology is deeply intertwined with foundational evolutionary concepts	1. Explain how diversity of life evolved over time by processes of mutation, selection, gene flow, and random processes
	2. Recognize when evolutionary concepts underlie biological research (e.g. explaining why studying one species can be informative about another species)
	3. Explain the difference between homology and similarity
	4. Develop example of an evolutionary process
	5. Interpret phylogenetic trees using an evolutionary model
B. Understand how comparative approaches combined with large datasets can uncover natural history of living beings	1. Communicate relevant content related to specified question or problem and be able to discuss broader implications of question or problem
	2. Analyze data related to a specified question or problem, particularly using online databases, shell scripting, and popular computational tools
	3. Relate concepts from multiple biological fields to interpret biological phenomena related to specified question or problem (e.g. molecular, computational, organismal)
	4. Propose potential solutions for specified question or problem, both through developing hypotheses and interpreting results
C. Understand how modern biologists derive important and answerable questions and design and execute experiments	1. Define the hallmarks of good experiments and/or studies that address specific hypotheses
	2. Use observational and organizational skills to record and annotate experimental outcomes and/or observations of the natural world
	3. Understand, execute, and troubleshoot protocols required for data collection and analysis
	4. Make measurements and use quantitative methods to evaluate results
	5. Recognize the power and limitations of the scientific process—describe how data is not inherently objective, and they are subject to error, misinterpretation, and bias
D. Learn how to interpret, summarize, and evaluate scientific research, both your own and that of others	1. Create and interpret informative graphs and other data visualizations
	2. Communicate experimental results in a concise and clear manner, both in written and oral formats
	3. Use online databases and data science tools to find information (e.g. previously published research) relevant to biological questions
	4. Distinguish between different types of information sources and develop skills for searching for primary literature
	5. Demonstrate conventional and proper use of sources in scientific writing to acknowledge intellectual contributions from others
E. Understand what it means to be an ethical scientist and why this is important	1. Identify and employ the characteristics of research integrity in experimental design, data collection, analysis, and communication
	2. Describe why accurate acknowledgement of intellectual contributions from others is important
	3. Describe the ethos of open access in science and its impact on the field of biology, particularly bioinformatics

Objectives specific to the BDB section of BIOL 199 are bolded. All other objectives are expected across BIOL 199 sections.

To engage students in genomics research, a bioinformatics-focused CURE through the GEP—the Pathways Project [20]—was implemented. This project uses web-based bioinformatic tools to annotate genes in the insulin signaling pathway of non-model Drosophila species, which later used by the GEP for a large-scale comparative genomics project examining the relationship between the rate of evolutionary change in proteins and the role of these proteins within signaling pathways. Curriculum related to the Pathways Project is available through the GEP on their website (https://thegep.org/). Prior to developing the BDB course, the instructor trained in the use of related GEP curriculum in the Summer 2020 and spent 1 year workshopping the material with five undergraduate research students. Two of those research students were hired as teaching assistants (TAs) for the first two iterations of this course, and no TA was hired for the third iteration of the course.

Structuring lab modules to effectively engage students in the GEP pathways CURE

Much of the course design was focused on how to introduce content so that students obtained the requisite concepts and skills needed to complete the GEP Pathways Project. Required content included the process of DNA sequencing, the central dogma of molecular biology, and evolutionary concepts for understanding comparative genomics. Required skills included effective use and interpretation of results from BLAST and the UCSC Genome Browser [21, 22].

The first three weeks (Weeks 1–3) introduce students to the basics of Linux and remote computing while simultaneously teaching them about sequence read quality. Students work through a lab module adapted from Jacques et al. [28] where they use fastqe [28] and fastp [29] on a remote computing cluster to examine the quality of sequencing reads. There are three main objectives of this lab module. First, students develop familiarity with basic computing. Medium fluency in the Linux language can facilitate the use of a wide body of open-source bioinformatic tools. While students do not continue to use Linux throughout this course, the early exposure de-mystifies the basics of computing and shows biology students the utility of developing skills in computing early in their undergraduate careers. Second, students understand that the data they examine are not pristine. In a computational research project where students work with processed data, this module helps students examine raw sequencing reads and assess data quality. Third, students develop basic skills in writing a results section. With a simple project of assessing read quality, students can examine how to communicate key results clearly and concisely.

The second three weeks (Weeks 4–6) introduce students to the two main web-based bioinformatic tools used in the Pathways Project: BLAST and the Genome Browser. Students work through lab modules based on the GEP curriculum “Introduction to NCBI BLAST” [30] and “Understanding Eukaryotic Genes” [31], which are pre-requisite modules for the Pathways Project. In these modules, students learn how to use and interpret blastn, tblastn, and blastx results, and how to visualize and interpret data tracks in a genome browser (e.g. RNA-Seq data) to determine coordinates and reading frames for coding exons, that is perform annotation of a gene. The main objective of this module is to develop proficiency in using BLAST and the Genome Browser while learning fundamentals on the central dogma of molecular biology. In addition, students gain awareness of the freely available resources and tools online that assist in analyzing biological data.

The next 5 weeks (Weeks 7–11) are dedicated to the Pathways Project [32, 33] (https://thegep.org/projects/pathways/). Groups of two to three students are assigned a non-melanogaster Drosophila species and gene from the insulin signaling pathway. In Weeks 7 and 8, they determine the location of the ortholog to their gene by comparing the known protein sequence from D. melanogaster to the genome of their non-melanogaster species using tblastn. Upon pinpointing a putative location, they next examine for local synteny in the wider genomic neighborhood of that location and use their findings to support or refute the putative location as containing the ortholog in their target species (Fig. 1). In Weeks 9 and 10, they annotate the ortholog in their non-melanogaster species using RNA-seq data, their knowledge of gene structure, predicted gene tracks, and a narrow search for the coding exons of the gene from D. melanogaster in the putative location. In the final week, they produce data files with their gene annotation and compare the resulting predicted protein to the known protein in D. melanogaster (Fig. 1). Through this project, they develop further proficiency in using BLAST and the Genome Browser and a deeper understanding of the power of these tools in helping to answer biological questions. They also engage in extended scientific inquiry, develop habits encouraging detail-oriented behavior, and contribute novel findings to an ongoing research project.

Figure 1.

Schematic illustrating the main steps of the gene annotation research through the GEP Pathways Project (https://thegep.org/projects/pathways/). Illustration is based on the Pathways Project Annotation Walkthrough (Sandlin et al. unpublished) and Workflow (Rele and Sandlin unpublished). Sections culminating in “Result 1” are the first half of the Pathways Project and the basis of their Pathways 1 paper, while the remainder correspond to the second half of the Pathways Project and are incorporated in the Pathways 2 paper (Table 2)

The remaining three weeks are dedicated to scientific communication. Students perform literature reviews and collect primary research articles related to their genes, species, and the insulin signaling pathway. They then prepare a short research talk that they present in the final week of lab.

An alternative assessment strategy: Specifications grading

Performing scientific research and communication effectively requires iteration, revision, and mastery of fundamental skills and concepts for sound application during the research process. To incentivize iteration, revision, and mastery, an alternative assessment system known as specifications grading was implemented.

Specifications grading focuses on the creation of tiered or bundled assessments in a course. Students complete different tiers or bundles of assessments, and grades are only assigned at the end of the course based on the number of assessments completed to specifications. Student work is evaluated using a pass/fail grading structure, where work that has failed to meet specifications is revised based on instructor feedback until it meets specifications [25]. Assessment types are usually tied to explicit learning objectives in the course, making it clear what skills or concepts students are mastering if they meet specifications on completed work. This structure typically increases transparency on what is needed to pass a course, creates flexibility in student deadlines, reduces student anxiety about performance as they have opportunities to revise, and helps students develop a growth mindset for learning [25, 34].

A tiered specifications grading strategy was used for this course, where students completed a variety of different assessment activities in the course to pass the class. Three tiers for A, B, and C final course grades were developed, with a different number of assessments required for each tier (Table 2). Minor edits were made across the three iterations of the course, and the details included here are for the third, most recent, iteration. Challenge problems were one assessment category, where students had to complete a challenge problem based on skills and concepts learned in class or lab. Problems were designed to take 2–3 h to complete, and one new problem and/or revision could be submitted each week. The C-tier required the completion of four problems, the B-tier required the completion of five problems, and the A-tier required the completion of seven problems (Table 2). Student submissions were assigned a check if completed to satisfaction, check-minus if done but needed revision, and an incomplete if major pieces were not attempted. The number of revision opportunities per problem was not limited, but because no more than one revision could be submitted each week, there was a practical limit on the number of opportunities available.

Table 2.

Assessments required for different grade tiers in the BIOL 199 Big Data in Biology course

Assessment types (Total possible)	Description	C-tier	B-tier	A-tier
1. Close readings (1–4/week)	Readings on Perusall requiring 1–2 high-quality comments per reading.	70%	80%	90%
2. Participation-class (28)	Class attendance each day. If missed and absence is excused, completing mini quiz based on class slides correctly.	21	24	27
3. Participation-lab (14)	Completion of lab assignment, usually a 3-page worksheet, to satisfaction. Typically group assignment.	12	13	14
4. Cumulative quiz questions (16)	Quiz once every three weeks randomly chosen from set of pre-provided questions that are based on concepts and skills learned each week.	6	8	10
5. Challenge problems (9)	Extended homework problems based on skills and tools learned in class or lab.	4	5	7
6. Annotated bibliographies (3)	Summaries of primary research articles discussed in class or researched for the Pathways Project.	1	2	3
7. Weekly Reflection (14)	Short metacognitive quiz asking students to reflect on topics or activities from that week's lessons.	11	12	13
8. Self-evaluations (3)	Long-form metacognitive assessment, asking students to evaluate their learning and specifications grading goals.	3	3	3
9. Paper write-upsa	Scaffolded writing assignments based on the initial lab module and the Pathways Project CURE.	NGS, Pathways1-Content Only	Peer Review, Pathways1	Pathways2, Report

a

Description of writing assignments: NGS = results section write-up from the first lab module focused on read quality; Pathways1 = results section for first half of Pathways Project (Content Only means the student included all necessary pieces of the project but writing quality still needs work); Pathways2 = results section expanded to include second half of Pathways Project + Discussion; Report = Document designed by the GEP to collect student findings for inclusion in the publication pipeline; Peer Review = anonymous review of two classmates' Pathways1 papers.

Details shown here are for the third iteration of the course, in Fall 2022.

In the third iteration of the course, there were nine types of assessments that could be placed into one of three categories (Table 2). Formative assessments included annotated homework readings through the online social annotation platform Perusall (https://www.perusall.com/), class participation, and lab worksheets. Summative assessments included challenge problems, annotated bibliographies, cumulative quiz questions, and papers. Metacognitive assessments included weekly reflections and self-evaluations. Opportunities to make up formative assessments or extend deadlines for summative and metacognitive assessments were available through tokens, where each student started with one token. In lieu of extra credit, additional activities like connecting a seminar they attended to course content or soliciting feedback from the university writing center were rewarded with additional tokens.

Troubleshooting specifications grading

In many bioinformatics research projects, understanding of multiple foundational biological concepts and competency in multiple skills and tools are required to fully work through the project. For instance, the Pathways Project implemented in this course requires familiarity with fundamental concepts in molecular biology, genetics, and evolution; and it requires proficiency in using BLAST, the Genome Browser, and comparative genomic tools like dot plots. Research is often collaborative, so students work in small groups to complete their projects. Successfully communicating their findings requires written and oral communication skills for conveying scientific information clearly and concisely, which involve a suite of skills that usually take many years to master. For students to successfully complete their research projects, there are multiple challenges for the instructor, including (i) how to scaffold assignments effectively to build necessary proficiencies while maintaining flexibility; (ii) how to incorporate and assess group work when final grade goals may differ among students; and (iii) how to help students assess their learning and accomplishments, particularly regarding their goals within the course. The next section focuses on the primary ways these challenges were navigated within the framework of specifications grading.

Scaffolding writing assignments effectively

Learning how to communicate scientific findings in a clear and concise written format is an important aspect of doing research. However, with numerous other learning objectives related to understanding and carrying out research effectively (Table 1), the opportunity for multiple writing assignments is limited. Many courses in other disciplines that use alternative grading formats to foster written communication skills often assign multiple writing assignments [35], where completing more written assignments to satisfaction plays a formative role in a student’s final letter grade. The focus on the entire research process in this course, however, necessitates balancing writing-based assessments with both foundational and bioinformatic-focused biology assessments. As a result, the writing assessments in this course involved narrowing the writing requirements to sections of a scientific report, with a particular emphasis on the Results section.

To emphasize effective communication of their scientific findings, two major writing projects focused on the Results and Discussion sections of a paper were implemented. In this way, students could focus narrowly on how best to communicate their findings and how to separate their concrete findings from interpretation of their findings, which may be speculative. Students focused on two assessments based on lab projects—a short Results section based on their read quality findings, and a Results and Discussion section based on the two-part research question from the Pathways Project (Table 2 and Fig. 1). All students were required to complete the read quality Results section to standards and to attempt a Results section for the Pathways Project for the C-tier. Students aiming for the B-tier were required to complete the Results section for the first question of the Pathways Project to standards with one required revision incorporating peer and instructor feedback. Students aiming for the A-tier were required to complete the above and extend their paper to include the second half of the Pathways Project and an extended discussion to standards. Breaking the scientific report into sections and focusing solely on the Results and Discussion section allowed repeated application of written communication skills and a directed set of feedback that was less overwhelming for writers at the earliest stages.

In addition, breaking the scientific report into sections helped to scaffold writing assignments so that students could make an informed decision on whether to focus on the A-tier writing assignment or focus on revisions for the C- or B-tier writing assignments. The read quality lab was completed in the first three weeks of the semester, with the writing assignment due by Week 5 and revisions due within a month (by Week 9). The first half of the Pathways Project was completed by Week 8, upon which they could practice their writing skills with a new Results section due by Week 10. They then were required to give and receive anonymous peer feedback and use peer and instructor feedback to submit a revised paper by Week 12. Instructor feedback would then include a recommendation on extending the paper for the A-tier assignment, based on the student’s understanding of the research project, and demonstrated competency in writing a Results section. Those students who chose to submit the extended paper by the final week of classes (Week 15) would have an opportunity to revise their extended paper once during Finals Week. This process helped students use the practice and feedback from previous writing assignments to improve later writing assignments. It also helped both students and the instructor to gauge where their writing needed improvement for a narrower set of objectives and whether it was advisable for the student to tackle a wider set of writing objectives.

Group work and specifications grading

A major component of scientific research is the ability to work collaboratively with others. Studies have shown that group work helps students achieve and maintain positive attitudes toward science [36]. Implementing group work can be challenging due to the planning and coordination necessary among students, varying levels of commitment to the project among students, and different levels of communication and initiative among students within a group. In specifications grading, the choice of a final grade is dependent on completing some number of tasks to standards. There can be an inherent tension between students in the same group who are aiming for different tiers. Matching groups based on their chosen tier is not practical as it would reveal which students chose which tier publicly, and it would not allow flexibility for students who decide later in the semester to attempt a higher tier or focus on a narrower set of learning objectives in a lower tier.

To integrate group assessments into a course with specifications grading, two major steps were taken. First, group work was prioritized in the classroom/lab. For student lab assignments, where students completed separate worksheets but in discussion with their group (Table 2), one assignment was randomly chosen on which the instructor provided feedback. The commented worksheet would be scanned and returned to all members of the group. In the next lab section, before they could move into new material, they had to work together to determine what needed revision. This incentivized students to discuss their responses to minimize the number of revisions needed, and avoided unequal contribution to revisions if they were submitted asynchronously, where one or two students would likely take on the responsibility.

The main group project in this course was the final presentation on the Pathways Project, which occurred in the final week of class during the lab period. Students were provided lab and class time to work together on the presentation, and students were required to meet with the instructor to go over questions they had on the presentation. Though the project took substantial time, the presentation counted as a lab activity rather than its own category. All lab activities had to be completed to satisfaction for the A-tier, one could be missed for the B-tier, and two could be missed for the C-tier (Table 2). The strict lab requirements helped to emphasize the course objective of learning to do research, and it incentivized student participation in lab days focused on developing the group presentation. The final presentation was framed as a celebration of their work and an opportunity to share their work with their classmates, rather than a major assessment to pass. Final presentations had to pass a content rubric, and feedback was given on quality. Anything missing or not described well from the content rubric could be revised by a written response generated by the group. Using this format, all groups gave well-structured presentations with contributions from every group member. A written revision was only required in 2 of the 13 groups that presented over the three semesters, primarily due to incorrectly describing a concept.

Helping students evaluate their learning (and grading tier goals)

With specifications grading, a heavy emphasis is placed on a student’s choice of which tier to aim for. Though the list of assessments required for each tier is transparent, what is less clear is the amount of time each assessment will take for each student. Students have different levels of pre-existing experience in the research, biological content, informatic, and communication skills addressed in this course. They also have different levels of energy, time, and interest to spend on areas where they require more practice and feedback. Because of different levels of student experience, comfort, and skill in different objectives, as well as the different levels of early effort students invest in the course, the actual engagement with each assessment is unique to each student. As a result, developing tools to help both the instructor and students gauge learning progress was paramount.

To help students gauge their progress within the course, metacognitive assessments were included. The major metacognitive assessment was a series of three self-evaluations whose purpose was to help students and the instructor assess (i) students’ mastery of course learning objectives (content mastery), and (ii)student needs and goals within the course, particularly regarding the specifications grading format (assessment progress).

The first self-evaluation (SE1) was administered at the beginning of the semester, where the content mastery questions were related to better understanding each student individually and setting a baseline, while the assessment progress questions were focused on addressing questions and concerns on the specifications grading format. Since students were new to course materials, most learning objective-based questions were replaced with questions on why they were taking the course, reflections on their needs and wants in group work, and their comfort speaking in public settings. The only content-based question asked for them to reflect on a “big picture” sentence “75% of the genes responsible for human genetic diseases can be studied in fruit flies.” They were asked to speculate on what the sentence meant, what questions they had on the sentence, and they were not required to do any external research. On assessment progress, students were asked questions about their experience with specifications grading, what was appealing about the format, and what questions they still had, but a specific grade tier goal was not requested.

The second self-evaluation (SE2) was administered halfway through the course (usually Week 7 or 8). Content mastery questions focused on learning objectives, where students were asked to choose three learning objectives (Table 1) across two different categories and demonstrate their mastery by explaining concepts and describing examples from class, lab, and assessments that highlighted their understanding. Students were also asked to choose two learning objectives in which they were still uncomfortable, explaining what they did understand, and where they still had questions. Students were also asked to revisit the “big picture” sentence and provide a greater explanation based on their learnings. For the assessment progress questions, students at this stage had experience with the specifications grading format, the revision process, and the different assessment types. They also had an initial body of evidence from the work they had completed in the first half of the semester. Thus, in SE2, students specified their grade tier goal, their accomplishments up to that point toward achieving that goal, and their plan for completing assignments in the second half of the semester.

The third self-evaluation (SE3) was administered at the end of the course and submitted by students during exam week, in lieu of a final exam. Content mastery questions asked students to demonstrate their mastery across two more learning objectives and explain the meaning of the “big picture” sentence from SE1 a final time. In addition, they were asked to write a short research proposal for studying a question that could use skills and content related to course objectives. Through these questions, students’ ability to retrieve appropriate content and skills and communicate their intent accurately using expert terminology could be examined. For the assessment progress questions, students had now completed or submitted all work, with no additional opportunities for revision. Thus, students were asked to examine the work they completed throughout the semester and state the final letter grade they should receive, justified by the evidence from their work completed to standards.

Students were required to complete all three self-evaluations to pass the course, and for each, feedback was provided on their responses. On content questions, what concepts, skills, and assessments they chose to spotlight could be examined, and their explanation and use of expert terminology helped to check their understanding of course concepts. In earlier self-evaluations, instructor feedback helped to highlight how they could improve their communications, and in the final self-evaluation, their explanations helped in gauging the letter grade within a grade tier (e.g. A−, A, or A+). On assessment progress questions, students had to provide evidence in SE2 and SE3 to support their chosen grade tier, making students articulate their accomplishments within the course. In SE1, they could focus on learning about the specifications grading format; in SE2, they could use their experience to choose a grade tier and receive feedback on how difficult achieving that grade tier would be based on the assignments completed thus far; and in SE3, they could argue for a grade based on the body of evidence they had created throughout the semester.

An important tool to help the instructor and students keep tabs on the body of evidence was a one-page checklist used to track student progress in completing assessments (Fig. 2). The checklist used color shading to indicate assessments needed to satisfy a grading tier. Hard copies were used to simplify the updating process, and scans of the document were returned every few weeks for students to check if their completed work was recorded accurately. Electronic versions could also work, but the hard copy facilitated the ability to pull out one or more student checklists depending on who turned in assignments that week. The hard copy was also helpful during office hours, where the single sheet could be pulled out quickly to discuss with the student while reviewing their progress. By being on a single page and committing to regular scanned updates throughout the semester, it was easy to quickly see where students had gaps in their records and reach out appropriately. By returning regular scans, particularly right before SE2 and SE3 were due, students could check whether items were recorded accurately, check their progress, and use the updated checklist to help develop a written response to the assessment progress questions in the self-evaluations.

Figure 2.

Specifications Checklist used by students and instructor for tracking completed assessments. Color-coding indicates the C-tier (yellow), B-tier (blue), and A-tier (red) required number of assessments. All students start with one token (‘Start’), and tokens are tracked under the chart. Comments are added two to three times over the semester, particularly in the second half of the course, to give students feedback on what they should complete. CQs refers to Cumulative Quizzes, and CPs refers to Challenge Problems. Details shown here are for the third iteration of the course, in Fall 2022

Synergies from specs grading and CURE implementation in course design

When implementing a CURE, students master concepts and skills through doing research. In this process, they apply concepts in new situations, repeat skills iteratively, and contribute novel research to a greater scientific question, usually with a contribution in some form to the wider community. These experiences have been shown to increase student learning and engagement within the sciences [8, 9]. CUREs focus on course curriculum, but they can be implemented with many different assessment types. Specifications grading is a relatively new assessment strategy [25] that shifts student focus from grades to feedback, where students utilize feedback and revision to achieve mastery. This format encourages instructors to build assessments around learning objectives, increases transparency on the relationship between assessments and grades, and has built-in flexibility for students who need it. This assessment structure of specifications grading lends itself readily to the learning objectives inherent to implementing a CURE in three distinct ways: (i) by shifting the focus from grades to mastery; (ii) by allowing for revision and iteration; and (iii) by promoting a growth mindset for students.

The shift in focus from grades to mastery is fundamental in specifications grading. In a CURE, the goal is for a student to investigate a research question and communicate their findings to others, where effective investigation typically draws on a wide spread of concepts and skills. To successfully complete a GEP Pathways Project, students must hold a strong understanding of gene structure and the central dogma of molecular biology, and they must develop strong proficiency in bioinformatic tools such as BLAST and the Genome Browser. Interpreting their findings requires familiarity with basic evolutionary concepts. By requiring that students perform to specifications in any submitted assignment, students were incentivized to learn foundational concepts and skills vital to the research process. Students could not ignore concepts and skills in which they struggled, as they were required to revise unsatisfactory work to standards to meet grading tier requirements. Paces differed, but this format encouraged students to use both written feedback and conversations during office hours to assess their gaps in knowledge. Students often expressed high frustration in the course, a feature that has been noted in other classrooms [37, 38], but in the second half of the semester, the energy and time they put into revising their work had a notable positive effect. While exams can be administered with specifications grading, for example [27, 39–41], major assessments in this course were not exam-based, so students could focus on scientific inquiry and communication through writing assessments.

Specifications grading prioritizes revision and iteration, which is also fundamental to the research process. In specifications grading, revision allows students to achieve direct gains from using feedback in the form of completing assignments to standards, and iteration allows students to demonstrate their mastery of a skill or concept in different contexts. Scientists continuously revise research protocols, experiments, analyses, and communications. For example, a fundamental aspect of the research process is peer review, where revision is typically required using feedback from other expert scientists. In this course, it was emphasized to students that the revision process students underwent in the specifications grading structure mimics the revision process that researchers typically apply in the lab. A peer review process was implemented after their second paper assignment, and students were required to complete the peer review process for a B-grade tier. Thus, an emphasis could be placed not just on instructor feedback, but also on peer feedback and their own reflection on their paper for improving their final written communication. By practicing revision and soliciting feedback, students would develop authentic practices integral to the research process.

Iteration of concepts and skills related to core learning objectives across different assessments improved student learning and showed students the rewards of spending time and energy on revising to mastery. Training labs and classroom activities were the initial exposure to concepts and skills, where completing lab worksheets to standards meant that students could not ignore feedback in areas where they were struggling. Challenge problems and quizzes pushed students to repeatedly practice those concepts and skills; again, completing to standards meant the students needed to practice with the tool enough to develop proficiency. Repeated use of these concepts and skills in the final research project emphasized the importance of the tools and knowledge applied and helped validate to students why learning these concepts and skills was important.

Lastly, both the CURE and specifications grading inherently foster a growth mindset. In an assessment of the impact of the GEP CUREs, a key idea regarding student learning was the concept of “formative frustration” [37, 38], where students found the work that they were doing frustrating due to the need to integrate many concepts and skills in new situations through the CURE. Ultimately, that frustration helped them better engage with the research process. Similarly, in specifications grading, the revision process was often described as frustrating and caused anxiety, but by undergoing the revision process, they successfully mastered materials and experienced a sense of achievement in completing assessments to satisfaction. In a CURE, students cannot fully complete the research project without developing a strong foundation in concepts and skills related to the project. Specifications grading facilitates this process, by using revision and iteration to assist students in developing mastery of learning objectives and emphasizing feedback and opportunities to revise as spaces where students could learn and grow.

Reflecting on student experiences

Collectively over the three iterations of this course, every student who completed the course finished the research project, wrote at least one Results section to satisfaction, and presented their work to the class. Twenty-six percent of students achieved a B-tier grade, successfully completing an additional Results section. Fifty-four percent achieved an A-tier grade, which involved extending their Pathways paper to incorporate their second set of Results and adding a Discussion. In addition, they had to individually fill out the research report required for the GEP project. About half of the students agreed to be co-authors on the gene annotation project they completed, which requires submission of a completed research report to the GEP, leading to 19 projects with 27 student co-authors that originated from the BIOL 199 course currently in preparation or under review for eventual publication through microPublications Biology [42]. Another ten projects from research students were also submitted in the last 3 years, in part due to students joining the instructor’s research lab after completing BIOL 199. Students from the first two iterations of the course (21–22 academic year) had the opportunity to present their work at a regional symposium in Fall 2022 through the Regional Node framework of the GEP [43]. For the symposium, eight past students worked together to develop three oral presentations to share their work with GEP faculty and students from other institutions.

A post-course anonymous survey was administered 1–2 years after the course was completed, which asked students to reflect on their learning in the class, and specifically on the role that specifications grading played in that learning. The post-course survey asked students to agree or disagree with 14 statements related to specifications grading and the course using a five-point Likert scale. Students were also asked to provide qualitative responses to questions about how specifications grading contributed positively and negatively to their learning within the course, and how their experience in the course compared to other college courses they have taken. Of the 41 students whose contact information was available, a 36.6% response rate was found (n = 15/41, Supplementary Materials), enough to allow exploration of the student perspective for the grading format used in this course. Of responding students, 73% completed or intend to complete the core Biology curriculum for majors, which consists of two survey courses (BIOL 200 and BIOL 202). Currently, 55% of the students who took this course have taken BIOL 202, which suggests that the survey respondents are slightly biased toward those who continued taking biology courses after BIOL 199.

Generally, students found that specifications grading had a positive effect on their learning. The most uniform agreement was that specifications grading helped them understand course content (100%, Fig. 3) and master foundational concepts to conduct genomic research (93%). Students also felt specifications grading improved their ability to learn and helped them to retain course content, to effectively investigate a research question, to develop authentic research practices, and to learn the process of scientific writing (73%–80%). Most found that specifications grading decreased grade anxiety, helped to prepare them for future biology courses, and helped them develop proficiency in bioinformatic skills (60%–67%). However, some students found that their anxiety over their grades increased due to the specifications grading format (33%, Fig. 3), indicating that the experience was not uniform.

Figure 3.

Post-course survey results for 14 statements where students were asked to agree or disagree using a five-point Likert scale. Statements 1–10 were prefaced with “The specifications grading format’s focus on mastery, iteration, and revision helped (me):” and Statements 11–14 were prefaced with “As a result of my effort in this class”. Numbers on the left (%SD/D) indicate the percent of “Strongly Disagree” and “Disagree” responses, and numbers on the right (%A/SA) indicate the percent of “Strongly Agree” and “Agree” responses. Fifteen of 41 students responded to the survey

The qualitative data collected provide additional insights into the survey results (Table 3). Students generally agreed that the grading format positively impacted their understanding of course content, concepts related to genomic research, and their ability to learn and retain that content knowledge. Their responses emphasized the development of semantic knowledge skills, specifically related to scientific research practices (“This class helped me develop authentic research practices”) and metacognitive awareness (“This grading format helps to shift students’ focus from binge studying the quizzes and exams to really mastering material that’s been taught in class”). Student responses also repeatedly underscored the grading system’s inherent reinforcement of a growth mindset, or as student responses put it, the ability to “make mistakes and grow,” to have “more room to be wrong,” or to be “able to revise and to constantly get better.”

Table 3.

Sample responses from qualitative post-course survey data

Common trends in experience	Language from student responses
Development of scientific research practices	“This class helped me develop authentic research practices…I felt that I have been taught solid background knowledge and skills that can help me to conduct authentic research practices and produce a short scientific paper.” “This class challenged my skills and abilities much more than my typical class as I learned a new program and got to experience first-hand a type of computational biology research, which was unfamiliar to me.”
Increased metacognitive awareness	“This course was very informative and taught me a lot about biological concepts, as well as my study habits and time management.” “This grading format helps to shift students’ focus from binge studying the quizzes and exams to really mastering on material that’s been taught in class.”
Evidence of growth mindset	“Compared to other college courses I have taken, I feel more confident to make mistakes and grow upon my mistakes in this class.” “I felt like I had more room to be wrong so I was less afraid to tackle any work, compared to other classes where it felt more daunting to try to study and be wrong in the end.” “I did appreciate the idea of being able to revise and to constantly get better …”
Challenge of increased workload	“I definitely felt less stress about my grade in this course thanks to the grading system, compared to my other courses…However, the workload was one of the heaviest that I have had…which was stressful in other ways.” “So much work … I probably spent like 15 hours of work outside of class each week just to try to keep up. I will say, that I appreciated the final project being actual research that I could write up and present and had something tangible and real-world to show for my efforts.” “This course was 100% the most work I’ve ever had in any one class…I definitely accomplished more than any other class, and the work was among the most rewarding, but absolutely took the most effort.”

Three questions were posed: “What aspects of the grading format contributed positively to your learning within the course?”, “What aspects of the grading format contributed negatively to your learning within the course?”, and “How would you compare your experience in this course to your experiences in other college courses you have taken?”. In addition, students could respond to statements they assessed using a 5-point Likert scale. Fifteen of 41 students responded to the survey, and 8 of the 15 students returned comments.

Finally, student responses revealed a complex relationship between grading, workload, anxiety, and other stressors that often categorize student experience within the college classroom. For instance, while for some students, specifications grading decreased grade anxiety, the high workload that comes with the revision requirements of this grading system contributed to increased stress (“I definitely felt less stress about my grade…However, the workload was one of the heaviest that I have had…which was stressful in other ways.”). For others, the high workload, while overwhelming at times, resulted in an increased sense of authentic learning (“I appreciated the final project being actual research that I could write up and present”) or accomplishment (“the work was among the most rewarding, but it absolutely took the most effort”).

Overall, patterns suggest that specifications grading assisted students in learning, decreased stress for at least a subset of students, and helped them engage in an authentic research experience in genomics.

Discussion

Here, the paired implementation of a GEP CURE and specifications grading in an introductory undergraduate biology classroom focused on genomics and bioinformatics is described. Multiple challenges needed addressing to successfully pair the two, but doing so led to synergies that helped students carry out their research projects and gain familiarity with genomic and bioinformatic concepts and skills.

One challenge was balancing flexibility typically allowed in courses using specifications grading with mastering enough concepts and skills, particularly in writing, to successfully undertake the research project. The approach used in this course was careful scaffolding of writing assignments, such that students wrote more than one Results section, with extension to a full Discussion section only after successfully writing a Results section—a practice that courses in other disciplines have also employed [44]. Another challenge was evaluating group work—a fundamental aspect of doing biological research—when using specifications grading. For this, the course was designed to bring group revision into the lab and made group assessments a required component for passing the class. A third challenge was how to help students assess their learning, especially given the diverse activities and assignments they needed to complete. For this, a series of metacognitive activities were assigned that built upon one another, requiring students to explore and document their learning over time.

Over the three times this course was offered, several aspects of the course structure were revised to improve both student and faculty experience. For instance, some benefits of the specifications grading format, such as saving faculty time and reducing student stress [25], were not easy outcomes to achieve with just the simple implementation of specifications grading. With each iteration of the course, the details of each grading tier underwent revision, leading to improvements. The most obvious demonstration of the power of effective specifications grading in saving faculty (and student) time came from the scaffolded writing assignments. In the first iteration of the course, the same tiered writing assignments were present, but students chose whether to do the shorter Pathways writing assignment with only a results section (B-tier) or the longer Pathways writing assignment that included a Discussion (A-tier). Almost all students aimed for the A-tier, even if they were not comfortable with the Pathways Project content or had limited experience in scientific writing. This led to a high grading load, as several students required more feedback and often, multiple rounds of revision. In the next iteration, the course was revised such that the two Pathways writing assignments were scaffolded as described here, and students were required to complete the B-tier assignment before moving on to the A-tier assignment. For this format, returning feedback quickly on the B-tier assignment was important, but student assignments generally reflected a higher level of quality. Thus, feedback could be more targeted, saving faculty time. While this was the most substantial change, other edits such as limiting the number of challenge problems that could be submitted each week and eliminating the revision process for lower-stake assessments, such as the cumulative quiz, helped to spread grading more evenly across the semester and reduce the number of revisions to track. Thus, specifications grading can easily add faculty time, if students attempt to complete large assignments that require several skills they have not yet mastered. Faculty transitioning courses they have already taught to a specifications grading format may more easily gauge how to structure their grading tiers, but when building a new course, it may take a few iterations to feel this benefit.

Second, reducing student stress did occur, but a subset of students did not experience this benefit. Most of this was tied to the challenge of the course—some student comments indicated that the time and effort needed to do well was too high. One possibility is that the number or types of assignments in this course was too challenging for the introductory biology level. However, the grade distribution across the three courses suggests this is not the case, given the high number of students who received an A-tier grade. Another possibility is the inherent tension between B-tier and A-tier grades. In point-based grading systems, instructors tell students what grade tier they achieved—there is less transparency on how to move between grade tiers aside from doing well on later assignments. In specifications grading, the assignments needed for each grade tier are transparent, but the amount of time each student needs to complete the assignment will always be unique and dependent on their comfort with the material. As a result, there may be students who aim for a grade tier that is difficult for them to achieve because they are still struggling with the material. Two tools that have helped in this course were the development of the checklist, so students could gauge their progress and creating spaces such as the second self-evaluation to discuss student course goals halfway through the course, allowing communication between instructor and student on setting realistic expectations.

One way to help students navigate the challenge of the course is to take better advantage of the aid of student TAs. In the first two iterations of the course, TAs helped to test the lab modules for clarity, accuracy, and timing, and they assisted students during lab with questions. While invaluable, TAs did not assist in tutoring outside of the lab, and they could have been a helpful resource for students. The GEP has implemented a virtual TA system with broad time coverage, to increase access to experienced gene annotators for students who need more resources [43]. This system has been shown to be invaluable to many students who have implemented GEP projects elsewhere [43]. While the virtual TA system was advertised, low class or lab time was dedicated to introducing the virtual TAs to students. As a result, this support system was under-utilized. Due to the high level of GEP curriculum implemented in the course, these TAs could have been additional support structures to help students learn the material sooner. Better integrating virtual TA resources into the class and increasing the support network for students may be one method of reducing student stress.

The ability to implement this course as described is also due to the removal or lowering of multiple barriers instructors often encounter. On the bioinformatics side, many studies have reported challenges in necessary infrastructure and instructor familiarity with bioinformatic tools [5, 45]. The instructor’s background in bioinformatics led to lower hurdles in solving computational challenges during course development and teaching. In addition, support and computational resources from both the university and the GEP were high. Working with the GEP eased the integration of the CURE with course objectives, while computational resources from the university were important for expanding the course beyond the CURE to introduce remote computing to students. Faculty interested in bioinformatics-focused CUREs can receive training opportunities through the GEP network to assist with reducing barriers related to instructor experience [37].

In addition, the department possessed a core curriculum that provided flexibility to teach a course implementing a bioinformatics-focused CURE at the introductory level. Learning objectives related to the research process and understanding evolution were required, but there was otherwise high freedom to experiment with introducing bioinformatics at the introductory level and implementing a semester-long CURE. Studies have shown that investing substantial instructional time to the CURE is very helpful for students to fully benefit [46], which can make it difficult for instructors to commit to a CURE in departments with greater curricular barriers. The use of a GEP CURE opened opportunities to use curricular resources designed to be flexible and adaptable to many classroom situations, further lowering barriers to introducing a bioinformatics-focused CURE at the introductory level.

Finally, infrastructure existed at the university and in the GEP to discuss and learn pedagogy, which allowed brainstorming and troubleshooting with a support network of colleagues. Implementing alternative styles of grading can take high time and energy, especially if students voice initial resistance to unfamiliar classroom structures. These resources greatly reduced barriers to implementing an introductory biology course using a CURE to teach bioinformatics and research skills with specifications grading. Pairing specifications grading with a CURE has promoted the learning of bioinformatics at the introductory level, helped students learn best research practices and scientific communication skills, and promoted student learning that is focused on feedback rather than points. In addition, the focus on feedback, revision, and mastery mimics how scientific research is performed outside of a classroom, further emphasizing that students are engaging in an authentic research experience that reflects best practices in the wider scientific community. Developing a course with specifications grading and a CURE takes time and energy, and it is our hope that the model shared here will ease the path for other instructors interested in implementing genomics/bioinformatics education, CUREs, specifications grading, or some combination of the three.

Acknowledgements

We would like to thank M.A.Y.’s research and BIOL 199 students from 2020 to 2022 for their hard work and time spent on the course and the GEP CURE. Specifically, we would like to acknowledge Betty Duan and Brittany Johnson, for being amazing TAs in the first two iterations of the BDB course, as well as Maddie Olvey and Elena Zerkin, who provided feedback on a presentation on the BDB course that led to the development of this manuscript. This course could not have happened without the endless support of faculty and staff involved in the GEP. Thank you as well to the organizers of the SMBE 2023 “Science for everybody: education and outreach in molecular biology and evolution” and “IDEA symposium” sessions, for the opportunity to reflect deeply on experiences teaching this course. Lastly, deep thanks to Dr Amy Hark for providing feedback on this manuscript.

Author contributions

Melinda Yang (Conceptualization [lead], Data curation [lead], Formal analysis [lead], Investigation [lead], Methodology [lead], Project administration [lead], Resources [lead], Software [lead], Supervision [lead], Validation [lead], Visualization [lead], Writing—original draft [lead], Writing—review & editing [lead]), Kylie Korsnack (Data curation [supporting], Formal analysis [supporting], Methodology [supporting], Writing—review & editing [supporting])

Funding

Curriculum development, faculty training, and the Genomics Education Partnership microPublication pipelines were developed with support by the National Science Foundation IUSE-1915544 and the National Institute of Health IPERT-R25GM130517.

Supplementary data

Supplementary data are available at Biology Methods and Protocols online.

Conflict of interest statement. None declared.

Data availability

Curriculum from the GEP can be found at https://thegep.org/. Materials from the Fall 2022 BIOL 199 course (third iteration) can be found at https://github.com/MYangLab/BDB_F22. The post-course survey was approved by the University of Richmond IRB (IRB no. URIRB231230), and a full set of results are found in the Supplementary Materials.