Although differently formatted cladograms (hierarchical diagrams depicting evolutionary relationships among taxa) depict the same information, they may not be equally easy to comprehend. Undergraduate biology students attempted to translate cladograms from the diagonal to the rectangular format. The “backbone” line of each diagonal cladogram was slanted either up or down to the right. Eye movement analyses indicated that the students had a general bias to scan from left to right. Their scanning direction also depended on the orientation of the “backbone” line, resulting in upward or downward scanning, following the directional slant of the line. Because scanning down facilitates correct interpretation of the nested relationships, translation accuracy was higher for the down than for the up cladograms. Unfortunately, most diagonal cladograms in textbooks are in the upward orientation. This probably impairs students' success at tree thinking (i.e., interpreting and reasoning about evolutionary relationships depicted in cladograms), an important twenty-first century skill.
Constructing the Tree of Life (ToL), a diagrammatic depiction of the evolutionary relationships among all extinct and extant taxa, is one of biology's most important tasks. The current—but woefully incomplete—ToL has already yielded numerous benefits for human health, agriculture, and biotechnology (e.g., Futuyma 2004, Yates et al. 2004). To mention just a single example, the discovery that unicellular microsporidians are fungi is helping scientists create better methods for controlling this pathogen in honeybee colonies (Freeman 2011). Because every third bite of food we eat originates from plants that rely on honeybee pollination (Morse and Calderone 2000), microsporidian infections in honeybee colonies are a significant agricultural problem worldwide.
Against this backdrop, it is clear that biologists and nonbiologists alike will need at least a basic understanding of the ToL to make informed personal and policy decisions concerning twenty-first century problems. Reflecting this reality, science educators have argued that college and high school biology curricula must include tree thinking—the skills required to interpret, reason about, and apply information about the evolutionary relationships depicted in the ToL (e.g., O'Hara 1988, Baum et al. 2005, Catley 2006, Gregory 2008, Halverson et al. 2011).
The most common means of depicting (portions of) the ToL in college-level biology textbooks is the node-based cladogram (Catley and Novick 2008). Such cladograms are acyclical directed graphs (i.e., hierarchical branching diagrams) that represent the hypothesized evolutionary history of a set of taxa, depicted at the tips of the tree branches, in terms of nested levels of common ancestry (Martin et al. 2010). The structure of these trees is derived from the principles of phylogenetic systematics (Hennig 1966), and the inferred common ancestors are placed at the branching points (nodes) of the diagram. Throughout this article, we use the term cladogram as it is defined above. Cladograms can be drawn in a variety of ways, a few of which are shown in figure 1 for taxa labeled E—I. Figure 1a shows the relationships among these taxa in a rectangular format. Figure 1b–1d shows the same relationships in a diagonal format. In all cases, branching points further back in time are toward the root of the diagram, and those that are more recent are near the tips. That is, relative time progresses from more recent to more ancient, going from the top to the bottom of a cladogram.
The cladograms in figure 1a–1d are informationally equivalent representations: All the information in any one cladogram can be inferred from each other cladogram (Larkin and Simon 1987); they depict the same set of relationships. Informationally equivalent representations are computationally equivalent if any inference that can be made easily and quickly from one representation can also be made easily and quickly from the other representation (Larkin and Simon 1987). A lack of computational equivalence means that even though the representations depict an identical set of relationships, students find it easier to understand and are more successful at reasoning from one representation than from the other. A critical question for biology education, therefore, is whether the various cladogram formats are computationally equivalent for students who are expected to be able to interpret and reason from the cladograms they encounter in textbooks and classrooms.
Theoretical psychology aims to understand and provide an explanation for any lack of computational equivalence between informationally equivalent representations. There is abundant evidence from the psychological literature on diagram comprehension that informationally equivalent diagrams (e.g., bar and line graphs) are often not computationally equivalent (e.g., McGuinness 1986, Zacks and Tversky 1999, Peebles and Cheng 2003, Körner 2004, Hurley and Novick 2010). In the present study, we investigated the question of computational equivalence for diagonal-format cladograms depicted in the up-to-the-right (UR) versus its mirror-image down-to-the-right (DR) orientation (figure 1b and 1c, respectively). A recent examination of the evolutionary diagrams printed in 9 high school biology textbooks and 18 college introductory biology, zoology, and botany textbooks revealed that at all levels, cladograms were most often drawn in the diagonal format (Catley and Novick 2008). In a further analysis of these diagonal cladograms for the present research, we found that 82% had a dominant “backbone” line slanted UR, as in figure 1b; 5% had a dominant “backbone” line slanted DR, as in figure 1c; and the remaining 13% were more balanced (i.e., lacked a dominant direction of slant). Similarly, in a recent tree-thinking quiz for scientists (Baum et al. 2005), most cladograms were presented in the diagonal format, and of those cladograms, nearly 80% were in the UR orientation.
In the present study, we tested the hypothesis that even though UR and DR cladograms are composed of the same parts arranged in the same way and differ only in orientation, they may not be computationally equivalent. More specifically, we predicted that the more common UR orientation is harder for students to understand. This hypothesis is suggested by cognitive psychological research that indicates that both general processing biases for scanning visual material and specific processing biases induced by a particular visual display may affect how people scan diagrams, which in turn may affect their understanding of the diagrams. We discuss three potential processing biases: (1) a bias toward a horizontal direction of processing, (2) a bias toward a vertical direction of processing, and (3) a bias toward directional processing induced by salient components of the diagram. We then report the results of an experiment in which we tested our hypothesis.
Literate, native English-speaking college students have accumulated more than a decade of experience reading and writing text from left to right. This extensive experience confers a preferred left-to-right direction of processing for a variety of nonlinguistic visual tasks (Nachshon 1985, Tversky et al. 1991, Fuhrman and Boroditsky 2010). Although this processing bias can be overridden by salient aspects of the visual stimuli (Taylor HA and Tversky 1992), reading order clearly exerts a powerful influence on cognitive processing on a variety of tasks that do not involve reading. We therefore predicted that in processing diagonal cladograms in both orientations, students would show a general left-to-right scanning bias.
The vertical dimension, in contrast, does not appear to exert a consistent directional influence on cognitive processing. Although people have easier (faster) access to information arrayed along the vertical dimension than along the other two spatial dimensions (Franklin and Tversky 1990), there are competing pressures for scanning downward and upward. Although gravity and reading order specify movement from top to bottom, y-axes on graphs are labeled with values that increase from bottom to top, and people and other living things are observed to grow upward as they get older. We therefore predicted no general bias to scan diagonal cladograms either upward or downward.
Finally, the direction in which people scan a diagram may be affected by its salient components. For example, in a study by Taylor and Tversky (1992), students tended to start their descriptions and drawings of studied maps (of an amusement park or a convention center) with the entrance, which was not located at the top left; moreover, the order in which landmarks were recalled tended to follow salient paths in the environment depicted.
In cladograms, each line represents a lineage ending at a particular taxon. In the diagonal format, there is generally a “backbone” line from which other lines project. Although this line is perceptually distinctive, it does not have a correspondingly distinct meaning. By rotating cladogram branches at the nodes, the “backbone” line can be made to end at any other taxon, as is illustrated by the cladograms in figure 1b and 1d (for which the “backbone” lines end at taxa E and G, respectively). Nevertheless, students interpret this line as a coherent entity: When they divide diagonal-format cladograms into parts, they prefer to keep the “backbone” line intact (Novick and Catley 2007). Therefore, we predicted that students' attention would be drawn to this line and that the direction of its slant would influence their processing of the cladogram. Combining this processing bias with the expected reading-order bias, we predicted that the students would process UR cladograms (figure 1b) from bottom to top but DR cladograms (figure 1c) from top to bottom.
Understanding the information depicted in cladograms means, at minimum, understanding the pattern of nesting. In figure 1, taxa E and F are more closely related to each other than either is to G, H, or I; taxa G and H are more closely related to each other than either is to E, F, or I; and these two groups of taxa (clades) are more closely related to each other than either group is to I. We evaluated students' understanding of nested relationships by testing their ability to translate diagonal-format cladograms to the rectangular format (Novick and Catley 2007, Novick et al. 2010). Because processing diagonal cladograms from top to bottom would lead students to reach the branching points (nodes) for more closely related taxa before those for less closely related taxa, we predicted higher accuracy for DR than for UR cladograms.
Our hypotheses concern the order in which students visually scan diagonal-format cladograms. Therefore, following prior research on the cognitive processing of both concrete, domain-specific diagrams such as pulley systems, weather maps, and cellular transport diagrams (Hegarty and Just 1993, Cook et al. 2008, Hegarty et al. 2010) and more abstract diagrams such as graphs of various types (Carpenter and Shah 1998, Peebles and Cheng 2003, Körner 2004), we recorded the students' eye movements while they examined the diagonal cladograms. To our knowledge, this is the first use of this methodology to investigate cladogram understanding.
Student comprehension experiment: Design and procedure
Nineteen college students at the University of California, Santa Barbara (UCSB)—15 women and 4 men (mean [M] age = 20.9 years, standard deviation = 0.88)—were recruited from two upper-level biology classes (n = 15) and from a paid subject pool sponsored by the psychology department and open to any student on campus (n = 4). They participated in exchange for $10 compensation. The students were juniors and seniors who were recruited because they had previously completed the yearlong introductory biology sequence for biology majors at UCSB (or an equivalent sequence elsewhere). Not surprisingly given this prerequisite, all of the students listed a biology-related major: aquatic biology (n = 7), zoology (n = 3), biology (n = 3), ecology and evolutionary biology (n = 2), biopsychology (n = 2), or physiology (n = 1). One student recruited from one of the biology classes failed to list her major but indicated a possible interest in attending graduate school in environmental studies. Two UCSB biology professors who teach courses in which cladograms are used also completed our task. An additional three students participated, but their data were not analyzed because equipment malfunction or student noncompliance with the instructions affected data collection on a substantial portion of their trials.
Diagram orientation and the order of the taxon labels were manipulated within subjects as is illustrated in the left column of figure 2. That is, every subject provided data for all four cells of the design. For the diagram orientation factor, we varied whether the “backbone” line of diagonal cladograms was slanted UR or DR. The taxa were denoted by single letters, and the order of these letters was manipulated by varying whether they occurred in forward or reverse alphabetical order. As was noted earlier, we expected the subjects to process the UR and DR cladograms differently, which would lead to differences in both scanning direction and accuracy. No predictions were made concerning the alphabetical ordering of the taxon labels. That factor served as a test of the generality of any diagram-orientation effects.
Diagonal-format cladograms can be drawn either with the root slanted so that it is simply an extension of the “backbone” line or with the root straight. As is shown in figure 2, we used the slanted form in the present materials. Of the 10 college biology textbooks surveyed by Catley and Novick (2008) that had at least six diagonal-format cladograms, 9 predominantly used the form with a slanted root, and only 1 predominantly used the form with a straight root.
Six diagonal cladogram topologies were used (see figure 3). Four depicted the evolutionary relationships among five taxa, and two depicted the relationships among six taxa. Although the middle two five-taxon topologies are rotations of each other, the students were not expected to realize this, because the cladograms look so different and because they were not presented side by side with the same taxon labels. In any case, that realization would not have helped the students complete the translation task. Four cladograms were created for each topology so that each orientation and taxon ordering was used with each topology (see figure 2), thereby generating six translation problems for each cell of the design. A different set of letters was assigned to each cladogram in a set of four (see figure 2). The letter sets were selected to preserve as much as possible meaningful alphabetic sequences (Klahr et al. 1983, Taylor DA et al. 1984).
A rectangular-format cladogram was paired with each of the 24 diagonal-format cladograms. For each cladogram topology, two of the rectangular cladograms were correct translations of the evolutionary relationships from the diagonal to the rectangular format. The remaining two rectangular cladograms were incorrect translations that were modeled on common types of errors observed in a previous study in which students drew translations (Novick and Catley 2007). Figure 2 shows the diagonal-format and its associated rectangular-format cladograms for one topology. As is illustrated in the figure, the taxon labels for each rectangular cladogram were in the same order as those in the diagonal cladogram. Correct and incorrect rectangular translations were presented equally often for each of the four cells of the design (diagram orientation × taxon ordering).
The subjects viewed the cladograms presented in the center of a computer screen with their heads stabilized on a chin rest set 30 inches (76.2 centimeters [cm]) from a 19-inch (48.26-cm) monitor. The screen resolution was set to 800 × 600 pixels, with a refresh rate of 75 hertz (Hz). Eye movements were monitored using an EyeLink I head-mounted eye-tracking system (SensiMotoric Instruments, Teltow, Germany), which was spatially accurate to within 0.5 degrees (°) and had a sampling rate of 250 Hz. By convention, a saccade (i.e., a transition from fixation at one location to fixation at a different location) was classified as an eye movement if the acceleration exceeded 9500° per second and velocity exceeded 30° per second. A nine-point display was used for calibration and validation prior to the beginning of each block of 12 trials. These specifications follow the standards set by the system manufacturer (SR Research 2006) and are commonly used within the field (e.g., Green et al. 2008, Hegarty et al. 2010, Stieff et al. 2011).
The subjects participated individually in a single session that lasted approximately 60 minutes. They viewed sequential cladograms in pairs and were asked to decide whether the second cladogram of each pair depicted evolutionary relationships among the taxa that were the same as or different from those depicted in the first cladogram. The first cladogram of each pair was in the diagonal format, and the second was in the rectangular format.
To begin each trial, the subjects were required to maintain eye fixation on a central point and to press the space bar on a keyboard. The subjects were free to move their eyes once the trial began. For each trial, they were asked to view the first cladogram of the pair without time pressure and to press the space bar when they had figured out how to translate the depicted relationships into the rectangular format. Once the space bar was pressed, the diagonal cladogram immediately disappeared and was replaced by a rectangular cladogram, which the subjects were asked to evaluate with respect to whether it depicted exactly the same set of relationships among the taxa as that in the first cladogram. The rectangular cladogram remained on the screen until the subjects pressed one of two buttons to indicate the accuracy of the translation.
The subjects completed two practice cladogram pairs, followed by 24 experimental pairs divided into two blocks of 12. No feedback regarding response accuracy was given. The 24 experimental pairs were presented in one of two quasirandom orders, with approximately half of the subjects receiving each order. Fixation positions (for the diagonal cladograms) and decision accuracy (for the rectangular cladograms) were automatically recorded by the eye-tracking system.
Student comprehension experiment: Results
In this section, we describe our analysis of the eye-tracking behavior and performance accuracy of the subjects when they interpreted the cladograms in the UR and DR orientations. Fixation locations in Cartesian coordinates were recorded in screen pixels. To facilitate the analysis of directional biases, we converted the Cartesian coordinates into polar coordinates, which yielded a ray with an axis (distance) and angle (direction) for each saccade. The average axis and angle measures were computed for each subject for each of the four cells of the design. Given that there were six trials per cell and roughly 40 saccades per trial, each of the four means for each subject was based on approximately 240 data points. With the first fixation of a saccade anchored at the origin of a polar graph, the axis and angle dependent measures depict the distance and directional trends, respectively. With such plots, angles from 0° to 90° in absolute value (i.e., data points to the right of the vertical axis) represent left-to-right scanning, and angles from 90° to 180° in absolute value (i.e., data points to the left of the vertical axis) represent right-to-left scanning. Angles less than 0° (i.e., data points below the horizontal axis) represent downward scanning, whereas angles greater than 0° (i.e., data points above the horizontal axis) represent upward scanning. Therefore, data points in the top-right quadrant indicate scan paths that generally proceeded from left to right and from bottom to top. In contrast, data points in the bottom-right quadrant indicate scan paths that generally proceeded from left to right and from top to bottom. The length of each ray represents the mean saccade distance in pixels.
Accuracy was computed as the number of correct responses for each cell of the design. The maximum accuracy score was therefore 6 for each cell.
We included stimulus order in the preliminary analyses of the data from the students. As was expected, that “nuisance” factor did not affect the results either on its own or in combination with either of the manipulated factors. For this reason, order was not retained in the analyses reported here.
To evaluate directional biases in the students' processing of the cladograms, we conducted a 2 (diagram orientation) × 2 (taxon ordering) repeated-measures ANOVA (analysis of variance) on the angle measure. An alpha level of .05 was the criterion for statistical significance. The main effect of forward versus reverse alphabetical ordering of the taxon labels was not significant (F(1,18) = 3.68, p > .07, η2p = .17). The interaction of taxon ordering and diagram orientation also was not significant (F(1,18) = 0.44, p > .50, η2p = .02). Therefore, the significant main effect of cladogram orientation (F(1,18) = 24.51, p < .001, η2p = .58, Cohen's d = 1.29) can be interpreted unambiguously.
The mean angle measure for UR cladograms is +4.86 (standard error [SE] = 2.07), which is significantly different from 0 (the value that would indicate no consistent upward or downward scanning) (t(18) = 2.34, p < .05). The mean angle measure for DR cladograms is –6.07 (SE = 1.81), which is also significantly different from 0 (t(18) = –3.35, p < .01). These results indicate that the students scanned up for the UR cladograms and down for the DR cladograms, as was predicted. The overall mean angle across all diagonal cladograms was not significantly different from 0 (F(1,18) = 0.14, p > .70, η2p = .01). Therefore, there was no overall bias to scan upward or downward. As is shown in figure 4, the data strongly support the predicted left-to-right processing bias, because the angle measure for every student for both diagonal-format cladogram orientations was less than 90° in absolute value. Regardless of the diagram orientation, the students scanned the cladograms from left to right (p < .001 for a binomial test for each orientation).
The repeated-measures ANOVA on translation accuracy yielded the same pattern of results as for the angle measure. There was a significant main effect of cladogram orientation (F(1,18) = 4.45, p < .05, η2p = .20, Cohen's d = 0.36). As was predicted, the students were more successful at translating the DR cladograms to the rectangular format (M = 8.42, SE = 0.53; 70% correct) than they were at translating the UR cladograms (M = 7.58, SE = 0.58; 63% correct). Neither the main effect of alphabetical ordering (F(1,18) = 0.04, p > .80, η2p = .00) nor the interaction of the two factors was significant (F(1,18) = 0.06, p > .80, η2p = .00).
The repeated-measures ANOVA on the distance between saccades (axis) revealed no significant effects (cladogram orientation, F(1,18) = 0.25, p > .60, η2p = .01; alphabetical ordering of the taxon labels, F(1,18) = 2.29, p > .10, η2p = .11; interaction between these factors, F(1,18) = 2.01, p > .15, η2p = .10). The mean distance between consecutive eye fixations was 116.41 pixels (SE = 4.02). This value is somewhat less than the distance between taxon labels (approximately 139 pixels) and nodes (approximately 168 pixels) on the diagonal cladograms.
Because only two biology professors participated in our study, we present their data descriptively, without statistical analysis. They showed the same scanning pattern as the students: Their angle measures for UR and DR cladograms were (a) 15.15 and –4.61, respectively, for the first professor and (b) 7.64 and –5.93, respectively, for the second. Their mean distances between consecutive eye fixations were 138.78 and 127.63 pixels. These data points are on the distributions of the student data, as is shown in figure 4. One professor followed the student pattern of higher accuracy for the DR than for the UR cladograms. The other professor got all the translation questions correct but required more than 20% longer to do the mental work of translation for the UR than for the DR cladograms (means of 11.54 and 9.48 seconds, respectively).
Because cladograms are diagrams, how people process them is influenced by general factors that affect the processing of visual stimuli, as well as by specific knowledge of evolutionary biology. Our results documented two general psychological factors that influence how biology students (and professors) scan cladograms. First, there was a very strong general bias to scan these diagrams from left to right, following the highly practiced order for reading text. Second, consistent with people's interpretations of the “backbone” line of diagonal cladograms as a coherent entity (Novick and Catley 2007), this line provided a strong perceptual cue that guided the direction of visual processing. Operating on top of the general left-to-right bias, this meant that UR cladograms were processed from bottom to top, whereas DR cladograms were processed from top to bottom. Because the UR and DR cladograms were not systematically ordered in the sequence of trials, this pattern of results indicates that the subjects switched between processing the cladograms in an upward direction and processing them in a downward direction on a trial-by-trial basis, depending on the orientation of the “backbone” line of the cladogram in the current trial.
Downward processing means that branching points (nodes) are generally encountered in an order that reflects the pattern of nesting, whereas upward processing yields the opposite order. In processing the cladogram in figure 1b in an upward direction, one might be tempted to combine taxon I with taxa H and G. With downward processing of this cladogram in the DR orientation (figure 1c), however, H and G are seen to join first with F and E, which is the correct order of nesting and, therefore, reflective of the correct relationships. Consistent with the implications of these different processing directions, translation accuracy was higher for the DR than for the UR cladograms. This accuracy result is especially notable in light of the fact that DR cladograms are much less frequently encountered in textbooks (Catley and Novick 2008), so the students would have been less familiar with that format.
These results suggest that use of the diagonal cladogram format must be accompanied by instruction to students on how to read such diagrams. In particular, students should be instructed to scan the tree in a downward direction, starting with the most derived sister group (i.e., the one farthest from the root) and then progressing toward the root of the tree. Indeed, this processing strategy applies to cladograms in the rectangular format as well. Downward processing means that for the standard UR orientation of diagonal cladograms, students generally need to go against their very strong reading-order processing bias and scan the tree from right to left. Teaching students to process cladograms in a downward direction would presumably help overcome the incorrect strategy of reading across the tips of the cladogram (Gregory 2008) by encouraging the students to follow the pattern of nesting in the body of the cladogram rather than focusing attention primarily at the branch tips. More generally, it is critical that instructors not assume that diagrams transparently convey meaning (Hegarty and Stull 2012). Rather, students need instruction in how to scan and interpret each new type of diagram they encounter.
It is worth noting that even for the easier-to-understand DR orientation, translation accuracy was not high, with a mean of only 70% correct. This relatively poor performance may be explained in part by the fact that the subjects had to perform the translation from the diagonal to the rectangular cladogram format in their heads. With the external support of paper and a pencil available, their accuracy would presumably be higher. Nevertheless, the results of earlier research by Novick and Catley (2007, 2012) in which students' understanding of the UR diagonal cladogram format was compared with that of the rectangular format suggest that some of the poor overall accuracy is a function of the diagonal format itself.
The present results support our prediction that although the UR and DR diagonal cladogram formats are informationally equivalent, they are not computationally equivalent. In prior psychological studies of diagram processing, the alternative, informationally equivalent diagrams were visually distinct. That is, they looked different, either because they were composed of different parts (e.g., bars versus lines in bar and line graphs, respectively) or because the parts were arranged very differently in the picture plane (e.g., hierarchical graphs with crossed versus uncrossed lines; Körner 2004). The present study is the first to our knowledge to document a lack of computational equivalence of diagrams that are composed of the same parts arranged in the same way, differing only in orientation.
Beyond the theoretical implications of these results for understanding the psychological factors that influence how students process domain-specific science diagrams and the practical implications for teaching cladogram-processing strategies to students in biology classes, there are further practical implications for how cladograms should be depicted in textbooks and biology classrooms. We found that more than 80% of the diagonal cladograms in 27 high school and college biology texts cataloged in a recent article (Catley and Novick 2008) were printed in the UR orientation. This means that, currently, students are routinely exposed to cladograms for which it is inherently more difficult for them to correctly interpret the nested pattern of evolutionary relationships. As a result, it is much more likely that students would make incorrect interpretations of critical aspects of evolutionary history, such as patterns of character evolution and distribution, convergent evolution, and most recent common ancestry. We encourage biology educators to use the simple strategies outlined in this article to overcome such inherent but eminently fixable effects of diagram orientation.
The research reported here was supported by the Institute of Education Sciences, US Department of Education, through grant no. R305A080621 to Vanderbilt University. The opinions expressed are those of the authors and do not represent the views of the Institute or the US Department of Education. We thank Ted Hzu for help with computing the eye-movement variables and Kristopher Preacher for consultation on statistical issues.