The Stockman’s Scorecard: quantitative evaluation of beef cattle stockmanship

Abstract An animal’s action, or inaction, is the direct result of a stockman’s action or inaction. The Stockman’s Scorecard is a novel observation instrument that has been proven to be a valid and reliable tool to measure the quality of beef cattle stockmanship. Specific handler actions have been weighted based on their perceived negative relationship to cattle stress from handling. The purpose of this article is to 1) document the initial use of the scorecard in a beef cattle feedlot setting and 2) provide further support to its validity by establishing an association with other quantitative and qualitative means of evaluating stockmanship. The Scorecard was used at 39 beef feedlots in Texas between March 2018 and April 2019. Eighty-four stockman were observed, and the average score received was 84.5 (SD = 14.73, range = 20 to 100). The most frequent mistakes observed were as follows: fills crowd pen/tub over half full (n = 39), slow to remove pressure (n = 29), uses unnecessary noise (n = 25), stands in front and taps rear (n = 24), and fails to regulate animal flow through a pinch point (n = 22). A strong negative association (ρ = −0.51) was found between the points deducted from the Noise and Physical Contact theme of the Scorecard and the number of animals touched with an electric prod from the BQA Feedyard Assessment. Moderate negative associations were found between the Scorecard final score and the number of animals that vocalize in the chute prior to procedures (ρ = −0.31). Those stockmen that scored above average on the Scorecard were qualitatively observed to be calm and quiet while working with the cattle (Kappa = 0.44). The qualitative disposition of cattle had little effect on the final score of stockmen using the Scorecard (Kappa = 0.17). The use of the Scorecard in a feedlot setting has demonstrated that as stockman scores decrease, there is an increase in the number of negative actions toward cattle and a negative behavioral response of the cattle themselves. Establishment of an association between a stockman’s score using the Stockman’s Scorecard and the animal-based observations from the BQA Feedyard Assessment further strengthens the validity of the Stockman’s Scorecard as a tool to measure the quality of beef cattle stockmanship. The Scorecard has application as a tool to identify specific stockmanship deficiencies in order to target stockmanship training.

ABSTRACT: An animal's action, or inaction, is the direct result of a stockman's action or inaction. The Stockman's Scorecard is a novel observation instrument that has been proven to be a valid and reliable tool to measure the quality of beef cattle stockmanship. Specific handler actions have been weighted based on their perceived negative relationship to cattle stress from handling. The purpose of this article is to 1) document the initial use of the scorecard in a beef cattle feedlot setting and 2) provide further support to its validity by establishing an association with other quantitative and qualitative means of evaluating stockmanship. The Scorecard was used at 39 beef feedlots in Texas between March 2018 and April 2019. Eighty-four stockman were observed, and the average score received was 84.5 (SD = 14.73, range = 20 to 100). The most frequent mistakes observed were as follows: fills crowd pen/tub over half full (n = 39), slow to remove pressure (n = 29), uses unnecessary noise (n = 25), stands in front and taps rear (n = 24), and fails to regulate animal flow through a pinch point (n = 22). A strong negative association (ρ = −0.51) was found between the points deducted from the Noise and Physical Contact theme of the Scorecard and the number of animals touched with an electric prod from the BQA Feedyard Assessment. Moderate negative associations were found between the Scorecard final score and the number of animals that vocalize in the chute prior to procedures (ρ = −0.31). Those stockmen that scored above average on the Scorecard were qualitatively observed to be calm and quiet while working with the cattle (Kappa = 0.44). The qualitative disposition of cattle had little effect on the final score of stockmen using the Scorecard (Kappa = 0.17). The use of the Scorecard in a feedlot setting has demonstrated that as stockman scores decrease, there is an increase in the number of negative actions toward cattle and a negative behavioral response of the cattle themselves. Establishment of an association between a stockman's score using the Stockman's Scorecard and the animal-based observations from the BQA Feedyard Assessment further strengthens the validity of the Stockman's Scorecard as a tool to measure the quality of beef cattle stockmanship. The Scorecard has application as a tool to identify specific stockmanship deficiencies in order to target stockmanship training.

INTRODUCTION
The behavior and actions of stockmen has a direct effect on the behavior and welfare of livestock (Zulkifli, 2013). The result of this humanlivestock interaction is dependent on the attitudes Translate basic science to industry innovation and behavior of the stockperson (Waiblinger et al., 2002). Behavioral research in beef cattle (Petherick et al., 2009a;Probst et al., 2013), dairy cattle (Rushen et al., 1998;Waiblinger et al., 2003), and swine (Tallet et al., 2014) has shown that an animal's response is dependent on the quality of treatment received from their human handlers. In dairy cattle (Passille et al., 1996;Munksgaard et al., 1997), beef cattle (Boivin et al., 1998), and sheep (Boivin et al., 1997), there is support that livestock may be able to differentiate between handlers based on their familiarity with the stockman and the quality of the stockperson's handling. Also, it has been shown in pigs that group behavior was altered when a single pen mate was subjected to negative handling practices although the others of the group did not receive the treatment (Reimert et al., 2017). Beef cattle will habituate to common handling practices and human contact by frequent exposure (Matson, 2006), especially at a younger age (Fukasawa, 2012;Etim et al., 2013). However, livestock will not habituate to painful procedures and adverse handling practices (Grandin et al., 1986).
Livestock handling involves both the restraint of animals, and encouraging a desired movement, in a way that minimizes fearful reactions (Gonyou, 1995). Stockmen are encouraged to be calm, quiet, slow, and deliberate when working animals (Grandin, 2015, 65-95). Furthermore, stockmen need to understand the behaviors of cattle, and their physiology, in order to take advantage of their natural prey instinct when herding (Grandin and Dessing, 2008). Evaluation of stockmanship is a critical component in assuring positive animal welfare (Grandin, 2001;Grandin, 2014). Assessment of stockmanship involves the observation of animal behaviors and quantitative measurements of their temperament. Chute scoring, chute exit speed scoring, vocalization tests, and aversion tests are all measures to evaluate the overall treatment of cattle (Grandin, 1994;Grandin and Shivley, 2015). The livestock industry has been proactive in assessing the care of livestock at the farm and processing levels through facility evaluations such as the BQA Feedyard Assessment (2017), the North American Meat Institute Audit (2019), and the European Welfare Quality Audit (Welfare Quality Network, 2009). As general themes, the assessments seek to discover whether appropriate management protocols are in place to insure the implementation of scientifically based, industry-recognized, best management practices. Within these evaluations, highly reliable, animal-based measurements are utilized to determine the quality of stockmanship (Grandin, 2015, 69 to 95). Specifically, the BQA Feedyard Assessment asks that 100 head of cattle be observed to determine the number of cattle that are touched with an electric prod, fall upon release from the chute, stumble/trip when released from the chute, vocalize in chute before procedures, jump or run when released from the chute, or miscaught and not readjusted while in the chute.
Although these measurements are appropriate to assess improvements in stockmanship within an operation (Rushen and De Passille, 2015), how are we to determine what stockperson actions caused any aberrations identified in these animal observations? The argument has been made that the human factor may strongly influence audit results (Rocha et al., 2016). Coleman and Hemsworth (2014) are quoted as saying, "While welfare monitoring schemes are likely to improve animal welfare, the impact of such schemes will only be realized by recognizing the limitations of stockpeople, monitoring stockmanship and providing specific stockpersons training to target key aspects of stockmanship." Gonyou (1995) stated, "The most important part of a livestock handling system are the persons who handle the animals and operate the facilities and equipment". He goes on to say, "the potential of well-designed facilities and equipment will only be realized if the stockpersons use them properly." The Stockman's Scorecard is an evaluation instrument designed to measure the quality of beef cattle stockmanship. The scorecard has previously been proven to be a valid and reliable tool for assigning a numerical score to the stockmanship abilities of cattle handlers (Yost et al., 2020). The card provides 30 observation points, identified from other published works, which can be interpreted as producing either a positive or negative animal behavior outcome. The observation points have been grouped into three categories identified as Situational Awareness, Herding Skill, and Noise/ Physical Contact. The Situational Awareness category includes 13 observation points meant to assess if the stockman can function as a member of the animal handling team, if they know the capabilities of the handling facility to properly move animals through gate openings and other pinch points, if they know the proper number of animals to load into the crowd tub, and if they avoid attempting to work in the animal's blindspot. The Herding Skill category groups seven observation points that seek to evaluate whether the handler understands how to use an animal's flight zone, and point of balance, to produce positive animal movement. The Noise/Physical Contact section contains 10 observation points that evaluate the handler's use of vocal and artificial noise during the handling activity, as well as if the handler is properly using electric prods or physical force to encourage animal movement. Each stockman begins an evaluation with 100 points. The observer deducts the specified points for each negative action performed by the subject. At the end of the evaluation, the total deductions are determined and subtracted from 100 to establish a final score. The purpose of this paper is to 1) document the initial use of the scorecard in a feedlot setting and 2) provide further support to its validity by establishing an association with other quantitative and qualitative means of evaluating stockmanship.

An Institutional Animal Care and Use
Committee protocol was not required for this study. Three assessment tools were utilized for this study. They are as follows: the Stockman's Scorecard (Yost et al., 2020); the BQA Feedyard Assessment (2017), which has six handling measures; and a qualitative scale describing the animals' and handlers' dispositions. To develop the qualitative scale, the observer was asked to use their own words to briefly describe the disposition of the cattle and the stockman. For a stockman, they could use words such as calm, angry, hurried, or nervous. For the cattle, they could use words such as calm, stubborn, flighty, or riled up. The handler and livestock disposition determinations were qualitatively evaluated by the researcher and condensed into themes. The themes were then coded to create a disposition scale. The coding for the stockman scale was as follows: 1 = calm/quiet, 2 = calm plus another descriptor, 3 = fast/rushed/excited, 4 = nervous/unsure/frustrated. The coding for the livestock scale was as follows: 1 = calm/quiet, 2 = slightly jumpy, 3 = excited/jumpy/wound-up, 4 = stubborn/hesitant. For analysis, codes were combined to create a nominal variable scale (handler, 1 = calm/quiet, 0 = other descriptor; livestock, 1 = calm/quiet, 0 = other descriptor), and Kappa was calculated with JMP (ver. 25) to determine the level of agreement between Scorecard score and the handler and livestock disposition scales.
Data collection was conducted through a cooperation with the Texas Cattlefeeders Association, Feedyard Services Division. Division personnel regularly conduct BQA Feedyard Assessments for member feedyards, and each employee conducting the audits has been certified by the Professional Animal Auditor Certification Organization (PAACO, paaco.org). All feedyards used in this study agreed to the use of the Stockman's Scorecard during a normally scheduled Assessment. The observers were provided with an observation instrument that included the Stockman's Scorecard and the animal-based observations recording component of the BQA Feedyard Assessment. Prior to any data collection, the observers were provided a narrated PowerPoint presentation that detailed the methodology of the scorecard and its use. The PowerPoint presentation provided an explanation of each observation point included on the Scorecard, and examples of situations that represent the inclusion of the observation point. Once the materials had been reviewed, a conference call was held with the primary researcher and the observers to explain the intent of the evaluation, the desired data to be collected, and to answer any questions or provide clarity on the methodology and use of the card.
Data collection occurred over the period of 1 yr (March 2018 to April 2019). The Scorecard was used to evaluate 86 stockmen from 39 cattle feedyards in Texas. Nine facilities were visited once, 19 facilities were visited twice, 6 facilities were visited three times, 4 facilities were visited four times, and 1 facility was visited five times. All subjects evaluated were stationed between the crowd pen/tub and the chute. The observers were asked to evaluate one to two employees at each facility, but not the same employee if the facility was visited on multiple occasions, using the scorecard as they were conducting a normally scheduled BQA Feedyard Assessment. The observers evaluated each subject using the scorecard criteria and collected the animal observation data on a maximum of 100 head through the handling system. Completed scorecards were scanned by computer and stored as PDF files to be emailed to the researcher. Once received, the individual scorecard results were entered into an Excel spreadsheet. The data for each observation point were recorded as a "zero" or a "1." If an action, on the part of the stockman, was observed, it was recorded as a "1." All unobserved observation points were recorded as a "zero." Frequencies and standard deviations were determined by analysis with Microsoft Excel. Spearman's rho correlation to determine the associations between the Scorecard, and BQA Feedyard Assessment results were performed with JMP (ver. 25). For the Spearman's correlation analysis, a Benjamini-Hochberg adjustment was used with a 10% false discovery rate used in the calculation. Statistical significance was set a priori at α = 0.05.

Quantitative Evaluation of Stockmanship
The average Stockman's Scorecard score received was 84.5 (SD = 14.73, range = 100 to 20). Forty-five percent of the stockmen observed (n = 39) received a perfect score or were documented to have performed one to two actions that would deduct points (Figure 1). The most frequent mistakes observed were as follows: fills crowd pen/tub over half full (n = 29), slow to add/remove pressure (n = 27), uses unnecessary noise (n = 25), stands in front of the animal and taps on rear (n = 24), and fails to regulate animal flow through a pinch point (n = 22; Table 1). In addition, other common mistakes were when the stockmen unintentionally worked in an animal's blindspot (n = 18) and were observed to be constantly, and unnecessarily, screaming or yelling at the cattle (n = 13).
In other studies that have documented stockman actions toward beef cattle, there has been a high level of variability between operations and individual stockmen (Hultgren et al., 2013;Ligon, 2014;Simon et al., 2016;Destrez et al., 2018). In all cases, cattle that were subjected to increased intensity of human vocalization and physical contact were also perceived as more difficult to move through the handling system. Beef cattle stockman should make a conscientious effort to handle cattle in a way that stress is minimized. Aversive handling practices induce significant fear in cattle, which can cause serious losses in productivity, increased handling problems and related injuries to both animals and handlers, and diminished animal welfare (Rushen et al., 1999). Specific cattle handling recommendations have been provided in published research (Grandin, 2008;North American Meat Institute, 2019;Grandin, 2015). Elevated stress has been shown to be caused when handlers scream and yell, crack whips, generate metallic noise by banging on gates, run at the animal, and aggressively hit cattle (Waynert et al., 1999;Grandin, 2008;Woiwode et al., 2016a).
Stockmanship assessments were also conducted for the facility during scheduled BQA Feedyard Assessments. The Assessment uses six animal-based observations to determine the quality of stockmanship. For each observation point, thresholds have been established to determine whether the facility "passes" or "fails" on cattle handling (Figure 2). Of the 39 facilities visited, 24 (61%) failed on one or more categories, on one or more visits. These 24 yards were visited a total of 53 times during the sampling period, and there were 30 documented failures. Six of these facilities were only sampled once, two feedyards failed on all visits, and the remaining 16 passed on at least one of their other sampling dates. The most frequent cause of a failure was the use of electric prods (20%), stumble/tripped when released from the chute (9%), and miscaught in the head chute and not readjusted (6%; Table 2). The number of facilities that failed our animal handling assessment is higher than other reported observations (Barnhardt, 2015;Woiwode et al., 2016b). The differences may be due to the fact that most of the yards we sampled were visited multiple times during the study period, instead of a single observation as in the other studies.
The average Scorecard score for facilities that passed the BQA Feedyard Assessment was 90.0,  whereas the facilities that failed on the animal handling component received a final score of 74.3 (P < 0.0001). For those facilities that failed the handling portion of the assessment, the most frequent mistakes observed were as follows: fills crowd tub over ½ full, slow to add remove pressure, used electric prod as the primary driving aid, applied the use of an electric prod at the wrong time, and used excessive physical contact. Likewise, for those facilities that passed the animal component of the assessment, the employees were observed to fill the tub ½ full or less, regulated the flow of animals through a pinch point, demonstrated effective use of flight zone pressure, and utilized appropriate physical contact.
Several negative associations were found between a subject's score on the Scorecard and the animal-based measurements collected with the BQA Feedyard Assessment (Table 3). A strong negative association (Robinson et al., 1991) was found between the number of animals touched with an electric prod and the subjects score on the noise and physical contact section (ρ = −0.51). This high association should be expected as both tools collect a similar measurement. Points are lost in the noise and physical contact theme and deducted from the stockman's final score if an electric prod is used excessively or if contact is applied at the wrong time. The Assessment asks the observer to count the number of animals that are touched with the prod. Moderate negative associations (Robinson et al., 1991) were found between the use of electric prods and the Situational Awareness (ρ = 0.31) score and Final Score (ρ = −0.43) on the Scorecard. Also, moderate negative associations were found between the number of animals that vocalize in the chute prior to procedures and the final score (ρ = −0.31) and herding skill (ρ = −0.31) section on the scorecard. Grandin (1998) has identified animal vocalization as a key indicator of stress from adverse handling practices. She observed that skilled handlers averaged 4.5% animal vocalizations where plants with aggressive handling approached 22%.

Qualitative Description of Stockman and Livestock Disposition
The observers were asked to provide a one word, or short phrase, description of the handler's and the livestocks' disposition. The majority of stockmen were described as being calm (n = 60; Table 4). There was an additional seven stockmen that were described as calm, but the observer also documented that they seemed rushed or were noisy. On 15 evaluations, the handlers were only described as being noisy, rushed, excited, jumpy, nervous, or frustrated. When describing the cattle being processed, 30% of the groups were categorized as being calm, whereas many groups were observed to be "slightly jumpy" (n = 16) or "excited/ wound up" (n = 34). A small number of the groups (n = 6), usually Holstein cattle, were described as being "stubborn." There was a moderate level of agreement (Stokes et al., 1995) between the qualitative description of  the stockman's behavior and their final score using the Scorecard (Kappa = 0.44, P < 0.0001). Those stockmen that were observed to be calm in their actions tended to have a higher final score than those that were described as noisy, rushed, jumpy, nervous, or frustrated. A very slight agreement (Stokes et al., 1995) was found between the stockman's final score and the livestock disposition descriptor (Kappa = 0.18, P = 0.01). In 43.9% of the cases where the livestock were described to have a negative disposition, the stockman scored high on the scorecard, and in 3.6 % of the cases, the livestock were described as "calm," but the stockman received a low score. Significant correlations have been found between stockman behavior, animal behavior, and animal productivity (Hemsworth et al., 2002;Waiblinger et al., 2002;Ellingsen et al., 2014). Livestock that are handled in a calm manner tend to behave calmer and have higher productivity than those that are handled more aggressively. We observed that there was a negligible association between handler score or disposition and animal behavior. The expressed behavior of cattle is related to a combination of environment, genetics, and handling factors (Grandin, 1994;Grignard et al., 2001). Cattle may initially react negatively to any handling practice but can habituate over time (Petherick et al., 2009a,b), although they will not habituate to extremely adverse handling practices (Grandin et al., 1986). We were not able to observe every stockman involved in the handling activity, nor did we collect data on the age of the cattle and their time at the feeding facility. Repeated interactions with humans have shown to reduce reactivity of cattle in a feedlot setting (Doyle, 2014). It is also believed that cattle can differentiate between handlers that treat them poorly and handlers that are gentle (Munksgaard et al., 1997).

CONCLUSIONS
In order for an evaluation tool to be useful to measure the underlying construct it needs to be determined if it is valid and reliable. The Stockman's Scorecard has been previously determined to be both valid and reliable in measuring the quality of stockmanship. This article has further strengthened the tool by establishing the criterion-related validity of the instrument (Huck, 2012, 83). To establish this type of validity, the new instrument is compared with current accepted measurement tools. The established associations between Scorecard's results and animal-based observations from the BQA Feedyard Assessment provide the criterion-related validity. Furthermore, we have been able to provide an association between an individual score and the stockman's behavior. The slight associate of the Scorecard results with a simple qualitative description of the cattle's behavior implies that the score received by the individual stockman was independent of the behavior of the livestock.
The BQA Feedyard Assessment is a proven, reliable instrument used to evaluate stockmanship at the facility level. It allows managers to see the progress of stockman training to reduce animal stress and increase operational efficiency. The Stockman's Scorecard now provides an additional resource to reinforce gains made through livestock handling training programs or to determine who may be the cause of deficiencies and establish targeted training programs to improve a handler's stockmanship. This tool has multiple applications. It may be used in a pretest/post-test format for educators to evaluate stockmanship training. It can be used by researchers to precisely define the stockmanship parameters of their animal handling studies. Future research should focus on evaluation of all stockmen involved in an animal handling activity to determine whether a specific stockman can be identified as the cause of handling aberrations. There is also the opportunity to begin to determine the physiological effects of precise adverse handling conditions on animal outcomes.
Conflict of interest statement. The authors disclose that there was no conflict of interest.