Governance in Motion: Service Provision and Child Welfare Outcomes in a Performance-Based, Managed Care Contracting Environment

Abstract

Examining the linkage between service provision and client outcomes is important in performance-based human service environments. Since most performance initiatives reward agencies for improving client outcomes rather than providing specific services, managers may have incentives to streamline workers’ efforts and reduce resources devoted to services considered nonessential. This article uses data from a performance-based child welfare initiative to examine the relationship between child permanency outcomes, the services provided by caseworkers, and the environment surrounding frontline service provision. Findings indicate that greater service efforts are required to reunify children with parents than to reach other outcomes, including adoption and placement with relatives. They also indicate that formal organizational responses to performance environments affect client outcomes partially through the services provided by frontline workers. These findings suggest that managers should attend to the interplay between the organizational environment, service technology, and client outcomes when designing performance-based systems in the human service sector.

INTRODUCTION

Since the inception of the New Public Management movement, child welfare agencies have been characterized as being inefficient, ineffective, and unresponsive to client needs (Tilbury 2004). In response to these concerns, policymakers have sought to reorganize child welfare programs to improve performance, which is often measured by the number of foster youth who exit the foster care system. In reaction to the Adoption and Safe Families Act of 1997 (PL 105-89) and performance-focused state laws, child welfare systems have experimented with various forms of privatization, performance contracting, and managed care to move children out of state custody without compromising their safety or well-being (Flaherty, Collins-Camargo, and Lee 2008; James Bell Associates 2007). These privatization and performance management initiatives have supplied child welfare agencies with financial incentives to increase the number of foster children who attain permanency (usually through parent-child reunification or adoption) within preset time periods.

These initiatives focus attention on how human service agencies and managers improve client outcomes in performance-based environments. In contrast to the image of the Weberian command-and-control bureaucracy that organizes and implements new programs incrementally, public and private human service agencies may initiate substantial, discontinuous programmatic changes in response to performance mandates (Frumkin 2001; Johnston and Romzek 2008). The responsiveness of agencies to performance-based conditions may be understood as a process in which frontline service provision is nested in an agency milieu that is affected by institutional factors (Lynn, Heinrich, and Hill 2001). The logic of hierarchical governance suggests that performance initiatives affect the administration and structure of service programming, which shape interactions between caseworkers and service recipients, thereby altering client outcomes. In human service programs, frontline workers have the principal responsibility for assessing, serving, and monitoring clients. The performance of human service programs may therefore depend upon how frontline work is structured: if agencies prepare staff to operate in performance-based situations, and if these workers serve clients appropriately, then client and programmatic outcomes may be expected to change and hopefully improve.

This article examines the relationship between service provision and client outcomes in performance- and nonperformance-based child welfare settings. This relationship might be expected to be stronger in performance-based environments if agencies respond to performance incentives by altering service delivery and job functions in a manner that is specifically designed to improve outcomes. Yet there are reasons to question the tenability of this scenario. Agencies may respond differently to technical uncertainty, which occurs when providers are unsure about which services are essential to improve outcomes and how, when, and by whom these services should be delivered. Even when an agency is aware of an effective service strategy, frontline workers may respond to performance mandates by serving clients in contrary, novel, and unexpected ways (Brodkin 2007; Sandfort 2000). When managers cannot identify services thought to improve outcomes or implement effective service strategies faithfully, it is difficult to establish the conditions for cost-effectiveness and have sufficient information for make-or-buy decisions, and thus to organize an optimal state of agents.

The extant literature has not directly modeled the relationship between human service provision and client outcomes, in part due to the difficulty of associating client-level changes in service activity and program outcomes. Controlled studies have come from the welfare-to-work service arena (Bloom, Hill, and Riccio 2003; Dyke et al. 2006; Hill 2006). These studies have examined the influence of different frontline job structures as opposed to the specific tasks carried out by frontline workers, under the scientific management premise that changes in staff training and job demands influence interactions between workers and clients and, ultimately, worker and unit performance (Taylor 1911). From the perspective of welfare-to-work implementation studies, however, such structural changes may not fully shape frontline service production or client behavior, since they present opportunities and constraints that may be acted upon by some frontline workers but not others (Cooney 2007; Sandfort 2003).

In addition, the conceptualization of “success” in relation to client outcomes and program performance in performance- and nonperformance-based environments has gone largely unexamined. Frontline human service work often involves deciding between mutually exclusive and morally ambiguous outcomes (Hasenfeld 2000). Placing a welfare recipient into a job may reduce the length of time that she can receive cash and other benefits; placing a foster child in a relative's home may reduce the likelihood of parent-child reunification. The manner in which agencies and workers move towards different client outcomes, and the crowding out that may occur when one outcome is privileged over others, remain unobserved when linked program outcomes are studied in isolation.

This study draws from a rich dataset from a natural experiment of a performance-based child welfare contracting initiative in Wayne County (Detroit), Michigan, and examines questions about frontline service effectiveness in performance- and nonperformance-based arenas. Two essential types of service production are examined: providing clients with in-agency, nontherapeutic services; and linking clients with community-based supports. Questions addressed include whether greater service-related efforts are required to achieve different outcomes, whether the crowding out of certain outcomes occurs in performance environments, and whether organizational responses to performance environments are expressed through the services that are provided by frontline workers.

This article is organized in four sections. The first posits explanations for the relationship between service provision and specific client outcomes and employs adaptive structuration theory to explain the association between organizational changes in response to performance demands, service production, and client outcomes. The interior sections describe the data from the study of performance contracting and the results of analyses. The article concludes with a discussion of theoretical and substantive directions for future research on the utility of performance management in improving human service provision and contracting.

THE RELATIONSHIP BETWEEN SERVICE PROVISION AND CHILD WELFARE OUTCOMES

Child welfare outcomes have been traditionally assessed in relation to the hierarchy of permanency, in which parent-child reunification is the preferred destination for children placed in foster care, and adoption is favored in situations where reunification is not appropriate or possible. This hierarchy has been altered as a result of a mandate within PL 105-89 that requires states to expedite permanency planning. Caseworkers are required to make reasonable efforts to reunify children and parents and must move to terminate parental rights if reunification has not occurred by the time a child has been in care 15 of 22 months, unless sufficient services have not been marshaled to support reunification planning or if the child has been placed in the care of relatives. Due in part to this “relative clause,” kinship care has become an increasingly common placement option and child welfare system performance measure, despite evidence that children placed with relatives take longer to reunify with parents, are less likely to be adopted, and receive fewer services and financial support than children placed with nonrelative caregivers (Berrick 2009; Cuddeback 2004).

State child welfare system architects have used performance contracting and managed care to tie child welfare financing to permanency outcomes. It has been assumed that the use of these approaches will lead to cost savings and improved performance through higher rates of achievement of reunification and adoption, less time to achieve permanency, fewer replacements while in care, lower rates of recidivism upon exit from care, and a reduction in the number of youth in foster care. Performance-based models are generally thought to improve these outcomes by penalizing agencies that fail to reach performance targets and rewarding those that do, assumedly by limiting ineffective and inefficient service provision.

There are two mechanisms through which managers in performance-based environments may seek to influence child welfare outcomes. The most direct mechanism is through the actual services provided to children and families in foster care. In attempting to move children into permanent placements, child welfare caseworkers may provide case planning and case management, supervised parent-child visits, and links to specialized services in the community, among other services. This article focuses specifically upon these services, which have been associated with increased odds of reunification (Children’s Bureau 2006; Miller et al. 2006).1 While no study has identified the independent effect of these activities on adoption or kinship care, it is likely that they influence these outcomes differently.

The second mechanism that managers may use to influence permanency outcomes is the foster care technical and task environment, which supplies the organizational context in which caseworkers serve clients. Structural aspects of the foster care technical environment include: workforce-related factors, including the type and number of employees within foster care departments, staff roles and responsibilities, and the provision of training to teach and reinforce work tasks; and foster care service technology, defined as the agency-controlled means of production, usually involving assessment of client needs and strengths, development of a personalized service plan, coordination of services on behalf of the client, and the supervision of case-based activities (Hasenfeld 1983). Evidence suggests that agencies reorganize their staff structures to manage the heightened financial risks present in performance-based environments. These adaptations include an emphasis on cost-effective service programming as well as increasing the speed of frontline decision making (McBeath and Meezan 2006; Provan, Milward, and Isett 2002). Because time is precious in these environments, agencies have financial incentives to initiate quick and accurate assessment procedures that result in immediate service provision leading to the movement of clients towards desired outcomes.

This article uses structuration theory (Giddens 1984) to examine the relationship between performance-based organizational change, service production, and client outcomes, and to provide a theoretical framework explicating how human service technology may be linked to positive client outcomes in performance-based environments. It draws, in particular, upon adaptive structuration theory (DeSanctis and Poole 1994), which describes the effects of changing organizational technology more clearly than the classic statement of structuration theory.

Adaptive structuration theory was developed to understand how groups use advanced information technologies to facilitate interpersonal communication and decision making as well as why groups with access to the same technology experience different outcomes. DeSanctis and Poole (1994) emphasize that a given technology contains formal properties concerning its operational design elements as well as identity-based properties that pertain to the normative values, goals, and meanings attached to its use. As individuals within a group interact with the technology, they gain information about its appropriateness for specific tasks within the organizational environment. Additionally, in response to emerging organizational concerns, a group may adapt a specific technology for use in a manner unrelated to its original capabilities and intention. This process of adaptation is reflexive and recursive: individuals come to understand, use, and modify a technology to suit their purposes through an iterative process in which they question and/or develop new structures, beliefs, and resources by using this technology. The effect of a specific technology thus depends not only upon its structural and identity-based features but also upon how, by whom, and for what purpose it is used.

A number of propositions result from applying the theory to the issue of child welfare outcomes in performance-based contexts. First, services provided to a child and family need not be associated with either the permanency outcome desired or achieved. For example, a service may be delivered despite the worker being aware it is inappropriate for a particular child and family, either because it is expected or mandated or because it is available but not designed to respond to a specific, identified client need. Other conditions that may alter the effectiveness or efficiency of a service include the sufficiency of organizational resources and staff expertise in administering the service, the degree to which the service is implemented as intended, and caseworkers’ emergent responses as they deliver the service to the client population. However, under exacting service production conditions—where appropriate services are delivered by appropriate staff in appropriate environments—positive client outcomes may be expected.

The second proposition underscores the goal, role, and task ambiguity present throughout human service work. Performance-promoting structures may influence client outcomes by reducing ambiguity related to technical uncertainty among frontline staff. Technical uncertainty in the human services may be understood to be a function of three different types of ambiguity: which client outcomes are desired; which service technologies should be employed to attain these outcomes; and the manner in which implementation ought to occur. All else being equal, the linkage between service provision and program performance may be strengthened through initiatives that clarify the goals of service provision and the pathways through which performance targets may be achieved and that train and provide resources for caseworkers to carry out the specific job functions needed to deliver such services.

Third, the meaning ascribed to and utility of a service technology should grow as it is incorporated into dominant organizational practices. In this situation, intensity of service utilization may signify the legitimation of a preferred path for achieving organizational goals, which may or may not correspond with performance targets.2 Classic structuration theory suggests that individuals search for and participate in the creation of organizing schemas (i.e., norms and myths) in situations of organizational and moral ambiguity. These schemas become institutionalized as they become attached to and influence the development of resources, rules, and lines of authority (Sandfort 2000, 2003). This process of structuration is not monolithic but is constituted through interwoven processes of conflict and consensus, since structures may be contested within and across organizational settings. In human service systems, there may be different expectations among workers and agencies as to which client outcomes should be preferred and which services should lead to these outcomes (Cooney 2007). Performance pressures may exacerbate this dissonance regarding which clients are meritorious and how these clients should be served (Hasenfeld 2000). Evidence of changing patterns of service production or nonproduction due to performance-based environments may therefore act as a beacon for a shift in deep-rooted organizational beliefs concerning what constitutes successful client outcomes and how these goals should be realized.

Considerable substantive and methodological effort is needed to examine these propositions. Linking client outcomes to service production in performance-based child welfare environments requires information on permanency outcomes and essential agency- and community-based services over the duration of clients’ system involvement. Additionally, the covariation between service provision and the organizational context, which encompasses agency-based differences in the client population and worker milieu, must be evaluated across performance-based and nonperformance-based environments. This field-specific process of measurement and model development questions the validity of testing the effectiveness of performance management systems by comparing group outcomes for clients in performance-based versus nonperformance-based environments. Our reading of adaptive structuration theory suggests a more nuanced examination of the manner in which performance initiatives (and specific structures derived from these initiatives) interact with particular types of services to produce different client outcomes.

METHODS

This study examines the relationship between service provision and permanency outcomes for a sample of 243 foster children and families who were served by nine nonprofit agencies in Wayne County (Detroit), Michigan from 2001 to 2004.3 Three agencies contracted with the State of Michigan Department of Human Services (then called the Family Independence Agency [FIA]) under a fee-for-service reimbursement system in which agencies were reimbursed for their service-related expenses on a per-child, per diem basis. The other six agencies contracted with FIA under a performance-based, managed care reimbursement system that contained a lower per diem rate and financial bonuses for moving foster children into permanent placements within specified time periods and for sustaining these placements (the “pilot” initiative).4 The fiscal structure of the pilot and nonpilot systems is presented in table 1.

Table 1

Per Child Reimbursement Structure of the Pilot Initiative

	Nonpilot ($)	Pilot ($)
Base Compensation
Payment upon intake into pilot	0	2,210
Per diem rate	18–34	13.20
Performance Incentives
Attainment of performance standard (either the child's return home or to a relative, or achievement of legal guardianship or independent living in 290 days; or achievement of termination of parental rights in 515 days)	0	1,900
Child at home 6 months after initial discharge	0	1,290
Child at home 12 months after initial discharge	0	1,600
Adoption placement made within 7 months of termination of parental rights	0	1,290

Research Design

This study took advantage of the conditions for a natural experiment to isolate the effect of the pilot initiative from case-level covariates. Nonprofit child welfare agencies, which provide care for approximately 85% of the foster children and families in Wayne County, receive all their foster care cases through the Family Assignment System (FAS). Under the FAS, a child entering foster care for the first time is assigned to the next agency in the alphabetical queue of service providers. If an agency is at full capacity, the next agency in the queue is contacted, and if a space is available, the child is placed with that service provider. As a result, foster children in Wayne County are assigned to nonprofit agencies on a rotating basis that is unrelated to case characteristics. Due to the FAS, random assignment equivalence was initially presumed and later demonstrated to exist between children and families in pilot and nonpilot agencies (McBeath and Meezan 2008; Meezan and McBeath 2008). Thus, comparisons between the performance-based (pilot) versus fee-for-service (nonpilot) contracting environments are derived from the equivalent of a randomized, posttest-only control group design.

Data

In order to examine the placement experiences of children in the pilot contracting environment, 243 foster children served by the six pilot agencies (n = 175) and three nonpilot agencies (n = 68) were identified between May and October of 2001.5 Information on the characteristics of these children and their families was collected once, 30 days after the child's entry into foster care. Information concerning the whereabouts of the children, the services provided to them and their families, and the characteristics of agency caseworkers serving them was collected at 30 days and then every 90 days thereafter. Data were gathered through reviews of case files, which contained reports from Child Protective Services, FIA-required assessments and service plans, and regularly updated case notes. Reliability checks of these data were completed regularly by project staff, and when inconsistencies surfaced between caseworkers and project staff, the data were reviewed and appropriate corrections were made to ensure consistency. Data collection stopped either when the court terminated FIA's supervision of the family or at 930 days in care.

Information on changes in the pilot agencies’ foster care technical and task environments that were made in the transition to the performance-based contracting environment was gathered through 42 telephone interviews completed between August and December 2001 with pilot agency administrators and supervisors. (For a description of the methods used to gather and analyze these data, see McBeath and Meezan 2006.) Thematic coding of these interviews allowed for identification of whether any respondent from a particular pilot agency identified a specific change in essential foster care structures and/or processes in response to the initiation of the pilot initiative. This information was used to construct dummy variables identifying which agencies made specific changes in foster care training and programming in transitioning to the pilot contracting environment. This procedure resulted in a data file containing information on: organizational adaptations to the specific performance-based contract; youth, caregiver, and caseworker characteristics; services provided to the foster children and their families; and permanency outcomes.

Dependent Variable, Essential Covariates, and Controls

The principal variables under examination are described in table 2. Five mutually exclusive categories operationalize the permanency outcome that each foster child reached by the end of the study: reunification refers to whether the child had been returned to his or her biological parent(s); kinship care pertains to whether the child had been placed with a relative caregiver; termination of parental rights (TPR) refers to whether the child's biological parents’ parental rights had been terminated but the child had not yet been placed in an adoptive home; adoption refers to whether the child had been adopted or had been placed in an adoptive home and was awaiting adoption finalization; and no permanency achieved refers to children who remained in care at the end of the study without being placed in any of these situations.

Table 2

Measures and Sample Characteristics

Variables	Operational Definition	Full Sample	Pilot Only
Permanency Outcomes
None	Child remained in foster care	8%	8%
Reunification	Child returned to biological parent(s)	33%	28%
Kinship care	Child placed with relative caregiver	15%	18%
TPR	Parental rights of child's biological parents terminated but child not adopted	21%	19%
Adoption	Child had adoption finalized or was awaiting adoption finalization	24%	27%
Essential Covariates
Pilot status	Child served by pilot organization	72%	100%
Trained staff	Child served by pilot organization that had trained staff to expedite permanency planning for foster youth	50%	69%
Nontherapeutic services	In-agency visitation and case management-related activities provided to foster child and family per day (α = 0.65)	0.18 (0.12)	0.16 (0.12)
Community services	Community services provided to foster child and family per day	0.02 (0.04)	0.02 (0.04)
Time in care	Days in care from initial system entry to exit or end of study	674.74 (284.35)	661.81 (297.40)
Child age	Age in years upon system entry	6.32 (4.57)	6.74 (4.79)
Child strengths and needs	Nine-item assessment of child emotional behavior, physical health, substance use, family relationships, and nonfamily social relationships (α = 0.75)	0.00 (4.17)	0.09 (4.01)
Control Variables
Abandonment	Child had been abandoned by biological parent(s) prior to system entry	13%	13%
Physical abuse	Child had been physically abused by biological parent(s) prior to system entry	9%	10%
Drug abuse	Drug abuse identified in household from which child was removed	37%	40%
Domestic violence	Domestic violence identified in household from which child was removed	11%	9%
Number of caseworkers	Number of caseworkers that served the child over duration in care	2.79 (1.38)	2.91 (1.48)
Caseworker experience	Average months of experience of each of the foster child's caseworkers	17.93 (24.26)	16.63 (16.94)
Caseworker caseload	Average number of families for whom each caseworker was responsible	22.37 (4.66)	22.14 (4.48)

Note: Full sample, N = 243; pilot-only subsample, n = 175. Numeric entries represent either percentages or mean values with standard deviations in parentheses.

Essential covariates of permanency outcomes include two measures of frontline service provision: the total number of in-agency, nontherapeutic service contacts provided to foster children and their families (primarily visitation and case management-related activities), divided by the number of days children were in care; and the total number of different community services that foster children and their families received, divided by the number of days children were in care. These measures of services per day in care represent the major services that child welfare agencies provide to foster children, their biological parents and/or primary caregivers, and their foster parents to facilitate movement towards permanency (Children’s Bureau 1997).6 It is expected that frontline caseworkers facilitate parent-child reunification by providing more intense in-agency and community-based services (Children’s Bureau 2006; Fanshel and Shinn 1978). It is unclear, however, whether other permanency outcomes are affected by increasing the intensity of these two types of services.

The variable capturing the influence of the performance-based contracting environment is pilot status, operationalized as whether the foster child was served by an agency operating under a performance-based contract (pilot agency) or by an agency operating under a fee-for-service contract (nonpilot agency). Evaluations of Title IV-E demonstration projects suggest that permanency outcomes may be affected by performance contracting and managed care mechanisms (James Bell Associates 2007), and prior analyses of these data determined that pilot children were less likely to be reunified and more likely to enter kinship care when compared to children served by nonpilot agencies (Meezan and McBeath 2008).

One dichotomous variable identifies whether the agency serving the foster child had made a specific structural change to its foster care technical and task environment in transitioning from fee-for-service to performance-based, managed care contracting: whether the child was served by a pilot agency that had trained staff to expedite permanency planning for foster youth. Welfare recipients have been found to earn more over time if they are served in welfare-to-work offices emphasizing the quick, intentional placement of clients in any job as opposed to improving clients’ employability through skill-building (Bloom, Hill, and Riccio 2003; Hill 2006). By extrapolation, these findings imply that children may be more likely to achieve permanency if they are served by agencies that train frontline caseworkers to reunify children or place them with relatives or potential adoptive families.

Client characteristics may also influence whether permanency is achieved, since the placements available to a caseworker may depend upon the extent to which a prospective placement can meet the needs of a child. Key measures related to the characteristics of each foster child include the child's age in years at the point of entry into foster care, and a measure of child strengths and needs, which draws from a multi-item assessment that was completed within 30 days of the child's entry into foster care by caseworkers using FIA-developed forms and that has been shown to have acceptable internal consistency (McBeath and Meezan 2008; Meezan and McBeath 2008). Studies have identified an inverse relationship between child age and reunification and adoption, and a positive association between child strengths and these two permanency options (Chamberlain et al. 2006; Courtney 1994).

Within a performance-based environment, frontline caseworkers may enact concrete responses to generalized performance demands by intensifying efforts in areas thought to lead to desired client outcomes. This logic suggests an examination of whether service effectiveness is influenced by changes in contractual and foster care technical and task environments. Interaction terms that operationalize this line of inquiry include pilot status × nontherapeutic services per day, pilot status × community services per day, staff training × nontherapeutic services per day, and staff training × community services per day. Four other interaction terms are included to examine whether more difficult-to-place children are less likely to achieve certain permanency outcomes in settings where workers are pressured to expedite children's removal from foster care (Smith and Donovan 2003): pilot status × child age; pilot status × child strengths; staff training × child age; and staff training × child strengths.

Specific control variables are included in the data analysis either because they were identified in other studies as being associated with specific permanency outcomes or they were found in preliminary bivariate analyses to be associated with the achievement of some permanency outcomes.7 Dichotomous variables pertaining to whether the child had been abandoned or physically abused by his or her biological parents reflect studies suggesting that children who experience different types of maltreatment may experience dissimilar permanency outcomes (Barth, Courtney, and Berry 1994; Brooks, James, and Barth 2002). Other dichotomous variables capture aspects of the biological household that can potentially influence permanency outcomes, including the presence of drug abuse and domestic violence in the home from which the child was removed. The inclusion of these measures reflects research identifying the co-occurrence of substance abuse, material hardship, and household violence among child welfare populations (Kohl et al. 2005; Smith and Marsh 2002). Finally, because information on caseworkers was gathered quarterly, it is possible to identify the influence of changes in the caseworkers on the outcomes of interest. Caseworker-specific control variables include the number of caseworkers that served the foster child over his or her stay in care, these workers’ average number of months of experience within the agency, and their average caseload. Large worker turnover and the presence of workers with little experience and high caseloads may hinder the achievement of permanency (Smith and Donovan 2003).

RESULTS

Descriptive Results and Pilot/Nonpilot Comparisons

As also seen in table 2, in the full sample of foster children (N = 243), 80 (33%) were reunified with their biological parents, 36 (15%) were placed with relatives, 51 (21%) had biological parents whose parental rights had been terminated but had not yet been placed in an adoptive home, and 57 (24%) either had their adoption finalized or had been placed in an adoptive home and were awaiting finalization; only 19 of the children (8%) had not reached a permanent placement by the end of the study period. As reported by Meezan and McBeath (2008), significantly fewer pilot children than nonpilot children were reunified, but significantly more pilot children than nonpilot children were placed with kin and adopted; there were no statistically significant differences in the proportions of pilot and nonpilot children who experienced TPR or who did not experience any formal disposition by the end of the study.

Roughly half of all study children (n = 121) were served by an agency that had developed trainings to assist frontline staff in expediting permanent placements. While all pilot agencies were required to participate in trainings convened by FIA administrators that reviewed the goals, timeframes, and contractual requirements of the pilot initiative, three pilot agencies implemented specific training activities to assist staff in moving clients into permanent placements quickly and identifying relatives who might be willing to care for a foster child. A supervisor at one of these agencies noted, “The deadlines—the purpose of the pilot is to do faster, better business—so you have to do more assessment, more visiting. You basically need to know what the kid's about.”

In terms of service provision, the full study sample was provided 0.18 nontherapeutic services per day and 0.02 community services per day. While in care, those served by nonpilot agencies received more nontherapeutic service contacts per day than those served by pilot agencies (0.23 versus 0.16, respectively; χ² = 19.88; df = 1; p = 0.000) but no more community services per day. Although 71 children (41%) in the study sample remained in care through the entire 930 days of data collection, children in the study spent an average of 675 days in care. The average standardized score for the caseworker-completed assessment of child strengths and needs was zero, suggesting that the study children demonstrated no more strengths than needs in terms of their social, emotional, and physical development at the time they entered foster care. There were no differences across pilot or nonpilot agencies in regard to duration in care or child strengths and needs. Whereas study children were, on average, 6 years old when they entered foster care, children served by pilot agencies were, on average, older than children from nonpilot agencies (6.74 versus 5.22 years, respectively; F = 5.39; df = (1, 242); p = 0.02).

Differences in Essential Covariates by Type of Final Disposition

As seen in table 3, children who experienced the various final dispositions differed significantly along essential dimensions. As previously reported by Meezan and McBeath (2008), proportionally more pilot children were placed with relatives or were adopted, whereas a smaller proportion of pilot children experienced TPR or were reunified (χ² = 13.62; df = 1; p = 0.009). Although 72% of the children in the study were served by pilot agencies, more than four-fifths of the children who were in kinship care or adoptive homes at the end of the study (89% and 83%, respectively) were placed from pilot agencies, whereas fewer than two-thirds of those who were reunified with their biological parents or had their parental rights terminated (61% and 67%, respectively) were served by them.

Table 3

Differences by Permanency Outcome for Essential Covariates: Full Sample

	None (n = 19)	Reunification (n = 80)	Kinship Care (n = 36)	TPR (n = 51)	Adoption (n = 57)	χ2 or F
Pilot status	68%	61%	89%	67%	83%	13.62**
Nontherapeutic services	0.16 (0.10)	0.25 (0.14)	0.15 (0.12)	0.16 (0.07)	0.11 (0.09)	16.45***
Community services	0.02 (0.01)	0.03 (0.06)	0.01 (0.01)	0.01 (0.01)	0.02 (0.03)	4.26**
Time in care	851.42 (209.46)	496.23 (279.73)	711.86 (291.33)	872.04 (149.21)	666.42 (243.85)	21.43***
Child age	9.01 (5.17)	6.59 (4.25)	9.90 (4.76)	4.70 (3.20)	4.23 (3.91)	14.48***
Child strengths and needs	−1.76 (4.48)	0.92 (4.48)	−1.03 (5.31)	−0.29 (3.44)	0.20 (2.90)	2.49*

Note:N = 243. Except where marked, entries represent mean values with standard deviations in parentheses.

*p < .05, **p < .01, ***p < 0.001.

Analyses of the data pertaining to the full sample reveal that service intensity was considerably greater for reunified children than children who experienced other outcomes. Reunified children received the most in-agency, nontherapeutic services per day (M = 0.25, 7.5/month), whereas adopted children received the fewest such services (M = 0.11, 3.3/month) (F = 16.45; df = (4, 238); p = 0.000). Similarly, reunified children received the most community services per day (M = 0.03, 0.9/month), whereas children who experienced TPR and kinship care received the fewest such services (M = 0.01, 0.3/month) (F = 4.26; df = (4, 237); p = 0.002).

In terms of characteristics, children in the full sample who experienced TPR spent, on average, more days in care (M = 872) than children who experienced any other final disposition; in contrast, children who were reunified had the shortest average time in care (M = 496) (F = 21.43; df = (4, 238); p = 0.000). Children whose final disposition was placement with kin tended to be older than other children in the study, particularly when compared to adopted children (F = 14.48; df = (4, 237); p = 0.000). Finally, children who were reunified or adopted by the close of the study exhibited more strengths than needs, whereas the opposite was true for children experiencing other dispositions (F = 2.49; df = (4, 230); p = 0.04).

These results largely mirror those from the pilot-specific subsample (table 4). Thus, it appears that reunified youth received more services, spent fewer days in care, and were assessed as having more strengths when compared to those with other placement outcomes.

Table 4

Differences by Permanency Outcome for Essential Covariates: Pilot-Only Subsample

	None (n = 13)	Reunification (n = 49)	Kinship Care (n = 32)	TPR (n = 34)	Adoption (n = 47)	χ2 or F
Trained staff	54%	67%	78%	59%	77%	5.63
Nontherapeutic services	0.16 (0.09)	0.23 (0.15)	0.14 (0.12)	0.14 (0.06)	0.10 (0.09)	8.64**
Community services	0.02 (0.01)	0.04 (0.07)	0.01 (0.01)	0.01 (0.01)	0.02 (0.03)	4.38*
Time in care	815.15 (247.58)	417.71 (274.38)	719.53 (291.67)	894.03 (90.94)	666.57 (253.82)	21.48**
Child age	9.62 (5.09)	7.56 (4.32)	9.90 (4.99)	4.89 (3.40)	4.27 (4.07)	11.76**
Child strengths and needs	−1.94 (4.00)	0.53 (4.80)	−0.31 (4.67)	0.17 (3.43)	0.42 (2.66)	1.13

Note:N = 175. Except where marked, entries represent mean values with standard deviations in parentheses.

*p < .01, **p < 0.001.

Service Provision and Permanency Outcomes.

Results from multinomial logistic regression analyses examining the relationship between service provision and permanency outcomes using the full sample are presented in table 5.8 In both this and the following table, which reports results from similar analyses for only the pilot-specific subsample, important results are described using predicted marginal probabilities. These estimate the odds of achieving a specific permanent placement associated with a one-category change in a categorical covariate or a one standard deviation change in a continuous covariate, holding constant the influence of other variables at their mean values (Long and Freese 2006).

Table 5

Cross-Institutional Comparisons of Service Intensity and Permanency Outcomes

	None	Kinship Care	TPR	Adoption
Pilot status	0.04** (0.04)	2.44 (7.89)	0.06 (0.12)	0.40 (0.87)
Nontherapeutic services	9.28 × 10−8** (4.38 × 10−7)	1.89 × 10−5*** (5.22 × 10−5)	2.46 × 10−7*** (1.06 × 10−6)	2.60 × 10−9* (2.11 × 10−8)
Community services	7.03 × 10−37 (4.90 × 10−35)	5.70 × 10−37 (4.36 × 10−35)	3.76 × 10−19** (5.58 × 10−18)	3.82 × 10−8 (1.04 × 10−6)
Pilot status × nontherapeutic services	3.05 × 104* (1.59 × 105)	145.27 (538.81)	7.04 × 103 (4.39 × 104)	4.06 × 102 (3.97 × 103)
Pilot status × community services	1.93 × 1024 (1.36 × 1026)	4.32 × 1012 (3.44 × 1014)	13.88 (4.23 × 102)	3.67 × 102 (1.03 × 104)
Child age	1.10 (0.14)	1.21 (0.18)	0.84 (0.11)	0.93 (0.10)
Child strengths and needs	0.82 (0.11)	0.68** (0.09)	0.74* (0.09)	0.84*** (0.04)
Pilot status × child age	1.01 (0.15)	0.93 (0.15)	1.08 (0.15)	0.92 (0.11)
Pilot status × child strengths and needs	1.13 (0.16)	1.42* (0.20)	1.27† (0.16)	1.16† (0.10)

Note: Results are multinomial logistic regression coefficients reported as relative risk ratios, with robust standard errors (clustered by organization) in parentheses. Parent-child reunification is the base outcome category. N = 232 and (−2lnL) = −250.58. Variables in the model omitted from the table due to space considerations include child abandonment and physical abuse, drug abuse and domestic violence in the household, the number of caseworkers serving the child, and caseworkers’ average work experience and caseload size.

†p < 0.10, *p < 0.05, **p < 0.01, ***p < 0.001.

As seen in table 5, when compared to the odds of reunification, the provision of nontherapeutic services is associated with a decrease in the odds for not achieving permanency and for achieving kinship care, TPR, and adoption. Thus, service provision of this type is related to an increase in achieving reunification in relation to other permanency options, controlling for other included variables. A one standard deviation increase in nontherapeutic service provision is associated with a 6% decrease in the odds of not achieving permanency, a 44% increase in the odds of reunification, and decreases of 1%, 13%, and 25% in the odds of achieving a kinship placement, TPR, and adoption, respectively. Furthermore, a one standard deviation increase in community service provision is associated with a 9% decrease in TPR when compared to the odds of reunification. The relationships between the service provision variables and permanency outcomes are generally unresponsive to differences in the contracting environment with one exception: for children served by pilot agencies, a one standard deviation increase in nontherapeutic services is associated with a 6% increase in the odds of not achieving permanency.

The other variable that is consistently associated with multiple permanency outcomes is child strengths and needs, which is negatively associated with the odds of being in a kinship placement, TPR, and adoption, when compared to the odds of achieving reunification. A one standard deviation increase in the overall rating of child strengths is associated with a 24% increase in the odds of reunification and decreases of 9%, 13%, and 1% in the odds of kinship placement, TPR, and adoption, respectively. This relationship appears to be moderated by pilot status, however. Among children served by pilot agencies, being assessed as having more strengths than needs is associated with greater odds of achieving a kin placement, TPR, and adoption, when compared to the odds of reunification. Specifically, among pilot children, a one standard deviation increase in the overall rating of child strengths is associated with a 17% decrease in the odds of reunification and increases of 7%, 8%, and 1% in the odds of kinship placement, TPR, and adoption, respectively.

Finally, there appears to be little direct effect of the pilot financial environment on permanency outcomes. Pilot children are 24% less likely to not achieve permanency by the end of the study, but there is no significant relationship between pilot status and the odds of reaching any other permanency outcome, controlling for the influence of other variables.

Table 6 presents results from analyses using only the subsample of children served by the pilot agencies. These subsample analyses allow one to examine the relationship between service provision and permanency outcomes within the specific performance-based contracting environment. Examination of the variable pertaining to whether children were served by pilot agencies that had trained staff to expedite permanency planning, as well as interaction terms associated with this variable, allow for testing of whether interorganizational differences exist in the effectiveness of in-agency and community-based service provision.

Table 6

Service Intensity and Permanency Outcomes within the Pilot Environment

	None	Kinship Care	TPR	Adoption
Trained staff	0.16 (0.34)	1.15 (0.89)	0.78 (2.22)	5.97* (4.75)
Nontherapeutic services	3.32 × 10−7*** (1.27 × 10−6)	2.99 (7.37)	5.26 × 10−5 (3.48 × 10−4)	9.33 (36.15)
Community services	1.40 × 10−3 (3.00 × 10−2)	1.20 × 10−101* (1.31 × 10−99)	7.33 × 10−20 (2.06 × 10−18)	4.02 × 10−79*** (1.13 × 10−77)
Trained staff × nontherapeutic services	3.28 × 103 (1.79 × 104)	6.74 × 10−7** (3.03 × 10−6)	6.46 × 10−2 (6.81 × 10−1)	2.15 × 10−13*** (1.02 × 10−12)
Trained staff × community services	1.81 × 103 (4.31 × 104)	7.86 × 1082† (8.40 × 1084)	1.39 × 107 (6.39 × 108)	4.27 × 1075*** (1.04 × 1077)
Child age	1.30** (0.11)	1.05 (0.22)	0.95 (0.08)	0.75*** (0.05)
Child strengths and needs	1.15 (0.14)	0.83** (0.05)	1.04 (0.07)	0.92 (0.06)
Trained staff × child age	0.82*** (0.04)	1.05 (0.23)	0.89 (0.08)	1.10 (0.16)
Trained staff × child strengths and needs	0.76* (0.10)	1.17† (0.10)	0.87† (0.06)	1.00 (0.10)

Note: Results are multinomial logistic regression coefficients reported as relative risk ratios, with robust standard errors (clustered by organization) in parentheses. Parent-child reunification is the base outcome category. N = 166 and (−2lnL) = −162.70. Variables in the model omitted from the table due to space considerations include child abandonment and physical abuse, drug abuse and domestic violence in the household, the number of caseworkers serving the child ,and caseworkers’ average work experience and caseload size.

†p < 0.10, *p < 0.05, **p < 0.01, ***p < 0.001.

These results suggest that the direct influence of staff training on permanency outcomes is limited: although children in these agencies are 26% more likely to be adopted as compared with their odds of reunification, no other direct relationship exists between staff training and other permanency outcomes controlling for the influence of other variables.

For pilot children, the direct relationships between nontherapeutic service provision and different permanency outcomes are largely insignificant. Among pilot children, a one standard deviation increase in this type of service provision is associated with less than a 1% decrease in the odds of not achieving permanency, as compared with the odds of reunification. For children served by pilot agencies that had trained staff to expedite permanency planning, however, a one standard deviation increase in nontherapeutic service activity is associated with decreases of 6% and 43% in the odds of being placed with kin and adoption, respectively, and an increase of 33% in the odds of reunification.

The relationship between community service provision and permanency outcomes also contains direct and indirect pathways. Among all pilot children, a one standard deviation increase in community service provision is related to decreases of 69% and 27% in the odds of kin placement and adoption, respectively, and an increase of 77% in the odds of reunification. Yet for children served by pilot agencies emphasizing speed in permanent placement efforts, a one standard deviation increase in community service provision is associated with increases of 51% and 45% in the odds of kinship placement and adoption, respectively, and a decrease of 65% in the odds of reunification.

The relationships between child age, child strengths and needs, and permanency outcomes are also influenced by staff training. For children served by agencies that had trained staff to expedite permanency planning, a one standard deviation increase in child age is associated with a less than 1% decrease in the odds of not achieving any permanent placement. And for children in these agencies, a one standard deviation increase in the overall rating of child strengths versus needs is related to a less than 1% decrease in the odds of not achieving any permanent placement and of adoption, and a 9% increase in the odds of being placed with kin.

DISCUSSION

In response to public demands for improved performance, human service bureaucracies have presumed that performance-based models will lead to improved program outcomes. Some public agencies have used performance-based contracts under the assumption that nonprofit agencies will respond to embedded financial incentives and penalties by selecting appropriate services and delivering them effectively. Performance contracting may therefore have cascading effects: changes in the contractual relationship between public fiduciaries and private agencies may lead providers to alter frontline service production, which may change client outcomes. Human service production in a performance environment involves governance in motion.

This article identifies some service-related mechanisms associated with the client outcomes commonly desired by child welfare performance initiatives. It finds that more service-related resources were expended to reunify children with biological parents than to achieve other outcomes. It took more organizational effort, as measured in services per day, to reunify children with biological parents than to place children with adoptive families or relatives, or to continue to serve them in foster homes. Reunification is generally considered the optimal outcome for most foster children and their parents and is usually the first option caseworkers consider in permanency planning. But the intensity of effort needed to achieve parent-child reunification may be daunting for caseworkers in performance-based environments, since workers who move clients towards this outcome appear to commit themselves to providing services well in excess of what is required for other placement types.

We also identify a structural mechanism through which the performance initiative was associated with these performance outcomes. Some pilot agencies trained their staff to locate potential permanency options soon after a child's entry into foster care and, where possible, place children in kinship homes in order to attain the first pilot milestone. In these but not other agencies, the provision of in-person, nontherapeutic services was related to increased parent-child reunification and decreased kinship placement and adoption, and the provision of community services was related to increased kinship placement and adoption and reduced reunification. Although all pilot agencies were under the same financial pressure to place foster children quickly into approved placements, agencies emphasizing this permanency search and placement function may have structured frontline activities so as to moderate financial risk within the pilot payment framework: caseworkers may have developed aggressive parent-child visitation schedules in order to expedite reunification planning, and where parents did not respond quickly, caseworkers may have decreased in-agency service activity and increased referrals for community-based supports to facilitate the placement of these children with relatives or prospective adoptive parents. We view these estimates and their interpretation cautiously, since the study design and statistical power available prevented more rigorous control of potential selection mechanisms that allowed only some agencies to make changes in the financially risky contracting environment. Additionally, the manner in which the data concerning agencies’ technical and task environments were gathered prevented more careful assessment of the depth of performance-related change enacted by each agency.

Evidence from the welfare-to-work sector suggests that changing workers’ focus from traditional casework to emphasize placement speed can influence client outcomes (Bloom, Hill, and Riccio 2003; Dyke et al. 2006; Hill 2006). Our findings complement these studies by incorporating how human service work may shift in response to new performance mandates. Specifically, we provide evidence that the influence of performance-influenced changes in organizational structure on client outcomes may operate through frontline service activities. This suggests that client outcomes respond to structural efforts to expedite clients’ movement from program enrollment to system exit.

From a theoretical perspective, the influence of organizational structure on service effectiveness is by no means obvious and requires more sustained examination. As embodied by welfare-to-work implementation studies, structuration theory suggests that human service production may be shaped by institutional, organizational, professional, and personal beliefs, resources, and structures (Cooney 2007; Sandfort 2000, 2003). This article provides a test of the linkages between specific organizational contexts, frontline service production, and client outcome, and reinforces the conclusion of these other studies in that the process of structuring service delivery may have real effects on client outcomes. Although it is important to emphasize that the choice, early use and adaptation, and legitimation of a specific service technology may not lead to effective or efficient outcomes, we would argue that as service technologies are believed to lead to favored program outcomes, staff activities and organizational resources may be diverted from the pursuit of other client outcomes, potentially reducing the service array and placement options available for clients.

This line of reasoning informs results from our earlier studies of this performance initiative, which determined that children served by pilot agencies were less likely to be reunified and more likely to enter kinship foster homes than nonpilot children, and which suggested that pilot children and families received fewer services than nonpilot children and families (McBeath and Meezan 2008; Meezan and McBeath 2008). These results accord with prior research on the effects of performance-based models in child welfare, although the precise means by which children receive fewer services and experience different outcomes in performance environments has been less clear until now. The service and structural mechanisms identified here may account for some of these performance-based disparities, since the services that were reduced for pilot children and families were associated with increased odds of reunification.

Although generalizability is naturally limited, these results imply that the incentive to reduce service costs found in many performance initiatives may lead to reductions in important services to clients and, by implication, decreased performance on certain client outcomes. Dias and Maynard-Moody (2007) characterize this situation as a performance paradox, in which agencies in performance environments pressure caseworkers to serve clients in ways that negatively affect client outcomes, thereby damaging program performance. How managers justify their choice of specific service technologies and how caseworkers understand, use, and even discard the tools at their disposal (Weick 1993) are crucial areas for research on performance paradoxes. What logics do agencies employ to alter their technical and task environments in performance-based and nonperformance-based environments? How do frontline caseworkers serve clients when faced with potentially competing professional, clinical, and supervisory expectations when agencies “look at (clients) as dollars,” in the words of one pilot administrator?

From a public management perspective, this line of inquiry directs attention to how human service system architects craft performance systems that help service providers respond to technical uncertainty in sensible and ethical ways. This street-level approach to performance management broadens the conversation around governance beyond a discussion of the contractual and network structures through which principals may control agents in situations of adverse selection due to information asymmetries and moral hazard related to unknown service production and monitoring costs. Imagining performance contracting as a vehicle for reducing private sector shirking through risk sharing shifts attention away from critical questions: How do we move clients towards positive outcomes? Which outcomes should be preferred over others? How do we train staff to make these decisions? These basic questions point to the technical and value-based difficulties of establishing the conditions for effective human service provision, and take us away from efficiency-based theories of organizational change.

This is not to suggest that public managers should ignore how responsive human service production can be to differences in client characteristics (Smith and Donovan 2003). We find that foster children with more strengths than needs are more likely to be reunified and less likely to experience kinship care. This relationship is reversed, however, for youth served by pilot agencies and for pilot youth served by agencies that had trained staff to expedite permanency. We interpret this finding as implying that easier-to-serve children, who might ordinarily be targets for reunification planning, are pushed towards kinship care in environments where caseworkers search for and move children quickly towards any permanency option that meets performance milestones. Given financially equivalent outcomes, agencies in performance environments may choose the path of least resistance (e.g., kinship placements over reunification and adoption); outcomes that require heavy service investments and involve significant risk of placement impermanence may be less likely to be planned.

We conclude with a normative note concerning the importance of developing human service contracts that respect the preferences and needs of clients. The New Public Management movement has been interpreted as requiring public managers to improve customer service and program outcomes (Paarlberg 2007). In performance-based human service environments, improved program outcomes may be only lightly correlated with client preferences regarding the services received (or needed) and the outcomes experienced. In the current study, over a third of pilot children experienced either kinship placement or had their parental rights terminated without being adopted. Although relative placement and TPR were valid dispositions under the pilot reimbursement structure, the service-related efforts made on behalf of children in these placements were less than those for reunified youth, and these children remained in care much longer than those who had been reunified or adopted. The pilot contracting initiative thus appears to have influenced some agencies to shift the practice of permanency planning from a reunification-first approach to the use of alternative placements, thereby allowing agencies to avoid the potentially time-consuming and expensive work of responding to the needs and problems that led families into the child welfare system. Although the performance contracting environment may have legitimized the creation of separate and economically equivalent methods for achieving permanency, we would argue that the experiences of reunified and nonreunified youth and their families were far from equitable.

Provided that these findings can be replicated in different performance-based, organizational, and client settings, we suggest that public managers interested in service equity (Frederickson 1996) and service effectiveness focus upon two critical responsibilities. The first is embedding client preferences into principals’ and agents’ utility calculations. A human service contract may be structured very successfully from a manager's perspective but very poorly from the perspective of clients (Hasenfeld 2000; Tilbury 2004). Including considerations of client equity and well-being alongside other standard performance metrics may guard against a situation in which program performance improves while service disparities increase and client well-being decreases. The second is ensuring that agencies in performance contracting environments provide services in response to clients’ needs related to system entry as well as services that are strongly associated with developmentally appropriate outcomes. These responsibilities entail that public managers have sufficient field-specific knowledge to represent client preferences reliably, recognize the likely provider- and client-level consequences of different service production choices, and assist providers in identifying and implementing appropriate service technologies. This suggests a public management that is both provincial and meaningful for the providers, frontline workers, and clients within human service systems.

Support from the Aspen Institute Nonprofit Sector Research Fund, the Michigan Department of Human Services, the Nonprofit Academic Centers Council, the Global Program for Youth of the W.K. Kellogg Foundation at the University of Michigan School of Social Work, and the Horace H. Rackham Graduate School at the University of Michigan is gratefully acknowledged. We would like to thank Anthony Bertelli, Yeheskel Hasenfeld, Carolyn Hill, Daniel Mazmanian, Matthew Potoski, Shui Yan Tang, and Carolyn Heinrich and Jodi Sandfort in particular, for their valuable contributions to this manuscript.

1

Visitation facilitates the reestablishment of parent-child attachment in the aftermath of child maltreatment and is a legal precondition to reunification. Community services may be accessed in response to biological parents’ material, substance abuse, health, and mental health needs related to the episode of child abuse or neglect, and are often required by court officials to be included in reunification planning.

2

This third implication draws upon what DeSanctis and Poole (1994) term the “spirit” or identity of a technology in noting, “The spirit of a technology provides what Giddens calls “legitimation” to the technology by supplying a normative frame with regard to behaviors that are appropriate in the context of the technology. It also can function as a means of signification, because it helps users understand and interpret the meaning of the technology” (126).

3

Much of the material in this section is taken from other publications by the authors (McBeath and Meezan 2008; Meezan and McBeath 2008).

4

Although the pilot initiative did not contain some common elements of managed care models, it was considered by FIA and the nonprofit agencies to be a performance-based, managed care initiative. Official documents described it as a performance-based, managed care system (Family Independence Agency 2000), and administrators from FIA and pilot agencies referred to it as such when they made public presentations.

5

In cases where two or more children from a sibling group were eligible for the study, only two siblings were included in the sample; where the sibling group consisted of more than two children, two children were randomly selected for study.

6

The measures of service provision are thus counts of either the number of contacts or services provided. Although studies have used count variables to capture the amount and intensity of child welfare service provision (James et al. 2004; Unrau and Wells 2005), count-based measures are not able to identify service quality. Additionally, because information on the nature of the activities that took place within each service or service contact was unavailable, it is not possible to identify whether similar services were provided in different ways or by different people. Furthermore, because these service measures were additive, it was assumed that individual service contacts were equivalent, although some service contacts might have been more critical than others from the perspective of caseworkers and clients.

7

Other variables present in the dataset were initially considered for inclusion in the study. However, they were found in bivariate analyses to be unrelated at the p < 0.05 level to all or most of the final dispositions under examination. The need to control the number of independent variables relative to sample size led to the exclusion of these variables: child race and gender; primary caregiver age, family structure, prior involvement with CPS, and needs and strengths; all formal allegations of child maltreatment except for abandonment and physical abuse; conditions in the biological household pertaining to alcohol abuse, inadequate housing, and failure to protect children from maltreatment; and caseworker race and educational level.

8

Multinomial logistic regression was used instead of ordinal logistic regression because it could not be safely assumed that child welfare workers held the same beliefs concerning the permanency planning hierarchy and, in particular, the status of kinship care in relation to reunification and adoption.