Human resource information systems in health care: a systematic evidence review

Objective: This systematic review aimed to: (1) determine the prevalence and scope of existing research on human resource information systems (HRIS) in health organizations; (2) analyze, classify, and synthesize evidence on the processes and impacts of HRIS development, implementation, and adoption; and (3) generate recommendations for HRIS research, practice, and policy, with reference to the needs of different stakeholders. Methods: A structured search strategy was used to interrogate 10 electronic databases indexing research from the health, social, management, technology, and interdisciplinary sciences, alongside gray literature sources and reference lists of qualifying studies. There were no restrictions on language or publication year. Two reviewers screened publications, extracted data, and coded findings according to the innovation stages covered in the studies. The Critical Appraisal Skills Program checklist was adopted to assess study quality. The process of study selection was charted using a Preferred Items for Systematic Reviews and Meta-Analysis (PRISMA) diagram. Results: Of the 6824 publications identified by the search strategy, 68, covering 42 studies, were included for final analysis. Research on HRIS in health was interdisciplinary, often atheoretical, conducted primarily in the hospital sector of high-income economies, and largely focused uncritically on use and realized benefits. Discussion and Conclusions: While studies of HRIS in health exist, the overall lack of evaluative research raises unanswered questions about their capacity to improve quality and efficiency and enable learning health systems, as well as how sociotechnical complexity influences implementation and effectiveness. We offer this analysis to decision makers and managers considering or currently implementing an HRIS, and make recommendations for further research. Trial Registration: International Prospective Register of Systematic Reviews (PROSPERO): CRD42015023581. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015023581#.VYu1BPlVjDU.


Administrative information systems as a topic of research in health
Administrative information systems (IS) in health organizations deal with such processes as records management, billing and finance, and aspects of human resource management (HRM), which can also help to support care delivery, quality improvement, and research. Despite their role as enablers of efficient, effective, and, potentially, "learning" health organizations, 1 administrative systems have been somewhat neglected as a topic of research in health informatics. 2 This systematic review focuses on a key subcategory of administrative systems, human resource information systems (HRIS).
What HRIS are and why they are so important Staff costs account for 65-80% of health organizations' total operating budgets. 3 Therefore, effective management of human resources (HR) is essential, from both a clinical and financial perspective. HRIS support a variety of HRM practices, including recruitment and performance management, and provide health leaders with crucial information guiding effective capacity planning and resource allocation. HRIS can take various forms, ranging from dedicated stand-alone packages (eg, payroll) to components of integrated enterprise resource planning (ERP) or hospital information systems (HISs). Not perceived as life-critical, HRIS have received very little attention in the health informatics literature, and their development, implementation, use, and impacts in health organizations are poorly understood compared with clinical systems (eg, electronic health records). HRIS research also tends to be distributed across the social (encompassing business and management), information and communications technology (ICT), and health sciences literature.

Why a systematic evidence review of HRIS in health care is needed
Although forms of HRIS have been used in the health sector for almost half a century, 4 this is still an evolving area. Increasingly sophisticated modular HRIS are being procured and implemented in health organizations worldwide, 5 often at high expense in terms of technology, support, and change management. While the benefits of these systems have been much vaunted by HRIS vendors 6 and policy makers, 7 there have also been spectacular failures, where large-scale implementations have encountered huge overspends, weak organizational buy-in, or poor interoperability with existing systems. 8 Given the opportunity costs of getting these projects wrong, developers, procurers, and managers require more guidance on the usefulness, effectiveness, and implementation barriers associated with HRIS, as well as how to evaluate them. Thus this systematic review is very timely.

What is new about this review
Our scoping study identified only 2 previous literature reviews specifically examining HRIS in health, both of which were limited in scope. 9 We therefore conducted an interdisciplinary systematic review utilizing sources of evidence from the ICT, social science, and health research literature, encompassing any ICT used for HR administration, management, and development practices in health organizations. The specific objectives were to: (1) determine the prevalence and scope of existing research and evaluation pertaining to HRIS in health organizations; (2) analyze, classify, and synthesize existing evidence on the processes and impacts of HRIS development, implementation, and use; and (3) generate recommendations for HRIS research, practice, and policy, with reference to the needs of different stakeholders and communities of practice.

Search strategy
A comprehensive search strategy was developed and tested iteratively during a scoping phase (see Supplementary Appendix 1). This was used to interrogate 10 international online databases indexing medical/health (Cochrane Library, MEDLINE, EMBASE); social science (ABI/INFORM, ASSIA, Sociological Abstracts), ICT (IEEE Xplore); and multidisciplinary research (Scopus, Web of Science Core Collection, ScienceDirect). Gray literature sources were also examined, including reports from the World Health Organization (WHO), relevant professional organizations (eg, Chartered Institute of Personnel and Development, Society for Human Resource Management, Healthcare Information and Management Systems Society), and consulting firms (eg, Deloitte, Ernst & Young, PricewaterhouseCoopers, KPMG). Academic dissertations were searched via Google, and the reference lists of qualifying articles were searched by hand to identify additional relevant studies. No restrictions were applied to publication year or language.

Article screening and selection
Procedure Outputs were stored in EPPI-Reviewer 4 software. After initial screening of titles and abstracts, the full text of potentially relevant articles was examined by 2 reviewers (AT, RB) to assess their fit with the inclusion criteria. Disagreements were resolved through consensus or arbitration by a third reviewer (CP).

Inclusion criteria
There were 2 inclusion criteria: (1) studies involving a formal or semiformal approach to the investigation or evaluation of HRIS, whether led by academia or industry (eg, consulting sector), or from within the health sector; and (2) studies of broader business/administrative/ERP/HIS systems that explicitly examine their application to HR practices.

Exclusion criteria
We excluded descriptive reports, pure market research, articles focused on software design issues, studies that were not primarily focused on HRIS or that mentioned HRIS without specifying the health sector, and articles examining generic ERP/HIS without referring to HR functionalities. Details of the filters applied at each screening stage are included in the PRISMA flow diagram.

Data extraction and analysis
One author (AT) extracted information from all eligible studies using a structured form containing the following fields: authors, publication year, setting (type of organization, country/region in which the study was conducted), innovation stage, journal discipline, HRIS functionality, research purpose/questions, theoretical basis, HRIS users, study design, and main findings. Extracted information was then verified by all team members (CP, RB, and MF).
To differentiate among HRIS project stages, we borrowed from existing innovation models (eg 10,11 ) and coded the results according to 3 main innovation stages: (1) development (eg, needs assessment, procurement initiation, prototyping, and user acceptance testing), (2) implementation (eg, purchasing, systems integration, organizational change management, and training), and (3) use (including adaptation of organizational procedures to accommodate routinization of the innovation as part of day-to-day working practices).
We also coded studies using Parry and Tyson's 12 framework to compare the intended and actual benefits of HRIS adoption. This includes 6 types of goals relating to operational efficiency, service delivery, strategic orientation, manager empowerment, standardization, and organizational image. Additional goals emerging from our analysis were added into separate categories.
Finally, of the various models of HRM practices described in the literature (eg 13 ), including in relation to HRIS (eg 5 ), we chose to adapt Foster's E-HRM Landscape model 14 to classify our studies (see Figure 3), as it covers the majority of the HRM practices mentioned in the reviewed articles. To the verbs describing core objectives of HRIS in the e-HRM Landscape we added "interact," taking account of HRIS modules described as self-service, HR portals, or HR Intranets. We also added several subcategories reflecting additional functions mentioned in the studies (eg, employee relations and qualifications tracking).

Critical appraisal techniques
Following recommendations for systematic reviews of qualitative research, 15,16 we adapted the qualitative Critical Appraisal Skills Programme checklist. 17 Questions concerning the appropriateness of qualitative methodology and ethical issues were eliminated, since a first reading of the material revealed that most eligible studies were qualitative and lacked ethical considerations (see Supplementary Appendix 2). In addition to the "yes" or "no" answers, we added a "not clear" option (corresponding to scores of 1.0, 0.5, and 0, respectively). One reviewer (AT) appraised all eligible studies. A second reviewer (CP) independently appraised a random 20% sample to assess interrater consistency and facilitate discussion about the process and any ambiguities. Since only a few minor discrepancies were identified, a secondary appraisal focused on studies about which the first reviewer was uncertain.

RESULTS
In all, 6824 results were generated by the search strategy and 6104 titles and abstracts remained after removing 720 duplicates. Of these, 399 qualified for full-text review, 232 due to their potential eligibility and 167 because there was insufficient information in the title or abstract to make a decision. After removing documents that did not meet the inclusion criteria, 68 publications representing 42 separate studies were included in the final analysis (see Table 1). The stages of selection are illustrated in the PRISMA diagram labeled Figure 1.

Publication characteristics
Included articles were published between 1979 and 2014. More than half entered the literature within the last decade, peaking in 2010, when 11 were published (see Figure 2).
Out of 68 publications, the vast majority (n ¼ 41) were journal articles. To test our observation that HRIS in health is a multidisciplinary topic, 9 these articles were first classified into subject areas according to the Scimago Journal ranking portal (Scimagojr) and afterward using broader discipline categories such as health, ICT, and social science. Nine articles were classified manually, as the journals were not covered by Scimagojr. 29 articles (71%) were published in a single discipline: 18 in health (44%), 9 in social science (22%), and 2 in ICT (5%). Just under a third (29%) were published in multidisciplinary journals, including 5 covering ICT and health (12%), 3 covering health and social science (7%), and 4 covering social science and ICT (10%).

Country
The majority of studies were conducted in high-income countries (see Table 1): 17 in Europe (4 each in the Netherlands and the UK, 3 in Finland, 2 in Ireland, and 1 each in Greece, Norway, Spain, and Turkey), 9 in North America (7 in the United States and 2 in Canada), and 1 in Australia (although several authors independently studied this case, it was classified as one study). Only 4 studies were conducted in Asia (2 in Pakistan and 1 each in India and Taiwan), 6 in Africa (2 in Kenya, 1 each in Malawi, Uganda, and Tanzania, and 1 covering 9 African countries). One study was conducted in South America (Brazil), and 1 in the Middle East (Saudi Arabia). Three studies either involved several countries across different regions or did not specify the countries covered.

Units of analysis
Although diverse health organizations were represented, more than half of the studies focused on hospitals in high-income countries, typically taking one hospital as their unit of analysis. Only one study focused on a primary health care organization (see Table 1). Studies in low-income countries mostly reviewed country-wide HRIS and/or systems developed, implemented, and used by government Departments of Health or professional organizations.

Research designs and study quality
Most studies (n ¼ 24) used qualitative methods. Nine employed quantitative designs, while 8 used mixed methods. One study was a systematic literature review (a second review identified by our search did not meet the inclusion criteria; it focused on ICT for enabling continuing professional development, and e-learning was out of the scope of this review 9 ).
Descriptive studies were excluded at the full-text review stage. None of the qualifying studies received a maximum score of 8 on quality assessment. Those scoring highest were quantitative studies and postgraduate research theses; those scoring lower did not adequately explain their units of analysis, research methodology, or sources of potential bias. Of the qualitative studies, very few scored higher than 6 (see Table 1 and Supplementary Appendix 2).

Theoretical frameworks
Over half of the studies (n ¼ 22) did not specify any theoretical perspective. The other 20 referred to a diversity of frameworks, most specifying only one (see Table 2).

HRIS types and their functionalities for HRM practices
Most qualifying studies (n ¼ 21) examined dedicated HRIS, comprising one or several modules for supporting particular HRM practices. Sixteen studies focused on generic integrated organizational systems, including modules dedicated to HRM practices. Five did not clarify whether the HRIS were dedicated or components of generic systems (see Table 1).
Descriptions of ICT for managing HR in health organizations lacked a common terminology (see Table 1). Organizational systems that included HRM functions were commonly described as        ERP (n ¼ 3), patient classification system (n ¼ 3), or Intranet (n ¼ 2). Dedicated systems were described as HRIS (n ¼ 7), payroll/ salary system (n ¼ 4), or electronic-HRM (n ¼ 2). HRIS (n ¼ 3) was used most frequently in studies not specifying whether the system was dedicated or generic. HRIS support various HRM practices in health organizations. However, as shown in Figure 3, most qualifying studies focus on operational HRM practices (eg, HR administration or scheduling).

HRIS users
HRIS are designed for a variety of users. The most commonly mentioned user groups were health sector leaders/decision-makers (n ¼ 6), hospital management, HR department/HR professionals, nurses, nurse managers/administrators, and employees (all with n ¼ 5). Less commonly mentioned were health organizations, government//professional authorities, line managers (all with n ¼ 3), staffing clerk/coordinator (n ¼ 2), clinicians, donor agencies, internal temporary employment agencies, rural primary care teams, and nurse educators (all with n ¼ 1). Seven studies did not specify any HRIS user categories.

Innovation stages
Innovation stage was classified based on our interpretation of a study's aims and findings rather than any authors' explicit statements, which often bore little resemblance to the stages described in the study.
Half of the studies (n ¼ 21) focused exclusively on a single innovation stage, mostly on HRIS use (n ¼ 17), with 2 studies focusing on either development or implementation. The other half encompassed several innovation stages, 9 covering development, implementation, and use, 5 development and use, 5 implementation and use, and 2 development and implementation. Table 3 indicates the innovation stages covered and shows that the studies focused mainly on (1) approaches to HRIS use, (2) factors of influence during HRIS implementation, (3) HRIS outcomes, such as realized benefits, and (4) drivers for HRIS.

Drivers and realized benefits
The majority of studies described HRIS implementation as being driven by expected benefits or goals. The most common related to strategic orientation -being able to use information about HR needs and performance for evidence-based decision-making, to inform HRM policy and planning, or as a means of migrating to a centralized, enterprise-wide HR shared services approach. This was followed by operational efficiency -reduction and control of costs, automation or augmentation of manual processes, time saving, and reduced bureaucracy. Improvements in HR service delivery were also expected, such as identifying current levels of provision, resolving issues with external service providers, and/or increasing the quality of information in HRIS. Other expectations driving implementation included standardization of systems, processes, or data; empowerment of managers and/or employees; compliance with statutory requirements for data on the health workforce; and helping to manage macro organizational changes, such as a planned hospital merger. We did not find evidence that health organizations adopted HRIS to improve their organizational image, as suggested in Parry and Tyson's framework.
The most commonly realized benefits of HRIS implementation related to strategic orientation and operational efficiency improvements, followed by empowerment of managers and employees, improvements in service delivery, standardization, and compliance with regulatory requirements. Another was improvement in patient Figure 1. PRISMA flow diagram. a Database has limitations on the number of keywords, therefore the search had to be run several times to ensure that all search query keywords were included (please see 9 ). b Book reviews, front and back covers, copyright notice, title pages, collection of conference proceedings' descriptions, tables of contents, press releases, announcements, descriptions of issues, advertisements, bulletins, questionnaires, notices of retraction, chair's messages, keynotes, plenary talks, welcome messages, news published in journals and magazines that have "news" in their title and news published by companies that do not provide any analytical or research materials, presentation description, very brief cases and analytical materials published in newspaper and magazines, company profiles, advertising/marketing articles. c Articles not related to HRIS in health organizations, research on HR practices in health organizations that do not defer to use of ICT in relation to HR activities. d Articles where no abstract was available or where title and abstract did not give sufficient detail to judge eligibility, articles on HRIS that do not specify the industry/sector in which they were implemented, articles on generic ERP/HIS that do not specify the module/functionality and/or industry/sector in which they were implemented. e Potentially relevant articles referring to HRIS in health organizations. f Articles focused on computer science models (eg, software specification) or management science models (eg, creating algorithms to enable staffing and scheduling in health organizations). g Generic analyses of principles, benefits, requirements, implementation methods of HRIS in health organizations, or pure market research.
care by facilitating minimum standards of nursing care. 43 One study reported that hospitals using HRIS had lower rates of vascular catheter urinary tract infections. 73 Generation of interest from other countries 27 and improved ICT infrastructure 18 were also reported as beneficial outcomes.
Only 5 studies reported whether projects had achieved their expected benefits, and even fewer described failure of the HRIS to influence specific goals, notably operational efficiency (n ¼ 3), strategic orientation (n ¼ 1), and service delivery (n ¼ 1) (see Table 1 for details).  S6 Game-theoretic model S6 Evaluation framework for business process projects S14 Knowledge-sharing concept S15 Evidence-based health care S23 Emancipatory principles and principles of critical social theory S38 Does not specify S2, S4, S5, S12, S13, S16, S17, S20, S24, S25, S26, S27, S28, S30, S31, S32, S34, S35, S37, S39, S40, S41 Only one study (S9) reported specific adverse effects of HRIS implementation within the organization, including negative perceptions of HR roles and increases in supervisors' workload associated with changing to new HRIS processes. More general adverse effects were mentioned in another study (S21), which described a regionwide HRIS project as a "catastrophic failure" 52 with multiple negative consequences for contractors and government, including staff strikes and the Minister of Health's resignation.

User satisfaction
Three studies reported users being satisfied with the system itself, 1 with its functions, and 4 with the information it provides, although 1 noted dissatisfaction with new HRIS procedures and forms. Two described HRIS satisfaction as being dependent upon ease of use, 2 upon types of users, and 1 each on users' familiarity with the system, time required to judge systems, whether systems reflect true workload, and time in use, satisfaction increasing with evolving user capabilities and organizational adaptation.

Factors shaping HRIS development, implementation, and use
Facilitators and barriers were reported across innovation stages (see Table 4). Success was influenced primarily by project-related factors, including governance structure, approaches to project management, and quality of execution, and by individual factors such as stakeholders' political behaviors and user involvement. Organizational factors, including organizational size, diversity, culture, degree of centralization, and availability of resources, were the most significant barriers. Some studies described technological barriers, including breadth of system functionality, degree of local configuration, and interoperability. Barriers associated with existing HR processes were also mentioned, and several studies recommended simplifying such processes prior to HRIS introduction, although none reported any evidence of this having facilitated a project's success. Macro-environmental influences, such as political reforms and inter-organizational relationships, were considered very little.

Summary
The intention of this review was to capture, synthesize, and interpret existing evidence on HRIS in health care organizations. We discovered that research in this area ranges across disciplines and varies widely in terms of its objectives, methods, theoretical orientation, quality, and language. As expected, the evidence base is sparse compared with clinical information systems research. Most studies focus, somewhat uncritically, on the use and realized benefits of HRIS in practice, rather than sociocontextual or technological factors influencing their development, implementation success, or impacts on strategic decision-making or cost-effectiveness. Most research comes from higher-income countries and examines small-scale systems in individual hospital settings. Nevertheless, several higherquality studies were found, including one national program evaluation, and we were able to adapt and apply existing theoretical frameworks to help organize and interpret the evidence, suggesting that it may be possible to build a more integrated body of research in this area.

Scope and meaning of HRIS
The plethora of terms used to describe HRIS, and variation across disciplines, suggests a lack of consensus and makes it difficult to build a coherent evidence base. This may explain why a previous systematic review on HRIS in health 64 did not identify any research prior to 2000, whereas our review, using a broader range of search terms, found 7 such studies. Therefore, we recommend that researchers go beyond obvious keywords (eg, HRIS) when undertaking background research for new projects (for list of relevant keywords, see 9 ).

Types and quality of research
Purely descriptive research was excluded at the screening phase, hence the methodological quality of the included studies was higher than in the literature as a whole.
Most included studies were published in health journals, but many in social science and ICT journals, with some crossing disciplines. Over half were qualitative, and of those reporting quantitative data, none evaluated cost-effectiveness or return on investment. Given the considerable expenditure on HRIS within the heath sector, this gap is surprising, although it reflects a broader evidence deficit in the health informatics literature. 85,86 Use of theory The use of relevant theories was an important consideration for our assessment of HRIS research. Although many studies mentioned one or more theoretical frameworks, half did not, confirming observations from a previous literature review on HRIS. 87 Most of the theoretically informed studies were published in social science journals or as academic dissertations. Of the studies mentioning a theoretical perspective, nearly all referred to different ones. As such, in line with clinical systems studies, which seldom build on prior research, 88 studies on HRIS research in health mostly represent ap-plied projects and do not advance theoretical understanding of HRIS development, implementation, or use.

International perspectives
The focus of HRIS research has varied across countries in terms of systems, contexts, and priorities. Most studies from high-income countries have focused on small-scale systems in individual hospital settings, with the key users being internal personnel and managers (clinical/nonclinical), although there are notable exceptions, such as a major program evaluation in Australia. 8 Moreover, nearly all user satisfaction studies have come from high-income countries.
Research from lower-income countries tends to concentrate on open-source HRIS to collect data at the national and regional levels, focusing on health leaders and decision-and policy-makers as the primary system users. Most studies, especially those from lowincome countries, prioritize operational aspects of HRM practices, despite WHO recommending in 2001 that effective HR departments should also undertake managerial or strategic HR activities. 89 We observed a general scarcity of HRIS research in health from East Asia and the Pacific, Eastern Europe, Central Asia, Latin America and the Caribbean, the Middle East and North Africa, South Asia, and sub-Saharan Africa. Moreover, we did not identify any study that compared HRIS projects across countries, supporting the call for more international comparisons of ICT research in health. 90

Key messages
HRIS are underrepresented in the health informatics literature, despite their potential to contribute to information-driven learning health systems and the substantial financial investments that are being made in them. Most research is based on softer forms of evidence, and there are important gaps in knowledge about the impacts and cost-effectiveness of these systems, which calls for further research. Interdisciplinarity is a positive characteristic of this literature, in view of the importance of sociotechnical factors for the success of HRIS projects, but the sheer variety of terminologies and theories represents a barrier to building the coherent evidence base needed to translate evidence into practice.
Of the many studies in our review, only 4 looked at the potential for HRIS to support wider aspects of health care and their indirect effects on patient outcomes, despite their having been characterized as "the only class of hospital IS that has a dual beneficial impact [on] patient care [and] operating costs." 76 Given the rising cost of health care and the growth in patient traffic, the future sustainability of health systems will depend on making the best use of information to optimize deployment of HR. 3 Linking the administrative data from HRIS with data on clinical processes and outcomes offers tremendous opportunities to enable real-time and predictive analytics alongside continuous monitoring and evaluation for smart, efficient, and "learning" health systems. 91

Limitations
By excluding descriptive HRIS studies, which are published mostly by HR and clinical practitioners, we may have missed applied case Table 4. Summary of influential factors mentioned in the included studies

CONCLUSIONS
This review addresses an important gap in the health informatics research literature and can serve as a helpful point of reference for managers planning or implementing HRIS, academics studying health IS, and policymakers or research sponsors considering an investment in health informatics. We also hope that scholars studying HRM practices in health organizations and HRIS in other sectors may find this a useful contribution to the field. We recommend new programs of interdisciplinary research, encompassing economic evaluations, sociotechnical analyses, studies of information flows, and systematic assessments of the impacts of better workforce information on health care efficiency, quality, safety, and patient care, as well as new exploratory research to understand the value of information for driving analytics in support of sustainable and effective health systems.