In 2013, many public health functions transferred from the National Health Service to local government in England. From 2006 NICE had produced public health guidelines based on the principles of evidence-based medicine. This study explores how the guidelines were received in the new environment in local government and related issues raised relating to the use of evidence in local authoritites.
In depth, interviews with 31 elected members and officers, including Directors of Public Health, from four very different local government organizations (‘local authorities’).
Participants reported that (i) there were tensions between evidence-based, and political decision-making; (ii) there were differences in views about what constituted ‘good’ evidence and (iii) that organizational life is an important mediator in the way evidence is used.
Democratic political decision-making does not necessarily naturally align with decision-making based on evidence from the international scientific literature, and local knowledge and local evidence are very important in the ways that public health decisions are made.
In 2013 in England, public health functions, which had been performed principally by the National Health Service (NHS) or by agents of the central UK state since 1974, were reorganized. Responsibility for the delivery and commissioning of many public health services were transferred to municipal or local government (in England called local authorities or local councils). This was the largest reorganization of public health since the creation of the NHS in 1948.1
Our aim was to understand how the way ideas derived from evidence-based medicine (EBM), which had been important in the NHS were received in the very different environment of local authorities after 2013. We initially set out to investigate how the raft of extant evidence-based public health guidelines from NICE published from 2006 and which therefore pre-dated the reforms (http://www.nice.org.uk/guidance/published?type=ph) fared as they found themselves vying for attention in local government. As we proceeded our focus broadened to consider the role of evidence and its use in local councils more generally.
NICE public health guidelines were generated by methods originally developed for constructing clinical guidelines and for conducting technology appraisals and were based on the principles of EBM.2,3 These methods evolved at NICE after 2006 to meet the needs of developing public health guidelines4–6 but the approach remained fundamentally one in which the goal was the scientific appraisal of evidence from international studies, often from the USA and almost exclusively from the peer-reviewed literature, to provide the best account of the state of the evidence in any given area. This evidence was then the basis for recommendations for practice or interventions. The idea that there is a single best way to generate and appraise evidence (a basic premise in EBM) has never taken root in local government. But evidence from engineering, planning, transport, housing, environmental health and trading standards are part of the common evidence currency within local government.7 These extant and varied bases for decision-making were already well established in the new arena of local government in which the NICE evidence-based public health guidelines had to find a place.
Local authorities and NHS organizations differ from each other in that local authorities are democratically elected political decision-making bodies whereas the erstwhile NHS public health delivery bodies (Primary Care Trusts) were not. Local authority members (but not their officers) are elected politicians who are accountable to their electoral constituents. This means that local councils are party political organizations in which political preferences and ideological partisanship are intrinsic to the way that they operate. Although the NHS is also political, as all organizations and bureaucracies are,8–10 for the most part, overt partisan party politics played very little role in the NHS. The local NHS was not accountable to the communities it served through the ballot box and was only politically accountable indirectly through the party politics of national government and scrutiny by the media.
Thirty-one public health local authority employees from four local authorities were interviewed about their experiences of implementing the evidence-based guidelines. We aimed to recruit a variety of local authorities. Potential case study sites were identified through contacts already known to research team members, a briefing article about the research project circulated in a newsletter produced by the Association of Directors of Public Health, and approaches to attendees at a NICE local government event. The four case study sites were selected to reflect a range of local authority characteristics: a unitary authority in the north of England; an outer London borough; a two-tier authority with rural areas and an inner London borough. With only four local authorities included in the research, this could not be a representative sample, but instead aimed to encompass a range of experiences, both common to all of the local authorities and unique to particular circumstances.
The first interview in each local authority was conducted with the Director of Public Health, who was asked to identify other officers and Councillors whose work related to public health. Those people in turn were asked to suggest others who would be useful to interview, to achieve a mix of interviews with people in different roles and at different levels of seniority. In each local authority, interviews were conducted with: the Director of Public Health, the Councillor holding the public health portfolio, another councillor with no specific public health remit, officers working directly on public health issues, officers based in other departments whose work intersected with public health, officers based in a district council in the two-tier authority and members of the local Clinical Commissioning Group.
Interviews were semi-structured. Two versions of the interview schedule were developed, one for officers and one for councillors. The same questions were asked in each interview, and respondents were prompted for further details and specific examples. The schedules were piloted with public health officers at a local authority not chosen as a main case study and then revised prior to the main fieldwork commencing.
Interviews were conducted face-to-face between October and December 2014 and took an average of 45 minutes to complete. The majority were conducted by C.L., with L.A. and H.G. conducting one each (see acknowledgements). Interviews were audio-recorded and transcribed.
Interviews were analysed using a theoretical model of behaviour, ‘COM-B’.11 The central tenet of COM-B is that capability, opportunity and motivation are needed for behaviour to occur. The model has been used to identify influences on behaviours such as the use of evidence-based guidelines.12 In this study, the model was used to identify the extent to which participants had the capability, opportunity and motivation to use evidence-based guidance for public health in decision- making. Topics covered in the interviews included the kinds of evidence or guidelines that were used when developing local public health policy, the perceived strengths and limitations of using local and national evidence and the observed barriers and facilitators to using NICE guidance for public health. A range of themes emerged in these data. We report here on three of the themes and have selected quotations to best illustrate them. A full description of the other themes and the methods can be found in Atkins et al.13
The data revealed (i) a tension between evidence-based and political decision-making, (ii) differences of view about what constitutes ‘good’ evidence and (iii) that organizational life is important in the way evidence is used.
Evidence-based versus political decision-making
The tensions between professional/scientific evidence-based decision-making on the one hand, and pragmatic and political decision-making on the other is illustrated by the following.
The assumption that evidence should precede decision-making, a basic premise of EBM, was not particularly prominent in local authorities.
‘[C]ouncils don't like being told what to do. They don't like dicta and whereas the NHS might be able to say through its hierarchy of command and control, NICE has said … You can't tell a local authority to do that. They say, well, thank you for your opinion. We'll weigh it up carefully and we'll do what our voters tell us. Welcome to democracy…’ (Site 3, Director of PH) [Quotes are anonymised and identifiable by site number and participant role (Councillor; Director of PH, PH team member).]
‘When you work in an NHS organisation obviously there's… You know, it's at the behest of central government and policies and stuff but once those are decided, you then get on in implementing…and there's not that much scrutiny…When you're at a county council because it's autonomous, it makes its own decisions, there's a huge amount of scrutiny. And trying to make anything happen, you can't as an officer, you can't make the decisions.’ (Site 3, PH team member 3)
‘So if our DPH puts something to our elected member, unless it contradicts their manifesto or their principles, political principles, then they are usually receptive to, okay, that seems reasonable, you're the expert, you're here to advise me, so let's look at that. So I feel like they're quite open to that kind of challenge, that kind of discussion.’ (Site 2, PH team member 6)
‘Well, the barriers are that the council… It has to fit with the council's agenda, and in terms of pure public health outcomes, that's not necessarily what their main drive is.’ (Site 3, PH team member 3)
‘Like, this is what we think we should be doing, this is what the evidence tells us, where we should be going..Rather than, I want this to happen, go and find the evidence to make it happen. It's a different way of working.’ (Site 4, PH team member 3)
‘Well, as you know, every politician works on an anecdote. We have to use evidence either to support or refute the anecdote. And sometimes you get overruled. If you manage to, you know, ensure the evidence base is followed 75–85% of the time probably in this environment, we're doing pretty well.’ (Site 4, Director of PH)
‘I think public health colleagues are used to being guided by evidence, NICE guidance, and not necessarily as comfortable as, what, as I as someone who has worked in local authority for a long time, the sort of, democratic environment where there's an elected representative who will take responsibility.’ (Site 2, PH team member 6)
But some councillors had a more measured view about this than their officers.
‘Yes, we do listen to our public health [officers]. And, 95% of the times, we'll listen to them. And take their advice on board. And then, provide the political clout, if you want, to achieve that goal.’ (Site 2, Councillor 1)
‘I'll leave [use of an evidence base], I'll leave that to our experts… And, I expect [the DPH] to know his stuff, and appoint the right people for it.’ (Site 1, Councillor 1)
What constitutes ‘good’ evidence?
A second and related issue was about what constituted good evidence. Participants ascribed different values to different kinds of evidence. Some views reflected the conventional hierarchy of evidence familiar in EBM, in which randomized control trials, large sample population data and meta-analyses, along with guidance based on such evidence, are valued above other types of evidence.
One participant, however, cautioned against over-simplification of the differences, drawing attention to the justificatory narratives used especially in the NHS.
‘To a public health person, evidence – what that means is very clear. There's a defined hierarchy of evidence. It's quite…clinically focused so it tends to be weighted towards things like randomised controlled trials and systematic reviews. If you speak to a councillor, an elected member, what they mean by evidence will be what their people tell them, what people and wards [electoral neighbourhood areas] tell them, because that's what they know to be true or what they see when they walk around their ward.’ (Site 1, PH team member 2)
There were also some participants who valued local data (epidemiological, local-level statistics, locally produced research) over national data or guidelines, reflecting a view that their local authority area is not typical and that nationally produced evidence or guidance needs to be translated or modified to be relevant to local circumstances.
‘In relation to the use of evidence, and things like that, I think, one of the key changes, was that the NHS has a very strong narrative, that it tells itself, that it's evidence-based, and scientific, and rigorous, and everything like that. And that local government decides things on whim, and political priorities. My experience of coming over, in to local government, was actually, that the truth of that is much more complex.’ (Site 1, Director of PH)
The data do not suggest that participants espoused a particular view about good evidence and dismissed the other approaches out of hand. But the analysis does suggest that there were tensions between the different views of what forms good, useful or appropriate evidence, and that guidelines from sources like NICE were only one among a plurality of evidence possibilities (see also McGill et al. 20157).
‘Well, national evidence, and NICE guidance, and stuff like that, is often more rigorous, in terms of scientifically defensible, but it's also very limited, in terms of it can be limited…in terms of its applicability.’ (Site 1, Director of PH)
‘Well, I think the dilemma is that NICE guidance is applying something, sort of, blanket…too generic…And local authorities are not generic. They are all fundamentally different organisations with their different political make-up and their political, you know, objectives and stuff like that.’ (Site 3, PH team member 3)
‘My personal view is that the experience elements are the strong elements. The evidence that I've gained from interacting with people specifically, is the strong evidence that I rely on more than the study evidence or the… I think it's very hard to argue, not hard to argue, as if you should argue it, if there is a quality study that says, this is the case, this has been done in an appropriate way and a robust way, I think you take that and you go right, that's good enough for me at the moment. Then I apply the experience, the interactions with people which I feel is good evidence, because I feel that gives me a judgement as to what will work on the ground. It actually goes in, because especially in the area that I'm used to, it's different for everyone. The reason why they're not active, there are themes, but it's actually, when you get down to it, the thing that's going to change them, it's almost like a fingerprint, it's different, there's a slightly different nuance to it. So to me, that level of evidence, if you can call it evidence is more useful than so and so did a really good study in Northumberland, because it followed x y z Guidance which makes it a good study. I would rely more heavily on the personal information.’ (Site 2, PH team member 6)
The nature of organizational life
A third dimension we draw out from the data is the idea that regardless of the provenance of the evidence and its method of collection the everyday and routine nature of organizational life was a key factor in the way evidence was used. So for example institutional inertia was seen to play a role.
‘…we've tried to prioritise but the difficulty is, if you've got a contract to provide a service you can't suddenly just step away from that, and also we've got the five mandated services which we all have to provide in some shape or form.’ (Site 3, PH team member 2)
‘Because if they're, if they've been commissioned to do a certain service, I think, it's very easy for them to just continue doing that service for as long as the contract runs, rather than, potentially, new evidence appears, you know, and we should all be doing evidence-based commissioning. But, you know, you commission something for three years and within that three years, things might change. But do the contracts change with it? Do service-level agreements change with it? Probably not.’ (Site 3, PH team member 4)
‘In my experience, in all honesty, [the council is] very bad at taking decisions and actually prioritising something over something else. What it tends to do is to follow on what was done before, or follow what has the most supporting evidence.’ (Site 2, PH team member 6)
Main finding of this study
In the transition of public health from the NHS to local government the mix of possible factors and types of evidence informing decision-making became potentially more varied and the party politics more explicit and open. However, it had never been the case that the NHS, local Primary Care Trusts or clinicians always followed precisely the principles of EBM or for that matter NICE guidelines, nor is it the case that the role of EBM in public health has been uncontested.14 All sorts of other types of knowledge and information played a part, including clinical judgement, and in the case of public health, knowledge about local communities. So the tension between evidence-based and other forms of decision-making, arguments about what constitutes ‘good’ evidence, also documented in the formation of recommendations in guidelines15 and the limitations on decision-making imposed by organizational life reported here, would have been quite familiar in the day-to-day activities in Primary Care Trusts pre 2013. The transition did though seem to make the issues more pronounced.
Before 2013, some of the outputs from the NICE public health programme had been specifically aimed at local authorities relating to transport, planning, education and social care for example. This meant that many of the issues relating to implementing guidelines in local authorities had been encountered by NICE before the transition. It is not clear whether the architects of the new arrangements imagined that the transition would give NICE guidelines greater traction than they had had hitherto or whether the issue was given any consideration. NICE were not consulted directly on the matter, nor was their previous experience used by officials as the new arrangements were drafted—in hindsight perhaps an opportunity missed. However, the role of evidence was not perhaps the most pressing issue facing those charged with implementing the reforms. As the system settles the role of the relationship between NICE, evidence, evidence-based approaches and the role of Public Health England (PHE) (itself established as part of the reforms) perhaps deserve to be revisited.
What is already known on this topic
To date, there have been few studies of the transition of public health to local authorities (but see McGill et al.;7 Phillips and Green;14 Oliver et al.16). We learned in our studies that professional local authority officers, including public health specialists traditionally owe their status and legitimacy to organizational power within the authority and not necessarily to their specialist knowledge. Their role as employees in the organization and the position they occupy in the hierarchy traditionally overrides other sources of authority. This seems to have continued after the transition with science, or medicine, professional training or NICE guidelines not primary sources of legitimate authority.
What this study adds
Our data show that the practitioners in this environment have had since 2013 to learn new ways of getting their arguments heard and acted upon, when appeals to the authority of science or NICE are not always seen as paramount.
Politicians draw heavily on highly localised knowledge7 in many ways the antithesis of universal scientific principles. Councillors assume the mantle of experts about the local communities they represent; we report that this type of expertise could trump specialist professional knowledge or national guidance. The status of councillors as elected representatives with their particular understandings of views ‘on the doorstep’ provides them with a legitimacy in decision-making which scientific knowledge does not. Politicians can lay claim with some justification to an understanding of what will be suitable for and acceptable to the local population. The fact that they are directly electorally accountable means that there is considerable pressure on them to be accurate in this regard. That said, important as politics are within local authoritites, it is not the case that all decision-making is explicitly party political or partisan. The fact that local authorities are bureaucracies is fundamental to the way that they work and the very many disciplines located in local councils, such as engineering, transport, housing, education and environmental health, all use evidence and this evidence is part of the decision-making process.7 What was not commonplace in local authorities before 2013 was the kind of methods and evidence associated with EBM.
The conventional EBM approach is that evidence is the product of scientific endeavour that is then subject to appraisal and testing on the basis of its methodological fidelity and ranked according to principles designed to reduce bias. In local authorities, the niceties of systematic review and appraisal of the scientific literature are mostly unknown and evidence derived from local knowledge is often seen as most relevant to local decision-making. In addition, the reality of political power will inevitably play a role, though not the only role in decision-making in political bodies. Elected local government councillors, especially those with special portfolio authority for a particular topic, like public health, are very influential in selecting and driving forward local priorities. Their locally and sometimes politically based priorities do not always align with the goals of the public health professionals whose aspirations and strategies derive from their professional medical and public health training and from evidence-based guidelines. Whilst NICE guidelines for example are apolitical, based on a dispassionate review of the evidence, which is appraised according to well-defined criteria and ranked on the basis of its methodological provenance designed to reduce bias, political goals, passion and bias are part and parcel of local authority life.
It is worth noting that the culture of using research evidence has taken many years to become established in healthcare, with early resistance from some clinical experts. Nowadays, evidence-based practice is the accepted clinical standard, within a framework that ‘Guidelines are guidelines not tramlines’ (Sir Michael Rawlins, first chair of NICE, personal communication) and may not always be relevant to an individual patient. In a similar way, we need to encourage a culture of routine reference to national evidence-based advice in local government, within the knowledge that local priorities and population differences may lead to a considered view that the guideline does not apply.
Limitations of this study
Data were collected in Autumn 2014, so after the initial turmoil of the 2013 reorganization was over but while the system was still settling down and the reformed arrangements were relatively new. Our data therefore reflect some of the early concerns that emerged in four local authorities. One critical issue, which will have affected the context of local authorities at the time of our data collection and subsequently, was the extent of the reductions in their budgets following the implementation of the UK Coalition Government's austerity programme. It is not clear from our data, and it probably was not clear to our participants at the time either, whether and the extent to which financial constraints were a factor in the way evidence was used in the transition period. Our sample size was small and although the local authorities from which it was drawn were varied, we cannot claim to have represented all English local authorities. We are uncertain as to whether the phenomena we report here have wider generalizability or indeed continue to be important. It also remains to be seen whether and how over time the system itself evolves to handle these kinds of issues. Our data relate to respondents’ general perceptions of differences in attitudes to evidence between the health sector and local authorities. We did not explore in detail how this played out in their day-to-day experiences. We suggest that further research exploring the inner workings of local councils, now that the new arrangements have had further time to embed, will reveal more about the dynamics of the processes involved than our methods allowed us to demonstrate.
The world of decision-making in local authorities is a far cry from the idealized models based on clinical decision-making out of which EBM developed.17 It is neither linear nor direct and there is certainly no genuflection in the direction of the traditional EBM hierarchy of evidence. There is a plurality of materials, which feed into the evidence mix which become a part of a political and a deliberative process. Local issues and perceptions of local needs and problems are very important in framing discussions and examples cherry-picked from other jurisdictions sometimes find a place too. Experience and indeed prejudice all contribute to the process. That it should be anything other than this is hardly surprising given the long and vibrant role of local democracy and local decision-making about local welfare matters in England, which predates even the Elizabethan Poor Law.18,19 EBM and NICE guidelines are very much the new kids on the block in this regard.
In local authorities, professional expertise finds itself up against democracy. The greater level of democratic scrutiny in local authorities was seen by some participants as a barrier to properly professionally informed decision-making. However, local authorities are, to a significant degree, autonomous bodies and they fiercely guard their independence. Many participants commented that local authorities do not like to be told what to do whether by locally based professionals or by central state organizations. The EBM approach can appear to be highly prescriptive and centralized. Participants acknowledged that this approach was unlikely to be warmly received in confidently independent organizations led by politicians. If, however, the role of the guidelines is framed as an important ‘starting point’ to address local problems, then in the complex political world of local authorities, the guidelines could find an important place. NICE public health guidelines do not for the most part describe simple interventions but acknowledge the complexity of the real world of public health. The optimal approach for NICE guidelines would be to help to shape the understanding of problems at local levels as a basis for decision-making, using the evidence base as the way to define the problem, rather than as a singular answer to complex problems.
Finally, it is worth noting that the relationship between evidence and action and evidence and policy is not a linear one at any level—national or local. The assumption that just because good guidelines exist based on the most careful consideration of evidence that that will lead to changes in practice or policy is naïve. As van Hulst20 showed in Dutch municipal public administration, local decision-making is a rich tapestry of sense making and interpretation as politicians and administrators work out what it is they need to deal with and how to go about dealing with it. What our respondents show us is that as much as anything there is a an interchange of ideas going on and that guidelines and evidence contribute to that interchange but do not determine it. This echoes Smith's obsevations at national level.21,22 Therefore seeing these processes at work in local authorities should not be a cause for rejecting the idea of EBM or evidence-based public health, so much as a reminder to see the evidence in the context of a broader set of processes, one of which is the democratic rough and tumble of local councils.
We would like to acknowledge our colleagues in local government for their participation in this study. We would like to thank Dr Heather Gainforth for her contribution to interviewing and data analysis.
This study was co-funded by National Institute for Health and Care Excellence and Economic and Social Research Council.
Conflict of interest
MK was Director of the Centre for Public Health at NICE from 2005 till 2014. GL is currently employed by NICE. SM was a member of the Public Health Advisory Committee of NICE.