Abstract
The National Health Service in the United Kingdom categorises research and research-like activities in five ways, such as ‘service evaluation’, ‘clinical audit’, ‘surveillance’, ‘usual practice’ and ‘research’. Only activities classified as ‘research’ require review by the Research Ethics Committees. It is argued, in this position paper, that the current governance of research and research-like activities does not provide sufficient ethical oversight for projects classified as ‘service evaluation’. The distinction between the categories of ‘research’ and ‘service evaluation’ can be a grey area. A considerable percentage of studies are considered as non-research and therefore not eligible to be reviewed by the Research Ethics Committee, which scrutinises research proposals rigorously to ensure they conform to established ethical standards, protecting research participants from harm, preserving their rights and providing reassurance to the public. This article explores the ethical discomfort potentially inherent in the activity currently labelled as ‘service evaluation’.
Introduction
The National Health Service (NHS) in the United Kingdom categorises research and research-like activities in five ways, such as ‘service evaluation’, ‘clinical audit’, ‘surveillance’, ‘usual practice’ and ‘research’. 1 Service evaluation is widely employed in the clinical research setting.
This article looks to raise an important issue for ethical review in the health services; that of the ethical rigour in service evaluation. Service evaluation laudably seeks to assess how effectively a patient service is achieving its intended goals. However, a concern has been identified by the authors that the very nature of this form of enquiry, commonly seen as not requiring specific approval from Research Ethics Committees (RECs), may also be at risk, inadvertently, of bypassing ethical principles.
Recently, the authors of this article were involved in a service evaluation to investigate patient experiences and outcomes of the care provided in two different heathcare settings.
The project methods included non-participant observations and interviews, but, as this enquiry had been predefined as service evaluation, this project was not submitted for review by the relevant REC. The project was given approval by the Caldicott Guardian, responsible for reviewing the arrangements for handling patients’ data 2 and also approved by the local Quality Improvement Team. This latter process focused on potential disruption to clinical areas, such as protecting participants’ confidentiality but not directly the ethical conduct of the study.
Once these approvals had been received, the researchers were permitted, quite properly in service evaluation, to proceed. The particular growing disquiet as the enquiry progressed was that an ethical dimension did not appear to be addressed and questions arose as to whether this was more appropriately seen as research. If it had been so, a full and rigorous ethical review would have been required.
Ensuring ethical behaviour and standards
There are fundamental, well-understood theories underpinning and ensuring ethical behaviour and standards. Virtue ethics focuses on the role of moral character of the individual from which choices and actions follow. Principle-based ethics, however, serves to guide morally right actions and is based on respect for autonomy, non-maleficence, beneficence and justice. 3 –5 From this, ethical rules, policies and guidance are widely employed by REC to make ethics or ethical principles more explicit. 3 Although ethical rules, policies and guidance are, indeed, derived from ethical principles, there are debates as to whether rules, policies or guidance can truly reflect morals. Accordingly, it is recognised that a more rule-based ethic in the real world of research also encompasses the ideals of virtue ethics to provide valuable guidance in establishing research integrity and the consequent accountability for the research process. 5 As indicated at the outset, five categories of research and research-like activities are identified as ‘service evaluation’, ‘clinical audit’, ‘surveillance’, ‘usual practice’ and ‘research’. Any activity collecting and/or analysing data on health or health services must be classified under one of these headings 1 (Table 1). All the above activities must adhere to ethical standards.
Differentiating clinical audit, service evaluation, research, usual practice and surveillance work. 1
However, once the research-like activity has been classified, different regulatory and ethical requirements are endorsed. Only activities classified as research are eligible for review by the REC. The classification of projects, at this point, therefore, significantly changes the extent to which they are subject to institutional oversight and formal ethical governance.
The uncomfortable question that arises is whether the current governance of research and research-like activities provides sufficient ethical oversight for the category identified as service evaluation. Distinguishing between the categories of research and service evaluation is not always clear as the guidance and definitions might suggest; some projects could fit into either category with relatively little or no changes in focus or content. As the experience of the researchers above illustrates, projects classified as ‘service evaluation’ may involve researcher-led activities and interventions that might equally be seen as research in other contexts or by other institutions. It can be then discomforting and difficult to understand why these projects receive no ethical review from REC, while other projects, involving similar types of activity, receive extensive ethical review. Current governance policy and processes require the individuals conducting projects classified as service evaluation to follow ethical principles and patient protection laws which should be trusted, virtuous and acted upon ethically. This assumption contrasts with the more principle-based ethics practised through the REC, where projects are examined in great detail, and each element of the project is expected to be defended against a pre-existing ethical framework. 6 It is argued here that it is the predetermined classification that can guide the researchers’ ethical decisions and actions. However, it must be that the first imperative of any research enquiry is the ethical consequence of the activity not merely what may seem as the more obvious, and even desirable, classification.
In light of these concerns, it serves to reflect on the historical development and implementation of research enquiry in general and of service evaluation in particular and explore the development of the ethical implications of categorising forms of research activity.
Development of the ethical milieu
Research ethics first became of critical concern at the Nuremberg trials after the Second World War. Dreadful crimes against humanity were identified, following immoral human experiments on concentration camp prisoner, undertaken under the guise of research. As a result, in August 1947, the Nuremberg Code was introduced giving the set of 10 ethical principles for conducting human experiments. 7 As ethical sensitivity developed, the World Medical Association developed the Declaration of Helsinki in 1964 8 seen as the cornerstone of modern human research ethics, whatever the current classification of such research that may currently exist.
The NHS Health Research Authority (HRA)
1
defines the activities as follows: ‘Service evaluation’: Designed and conducted solely to define or judge current care. ‘Research’: The attempt to derive generalisable new knowledge including studies that aim to generate hypotheses as well as studies that aim to test them. Specific questions generate a protocol-driven project to derive new knowledge and understanding. ‘Clinical audit’: Designed and conducted to produce information to inform delivery of best care, which serves to identify if desired standards of service delivery are being met. ‘Surveillance’: Designed to manage outbreak and help the public by identifying and understanding risks associated. ‘Usual practice’: Designed to investigate outbreak or incident to help in disease control and prevention (p. 4).
The categories of ‘research’, ‘service evaluation’ and ‘clinical audit’ have consistently been present in health service guidance since the development of Research and Development (R&D) governance in the early 1990s. However, the range of terms used to classify R&D activities, and their definition, changed. A brief account of these developments can serve to identify some of the processes, motivations and definitions that have contributed to the category of service evaluation.
R&D governance 1948–1990
With reference to Figure 1, it can be seen that, initially, research and related activities were given little attention within the NHS; there was no centralised governance for research and any such research governance occurred at a local level. 9 It was not until 1989 that the British Government appointed a National Director of R&D, tasked with overseeing patient-based activity related to teaching and research in the clinical environment in the NHS. 9 At this time, key terms used to describe R&D activities in governance documents included research, and clinical audit, the latter key to the quality assessment processes for clinical practice. 10

R&D governance 1991–2000
Despite this movement towards research governance, progress was slow and little central funding was allocated to the R&D department. 9 However, the implementation of the European Good Clinical Practice (GCP) regulations and guidelines in 1990 11 marked a ‘sea change’ in the primacy of ethical behaviour in research.
In 1991, local REC were established to review the ethical quality of proposed research studies, at this time, predominantly biomedical research. 12 These committees were under the aegis of local health services, with no centralised oversight of research activities in the health service as a whole. 13 Each local NHS health board established its own administrative structure and management according to the local interpretation of the latest Research Governance Framework.
In 1996, Regional Health Authorities were established who was responsible for, among other things, research development. 9 For the first time, research and development activities in the NHS were incorporated into a clear framework for governance.
In 1997, the first national system for ethical review was established with the development of the Multi-Centre REC responsible for research across different local governments. However, the Multi-Centre REC did not have the authority over local RECs. 13 On a somewhat separate pathway, it was in 1997 that the idea of service evaluation emerged, with a particular focus on primary care, and was put forward by Evans and Steiner. 14 Their suggestion was that this term could be used to describe a range of quality improvement studies where the specific purpose would be to judge the quality of care against existing approved standards.
In 1998, in response to the widely recognised GCP, the first national research strategy was developed. 15 A funded NHS R&D programme was established with the aim of improving the research environment within the NHS. 9 The approach to research and development was becoming more strategic and unified, but oversight of the ethical conduct of individual projects continued to function at a local level and, arguably, activity under the umbrella of service evaluation developed by a means of pragmatic gradualism.
R&D governance 2001–2016
In 2001, the European Directive, responsible for the GCP regulation and guidelines, 16 required more rigorous governance of research activities within the NHS. From this, the existing system of ethical review was established under a centralised REC. 17 REC now acted as a gatekeeper to ensure all research studies were carried out in accordance with ethical standards on their approval. Unlike previous systems, the current REC gives ethical advice and review for studies that are identified as research and, importantly, are required to be independent of any local health service. 17,18
In 2001, the Department of Health published the Research Governance Framework for Health and Social Care, 17 introducing a definition of research as ‘the attempt to derive generalisable new knowledge by addressing clearly defined questions with systematic and rigorous methods’. The Research Governance Framework was central to changing the landscape for NHS research review. According to this Framework, all researches were required to meet the ethical and scientific standards established by research governance requirements. Critically, for the authors’ thesis, in contrast, activities, such as clinical audit, service evaluation and practice development, fell within a clinical governance framework, which was intended to safeguard the quality of care and healthcare delivery. The Research Governance Framework stated that there was no need for clinical audit, service evaluation and practice development to undergo ethical review. 19 Despite the obvious rigour identified above, it is hard to find the rationale for the essential categorising of studies into research or non-research, with service evaluation firmly in the latter. In 2006, the NHS introduced a new/revised system for classifying research and development activities under the headings of research, clinical audit and service evaluation. 20 In this guideline, service evaluation was defined as procedures, whereby medical service was judged ‘…by providing a systematic assessment of its aims, objectives, activities, outputs, outcomes, and costs’ (p. 9) 21 while clinical audit was defined as a ‘quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change’ (p. 9). 21 Figure 1 serves to demonstrate the evolution of this different terminology and associated definitions. As before, only research required review by the REC. 20 The decision as to classification could be made according to the recommendation from the local R&D office, arguably adding considerably to their work burden if, by the same token, reducing that of the REC. 22
In 2009, the initial typology of research and development activity was increased to five categories: ‘clinical audit’, ‘service evaluation’, ‘research’, ‘usual practice’ and ‘surveillance work’. 23 In 2011, this was reduced again to ‘research’, ‘clinical audit’ and ‘service evaluation’. It is difficult to track these changes through government documentation, but different classifications can be found in local NHS documents, such as guidelines published by NHS Wirral. 24 No explicit rationale could be located for the change of terminology but, as in previous iterations, only activities classified as research required review by the REC.
In 2013, there was further alteration, with a revised version of the 2009 terminology with the current five categories. 1 The same classifications are reviewed again in 2016. In all these iterations, only activities classified as research require, or indeed, of interest to the current debate, are permitted, review by the REC.
The HRA, established in 2011 to raise awareness of the rights of patients and the public in health and social care research, was also tasked to co-ordinate the REC and promote transparency in research. It constitutes the lead R&D office in the United Kingdom. 25 It is worth noting that, in England, the HRA is responsible for recommending which studies to go forward for review by a REC, while in Scotland, R&D is the decision maker. 25
In summary, governance of R&D in the UK NHS has developed significantly since the implementation of the GCP in 1991, and any activity meeting the criteria of research is now subject to independent review according to centralised standards. However, the definition of research, although very specific, excludes activities which might, arguably, be regarded as research in other contexts or others’ views. Since the 1990s, health research has shifted from being almost entirely biomedical in focus towards a proliferation of studies that focus on the quality of care, 3,26 and the question arises as to whether the research governance has properly responded to this significant shift. It is clear that the typology of research and development activities is intended to facilitate and clarify both the organisation of research governance and the practical and timely conduct of R&D activities within the health service. Although no rationale was found for differentiating research activities and non-research activities, the report written by Evans and Steiner 14 was clearly influential on the activity of service evaluation as identified currently, despite the lack of a clear mandate. It has to be pragmatically acknowledged that, by excluding certain activities from the category of research, the number of studies requiring review might be kept at a manageable level, allowing non-research activities to proceed relatively untroubled. This exclusion of certain activities from ethical review can also be found in other national systems of research governance, for example, New Zealand and Australia both exclude certain activities from the category of ‘research’ and thus from ethical review. 27,28 However, this exclusion has been criticised. For example, Gerrish and Mawson 19 and Wade 29 suggest that every quality improvement study should be categorised as research and even studies not deemed research still require independent ethical review. Surprisingly, these critiques have not generated any real debate, which may be due to the understandable paucity in published service evaluation studies. 29
Service evaluation in the NHS
What is indisputable is that the volume of health research being carried out in the NHS has increased enormously over the past few decades. 30 However, as alluded to above, until relatively recently, very little of this research was about the health service itself. In 2000, the newly developed Service Delivery and Organisation (SDO) encouraged the development of research projects to investigate the quality of care, and the experiences of service users. 30 During the first 5 years of the SDO, its steadily increasing budget reflected the growth in the amount of commissioned research from £167,000 in January 2000 to £7 million in July 2006. Between 2001 and 2006, a total of 23 research projects were commissioned by the SDO, with an average budget of £102,000 per project. 30 These projects, meeting predetermined SDO themes, were identified by the SDO as ‘research’, 30 but arguably if they were to accord with the recent and latest definitions, they might have been classified as ‘service evaluation’. At the very least, there is inconsistency in decision-making and subsequent ethical activity.
Recording service evaluation activity
It is difficult to give any accurate statistics as to the prevalence of service evaluation in the NHS in any given year, but, as an informal illustration, the South East Scotland Research Ethics Service 31 estimated that they had given advice on a total of over 1300 studies over the past 6 years, of which approximately 70% of these were classified as not research (see Table 2 and Figure 2). While this is not representative data, it is nevertheless interesting to note the proportion of studies advised as non-research.

Advice given to the nature of research activity in South East Scotland Research Ethics Service from 2010 to 2015. 31
Service evaluation: a distinct enquiry?
Despite the lack of rationale for clear justification for its emergence, service evaluation has become embedded as a form of service enquiry distinct from research. Gerrish and Mawson 19 and Wade 29 point out that, in the NHS R&D typology, the categories of research and service evaluation necessarily have many similarities. They both include projects that start with a question, expect the answer to change or influence clinical practice, may involve the collection and analysis of new data or the analysis of already existing data, and both depend on using an appropriate method and design to reach sound conclusions. 19,29
The most marked difference between the categories is that a service evaluation can only employ an intervention that has already been undertaken in the health service. Put simply, research investigates what should be done, whereas service evaluation investigates whether it is being done and to what standard. 1,23,24
Although guidance on making the distinction between service evaluation and research is available, 1,23,24 the distinction can be difficult to agree or make in practice. The NHS Quality Improvement Strategy (QIS), 2011, has acknowledged that there can be a grey area when it can be difficult to decide where the project fits, and R&D would only advise the researcher of the likely classification. Casarett et al. 26 and Wade 29 agree that documents discussing the distinction between audit, service evaluation and research often base this distinction on the methodological ‘process’ of the project rather than the defined objectives. This means that projects on the border between service evaluation and research could easily be aligned to either category with relatively small, or even no, adjustments to methodology or design. For researchers facing time and resource constraints, it may be more attractive to position their project towards service evaluation, thereby avoiding the need for an in-depth ethical review. Equally, it may seem disproportionate that small changes to the design or presentation of a project may have such significant consequences for the degree of ethical oversight required for the project. These are difficult and ethical issues in themselves to confront.
The current system of research governance has evolved an ‘all or nothing’ approach to ethical review, arguably inadequate if it means that activities with potentially significant ethical consequences are not reviewed. Two problems are positioned here. First, by adopting the absolute ‘review/no review’ approach, current research governance ignores the grey areas of research ethics. This presents an absence of ethical scrutiny. Second, there may be an implicit assumption that the label service evaluation poses thereby less of a risk to participants than research, when this may not be the case. Challenging this, Twycross and Shorten 32 argue that the standards expected of service evaluation in terms of design, data collection and analysis should be at least as high as for research because service evaluation or audit may ‘quickly move findings to create tangible practice change’. Service evaluation is often embedded within day-to-day practice, the latter being the very means of evaluating service provision. 33
Ethical review
Ethical review is axiomatically beneficial for research. According to Wade, 29 ‘Ethical concerns arise when the involved parties have different interests or values in a situation in which a potential conflict exists between the burden and risk imposed on patients or others, including society, and the likely benefit’.
Ethical review provides guidance for researchers and safeguards for participants. Although service evaluation does not require specific approval from a REC or R&D approval, ethical principles must still be adhered to in terms of such as consent, anonymity, data protection and privacy of patients. 29 However, it can be challenging for researchers, particularly novice researchers, to conduct a service evaluation in clinical settings without any ethical advice and support from an ethics committee.
It is difficult to conduct a meaningful review of studies classified as service evaluation, as, as indicated, few published studies are identified in this way. 26 The authors can only speculate as to why this is the case. However, based on information gathered from the South East Scotland Research Ethics Service and the authors’ own experiences, one suggestion might be that many service evaluations are undertaken for purely pragmatic, service-led reasons, not deemed a priority for peer-review journal publication. In this way, it could be argued that most service evaluations vanish from view.
Service evaluation case studies
In light of this deficit of published service evaluations, three examples are discussed below. A detailed description will be given of the three service evaluation case studies in the following to demonstrate that they easily have met the criteria for research. It is noted that this case analysis is not for punitive purposes, but purely to demonstrate the dilemma and disquiet.
Evaluation of PIMA point-of-care CD4 testing in a large UK HIV service 35
This service evaluation was undertaken to evaluate the performance and patient acceptability of a new laboratory service for patients with HIV. Capillary blood samples were collected from consented participants for the new laboratory service. The participants were asked to complete a five-point Likert questionnaire to assess their views about the laboratory service. Surprisingly, a study involving blood sampling and direct patient involvement was still classified as service evaluation not requiring ethical review from the NHS. Studies that collect participants’ blood are normally defined as research because collecting patients’ tissues or anything from their body will require the highest ethical standard. The Human Tissue Act 2004 36 stated that all tissues collected require consent and advice from the REC. Although this study could entail risk, it had met the criteria for service evaluation. Even if not deemed research on the determined criteria, it is argued that the study carried a form of risk that should have merited closer ethical scrutiny.
The effect of anaesthetist grade and frequency of insertion on epidural failure: a service evaluation in a UK teaching hospital 37
This service evaluation investigated prospectively all patients undergoing either intra-abdominal or thoracoabdominal surgery who received epidural analgesia. Health records were examined to identify the reason for, and the method of care for, epidural catheter removal. Although it analysed existing data, it was interesting that neither ethical approval nor informed consent from patients concerned were required. Using patients’ data often raises ethical concerns. The classification of this study allowed easy access to the relevant databases without any reference to ethical guidance.
A service evaluation of the feasibility of a community-based consultant and stroke navigator review of health and social care needs in stroke survivors 6 weeks after hospital discharge 38
In this last service evaluation, focusing on stroke survivors, all the stroke survivors took part in a joint review under the auspices of the Department of Health’s National Stroke Strategy Quality Marker. The joint review clinics were held twice a month by a stroke consultant, who completed a comprehensive and holistic assessment of the individual, alongside quality-of-life questionnaires and focus groups. This was in addition to the standard stroke care and constituted a new intervention whose value had yet to be proven. It is argued that such a study involving patients should perhaps have included a REC-driven assessment for risk of harm.
The authors contended that those concerned in the above-defined service evaluations were not encouraged to think about such ethical considerations, distracted by the comforting label of service evaluation.
As noted at the outset, the authors of this article were also involved in a service evaluation. This service evaluation in an acute clinical setting included interview and observation with staff and patients without either ethical approval or informed consent. The study could be intrusive and key personal data were included without ethical scrutiny. At face value, it was hard to foresee risk of harm in such a defined service evaluation without the required ethical prompt axiomatic in research. The evolving ethical concern for participants’ well-being led to the evolution of this position paper.
All the above were classified as service evaluation. However, similar methods of the study and population groups could be found in many studies classified as research required to provide extensive justification for their methods and recruitment strategies with detailed safeguards put in place to protect research participants. What has been established is that it can be hard to determine whether a study is research or not. Although guidance on making the distinction between service evaluation and research is available, and indeed looks superficially clear, the distinction can be difficult. However, it can be argued that the arbiter of this distinction can be the ethical review seen as a ‘gatekeeper’ for a study’s category. This, rather than the label, determines the route to, or away from, ethical review. Despite the fact that the process can be complicated and time-consuming, it can help the researchers to identify potential harm, which will not only protect participants but also protect researchers. It is accepted that, at present, a service evaluation may not require specific approval from an REC or R&D, but ethical principles must still be uppermost and adhered to for the protection of participants and vulnerable. 18,29 It is important for all undertaking research activity, however defined, of any sort to reflect on their own role in the study and critically think about the ethical issues during the study. 19,29,34
Discussion
Ethical principles remain at the heart of all research-like activities. The development of ethical review processes should be the guardian for all studies. Although regulation for service evaluation has been established, it has been argued that ethical dilemmas in such classification and guidance clearly exist. Fundamental is that in any research activity, the researchers must do no harm. 7 Before any research activity or service evaluation involving individuals is undertaken, the foreseeable risks and discomforts, as well as any anticipated benefit for the individual, are identified. Risk of harm can, indeed, on occasions be difficult to predict. 39 As service evaluation is commonly embedded in the practice it is evaluating, it can pose particular challenges and complexities, particularly for a novice researcher. 33 It is not an easier route. In any research activity, REC guidance is to assist decision-making when encountering ethical dilemmas. 3 Inevitably, there may be gaps in a rule-based system. 32,40 Reviews from the REC act as the default system, a safety net, that may reveal potential harm and/or minimise such harm, ensuring that the potential benefits outweigh any risk. 39 The problem identified is that in service evaluation, this vital step is not present, the choice of service evaluation even preferred to avoid the potentially complicated ethical review process.
Conclusion
The difficult debate put forward here is whether the main ethical concern is wrongly labelling enquiry as research and non-research activities. No published evidence could be found to explain fully the purpose of the current classification system. Whatever the classification, or when this is determined, the key driver of all such activity is its ethical component and this truism goes back over 70 years.
Footnotes
Acknowledgements
The authors are grateful to Mr Alex Bailey who provide expertise and greatly assisted us in this project.
Conflict of interest
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
