Original Research•

Characteristics of positive feedback provided by UK health service users: content analysis of examples from two databases

Abstract

Background Most feedback received by health services is positive. Our systematic scoping review mapped all available empirical evidence for how positive patient feedback creates healthcare change. Most included papers did not provide specific details on positive feedback characteristics.

Objectives Describe positive feedback characteristics by (1) developing heuristics for identifying positive feedback; (2) sharing annotated feedback examples; (3) describing their positive content.

Methods 200 items were selected from two contrasting databases: (1) https://careopinion.org.uk/; (2) National Health Service (NHS) Friends and Family Test data collected by an NHS trust. Preliminary heuristics and positive feedback categories were developed from a small convenience sample, and iteratively refined.

Results Categories were identified: positive-only; mixed; narrative; factual; grateful. We propose a typology describing tone (positive-only, mixed), form (factual, narrative) and intent (grateful). Separating positive and negative elements in mixed feedback was sometimes impossible due to ambiguity. Narrative feedback often described the cumulative impact of interactions with healthcare providers, healthcare professionals, influential individuals and community organisations. Grateful feedback was targeted at individual staff or entire units, but the target was sometimes ambiguous.

Conclusion People commissioning feedback collection systems should consider mechanisms to maximise utility by limiting ambiguity. Since being enabled to provide narrative feedback can allow contributors to make contextualised statements about what worked for them and why, then there may be trade-offs to negotiate between limiting ambiguity, and encouraging rich narratives. Groups tasked with using feedback should plan the human resources needed for careful inspection, and consider providing narrative analysis training.

What is already known on this topic

  • Our scoping review systematically identified all empirical studies presenting evidence on how positive feedback received from service users creates change in healthcare settings, and which were published in English before 18 March 2022. 68 papers were included, with qualitative (n=51), quantitative (n=10) and mixed (n=7) methods, and only two interventional studies. Most outcomes described were desirable. These were categorised as (1) short-term emotional change for healthcare workers (including feeling more hopeful, motivated and empowered); (2) work-home interactional change for healthcare workers (including improved home-life relationships); (3) work-related change for healthcare workers (including reduced burnout, increased staff retention and increased team performance). We identified a knowledge gap around the content of positive feedback, as most included studies focused on the change that was created, and did not provide specific detail on the feedback content that created this change.

What this study adds

  • This study describes the characteristics of positive feedback examples selected from two feedback databases collected in the UK. It describes heuristics for identifying positive feedback in large electronic databases, and presents a preliminary typology, which proposes that tone (positive, mixed), form (factual, narrative) and intent (grateful) are critical characteristics of feedback to consider. It demonstrates that feedback is sometimes presented as a lived experience narrative, which can provide a rich description of the cumulative impact on a service user of their interactions with a range of health service units, healthcare providers, healthcare professionals, influential individuals and community organisations, sometimes over a multi-year period. We demonstrate that some feedback aggregated in these two databases has important ambiguities that potentially constrain its value, such as the precise target of grateful feedback, or the separation between positive and negative elements of mixed feedback. To obtain the greatest value, some feedback examples require close inspection.

How this study might affect research, practice or policy

  • Our study might influence the design of feedback collection systems; we conclude that people commissioning these systems should consider mechanisms to maximise utility by limiting feedback ambiguity, but without compromising the ability of service users to describe their experience in a narrative. Our study might influence resource allocation in groups tasked with interpreting feedback; we conclude that such groups should seek to allocate the required human resources to closely inspect feedback, and should consider providing training in narrative analysis and interpretation methods. Our study might influence future research around the characteristics of service user feedback; for example, future studies might extend our heuristics for identifying service user feedback, or extend our feedback typology. Our study might draw attention to the value of positive feedback for health service improvement. Our study might draw attention to feedback published in Care Opinion as a resource for research.

Introduction

Health services around the world receive substantial quantities of feedback from their users,1 the exchange of which can be initiated by either the provider or the user.2 In England, the Care Quality Commission has demonstrated that the aggregation of very recent feedback can be used to identify high-risk priorities for inspection.3 Case studies have documented how analyses of service user feedback have contributed to quality improvement initiatives.4 5 A review of UK empirical studies concluded that health service providers tend to focus on complaints and concerns raised in feedback,6 and raised concerns that providers allocate insufficient resources to analysing feedback in ways that lead to change.6 Many people give feedback to service providers because they want to give praise for their treatment,7 and a multi-method study has concluded that most feedback provided through online mechanisms has a positive tone.7 If healthcare providers prioritise the processing of negative feedback, and also fail to adequately resource the processing of feedback as a whole, there is a risk that potentially useful positive feedback may be left substantially unattended to, indicating a mismatch between the types of feedback that service users provide, and the purposes for which this data is collected.

A better understanding of the mechanisms by which positive feedback can support quality improvement might support its use within health services. We conducted a systematic scoping review to map the available evidence for how positive feedback can be used to create health service change, which included 68 papers from 32 countries. We found preliminary evidence that receiving positive feedback can create short-term emotional benefit for healthcare workers, and can benefit healthcare systems (including through improved team performance, increased staff retention and reduced staff burnout). These benefits were not evenly distributed, due to structural factors that influenced whether healthcare staff received positive feedback. For example, staff allocated to nightshifts were less likely to receive positive feedback due to less service user contact time, and ambulance staff were less likely to receive positive feedback, because service users did not know how to contact them. Most included papers used qualitative methods, and observed routine use of feedback. We concluded that interventional research is required to assess the effectiveness and cost-effectiveness of positive feedback in improving organisational effectiveness, or changing outcomes for staff.8

Much of the research cited in the review was ambiguous about the nature of this feedback. For example, in included qualitative studies, healthcare staff were frequently reported to have described the benefits of receiving positive feedback, but rarely discussed the content of feedback that created these benefits. This means that there is currently little research knowledge about the form or content of positive feedback, or about how form and content can facilitate or enable health service improvement.8 The aim of the current paper is to describe the characteristics of positive feedback collected from UK healthcare service users. The objectives are (1) to develop heuristics for identifying positive feedback within electronic databases (‘heuristic development’); (2) to share an annotated set of feedback examples to enable future analyses (‘data sharing’); (3) to describe positive content identified in feedback (‘content description’). Heuristics are mechanisms which are efficient to apply, and frequently successful at achieving desired results.9 A heuristic approach to the identification of positive feedback items was critical when our chosen data sources contained nearly 600 000 records.

Methods

We analysed 200 feedback items from two databases described in online supplemental appendix 1. These databases were selected because they used contrasting mechanisms for feedback collection, which had the potential to influence feedback content. Positive content was defined as a response from healthcare service users, families or the community indicating concordance between desired and actual experiences regarding care or treatment.8 Database items were in scope if they contained any positive content.

Our exploration began by retrieving a convenience sample of twenty items per database. We used this sample to identify preliminary heuristics for identifying positive feedback, and five preliminary positive feedback categories, selected for their utility for feedback analysts, guided by a steering group with expertise in using feedback for health service improvement work (see the Acknowledgements section). Categories were non-orthogonal. Most items could have been placed in more than one category. Our heuristics were iteratively applied and refined to select a total of 200 items, as were category names and definitions, the latter guided by our steering group. Our heuristic development process is described in full in online supplemental appendix 1. For variation in form and content, 40 items per category were eventually retrieved. We identified the dimensions of a typology that subsumed these categories.

Results

Table 1 presents our final categories and definitions, organised into three dimensions (tone, form, intent).

Table 1
|
The five categories and category definitions identified in our analysis, organised by typology dimension, presented with number of items retrieved per category

For objective 2 (data sharing), online supplemental appendix 2 presents our 200 positive feedback items. For objective 3, online supplemental appendix 3 presents numbered feedback examples to illustrate category content variation, with shorter examples also provided in the body of the text below.

Objective 3: content description

Positive-only feedback

Some positive-only feedback lacked specific detail about what went well, which may limit its value for health service improvement. Example 1 is a single sentence in which this detail is completely absent.

There is nothing you do not do well. (Example 1, FFT)

Positive-only feedback from Care Opinion was generally longer and more detailed than the Friends and Family Test (FFT). Example 2 (online supplemental appendix 3) draws attention to staff attitudes that were important to the contributor (‘very professional’, ‘caring’, ‘friendly’). These may have value for health service improvement work because they provide insights into the characteristics of services that this user cared about.

Some positive-only feedback was presented as a direct message from a service user to a staff member, or a health service unit. The exact target of the message could be ambiguous, particularly for the FFT, which did not include structured questions enabling target identification (Example 3).

Everything I learnt is spot on. I think you are doing a very good work so I say, keep it up. (Example 3, FFT)

Mixed feedback

In some mixed feedback, the divide between positive and negative content was distinct. In Example 4 (online supplemental appendix 3) negative content begins with ‘My only criticism … ’, and all other content is positive.

Some mixed feedback had a less distinct separation between positivity and negativity. In Example 5, the contribution implies a period in which a difficult situation was not fully acknowledged, describes a response now perceived as thorough, but identifies that the treatment location is perceived negatively.

Now that my daughter’s situation has been fully acknowledged, the ED service has been doing everything it can to get her an emergency bed anywhere in the country. It is just a shame that this will most likely be in X. (Example 5, FFT)

Mixed feedback can sometimes include contrasting positive and negative evaluations of the same object.

A fantastic service just a shame it’s not a permanent service. (Example 6, FFT)

Longer mixed feedback can contain a substantial number of positive, negative and neutral elements, as in Example 7 (online supplemental appendix 3).

When providing mixed feedback, service users sometimes offer a distinction between individual-level evaluations of staff, and organisational-level evaluations of services and systems. In Example 8 (online supplemental appendix 3) there is a hint of a systemic problem with nurses having an ‘endless amount of tasks’.

Grateful feedback

Some grateful feedback was in the form of a factual statement, accompanied by an expression of affect.

X’s care saved my life, very grateful. (Example 9, FFT)

Some expressions of gratitude were prompted by specific individual staff members who were perceived to have offered care that was beyond the expectation of the service user (Examples 10 and 11, online supplemental appendix 3).

Some expressions of gratitude were directed to specific units within a named NHS unit, such as all ward staff, rather than to specific members of staff (Example 12, online supplemental appendix 3).

Factual feedback

The defining feature of factual feedback was that it provided a clear and objective summary of what went well, with the potential to be replicable in the future.

A thorough assessment very quickly that summarised all the issues and offered a realistic treatment plan. (Example 13, FFT)

Clearly explained the essential information. Provided leaflets to take away which was useful. (Example 14, FFT)

As for other categories, items on Care Opinion were typically longer than in FFT (Example 15, online supplemental appendix 3).

Some feedback reflected on the personal impact of good practices, providing insights into inner experiences that might not otherwise be available to healthcare staff.

The staff have helped me a lot. My key worker/1:1’s with nurses & HCA’s has really helped. Having someone to talk to when I am anxious has helped. Relaxation at OT has helped me feel calmer. (Example 16, FFT)

Narrative feedback

Narrative feedback was rare in FFT, and frequent in Care Opinion. It often described the cumulative impact on a service user of their interactions with a range of health service units, healthcare providers, healthcare professionals, influential individuals and community organisations, and also inner events (such as changes in thinking that supported depression recovery). Hence, even though this feedback was directed to the NHS, it also provided information about the influence of non-NHS entities and personal factors on health and well-being. The length and richness of many narrative feedback items were noteworthy, as some items were in excess of 1000 words, representing a substantial investment of writing effort by their contributor.

Six examples of narrative feedback have been provided in online supplemental appendix 3. These have been selected to demonstrate a range of ways in which narrative feedback is expressed. Example 17 is one of a small number of FFT narrative examples. Example 18 summaries treatment experiences spanning 2.5 years. Example 19 provides a rich description of interactions with a broad range of staff and services, illustrating how these intersected to impact on their perceived quality of treatment. Example 20 focuses strongly on the internal mental events that occurred for a service user in the relation to their treatment. Example 21 contrasts the positive impact of a treatment setting on their problems with alcohol, with the negative impact of their home environment. Example 22 provides specific detail about the mechanisms by which a coaching intervention impacted positively on the mental health and life opportunities of the contributor.

Discussion

Principle findings

We developed preliminary heuristic approaches to identifying positive feedback, which would benefit from further development and evaluation, including implementation as an automated algorithm. Positive feedback was present on a substantial scale in both the Care Opinion and FFT databases, and was categorised as positive-only, mixed, narrative, factual and grateful. A broader typology may consider the tone, form and intent of positive feedback items. Narrative feedback sometimes described non-health service influences on health and well-being, such as beneficial interactions with community groups. Challenges to the use of feedback in quality improvement initiatives include disambiguating and separating positive and negative content in mixed feedback, and resolving ambiguities about feedback target, for feedback presented as direct messages to individuals or health service units.

Relationship to prior work

The NHS Choices website allowed service users to share experiences of NHS.10 A corpus linguistics analysis assessed 500 items against criteria established by narrative theorists.11 It found that 302 (60.4%) items contained narratives, providing confirmatory evidence that narrative feedback is an observable phenomenon in health service feedback databases. Narrative feedback was most likely for health service units with the lowest quality ratings, suggesting a hypothesis that the nature of the health service experience will influence the form of feedback provided by service users, as well as its content.

In the items that we analysed, some grateful feedback was in the form of direct messages to individuals or health service units. This is in keeping with a definition of gratitude as the communication of an emotion or state which signals recognition that others have done something to benefit us, often for the purpose of reciprocating the other’s actions.12 While service users expressing gratitude to healthcare staff or services may not have intended their expressions of gratitude as a form of feedback, our scoping review found that expressions of gratitude can draw attention to aspects of provision which have worked effectively8 and hence can highlight possibilities for service maintenance or improvement. For example, postcards and letters expressing gratitude to palliative care units also served a function of offering encouragement to the purpose of the service.13 This is in keeping with a research meta-narrative, identified by Day et al in a literature review, of gratitude as an indicator of quality of care.14 While some service users habitually offer gratitude to healthcare staff in the expectation of ensuring continuation of good treatment,15 we anticipate this purpose as unlikely for grateful feedback provided to feedback databases, since the perceived opportunity to influence personal treatment will be low.

A challenge of using feedback in service improvement work is the range of factors that can influence whether feedback is provided at all. Challenges include an over-representation of people with very positive or very negative experiences,16–18 and who are younger, more educated and with a long-term condition.19 This makes obtaining a balanced view of service quality difficult. Perhaps, this points to mixed or narrative feedback providing the most useful insights, but mixed feedback may still focus on describing the extremes of good and bad experiences. We noted that narrative feedback was frequently provided about addiction and mental health services, which may reflect duration or impact of treatment, but may be influenced by the routine use of therapeutic storytelling approaches in these settings,20 so that people using these services may be sensitised to narrative communication approaches. A sentiment analysis of 33 654 reviews of 12 898 medical practitioners in the New York State area found that, on average, reviews were short (mean 4.17 sentences long), with longer commentaries more likely to be negative.21 However, it is not yet clear whether negative sentiment is associated with length across the range of health service activities. In our own analysis, we found a range of examples of longer feedback about mental health services that had a broadly positive sentiment about their contribution to health.

While healthcare systems frequently valorise feedback collection and provision, features of healthcare systems may influence the impact that feedback has in practice. The INQUIRE study7 has drawn attention to the potential for healthcare staff concerns about service user feedback to act as a barrier to its use, noting the presence numerous editorials and opinion pieces written by, and for, health professionals who were sceptical about feedback provided online. Their scoping review integrated work by Patel and colleagues, who found that general practitioners in England described being concerned about the usability, validity and transparency of feedback collected online.22 As noted in our introduction, concerns have been raised that UK health provider allocate insufficient resources to analysing feedback in ways that lead to change, and tend to focus on concerns and complaints.6

Strengths and limitations

A strength is that we considered two databases with different feedback collection characteristics (solicited vs unsolicited feedback; different numbers and styles of stimulus question; different interfaces for providing responses). All categories were present across both databases, hence they may generalise to other databases. The 200 positive feedback items that we collected will provide a resource for secondary analysis. A limitation is that we only used FFT data from a single trust. There are significant variation across and within NHS trusts in how FFT data is generated and processed,23 and hence we cannot draw conclusions on the characteristics of FFT feedback as a whole. Contributor characteristics were not available in the data sources, so their impact is unknown.

Implications for practice

We have argued that potentially useful positive feedback about UK health services may be unattended on a substantial scale, if healthcare providers allocate insufficient resources to analysing feedback in ways that lead to change, and focus on concerns and complaints rather than positive evaluations.6 In some of the positive feedback examples described in our paper, there were clear ambiguities, such as mixed feedback where there was no clear separation between positive and negative elements, and grateful feedback where the target of gratitude was not clear. In a system that has resource limitations for feedback processing, then these kinds of ambiguity may limit their value for improving service quality, by requiring substantial analyst capacity to process. People who commission and implement feedback collection systems should consider raising the value of feedback by prioritising changes that reduce unnecessary ambiguity, for example, by asking what it was about a service that made a difference, and what it was about the person and their personal situation that meant this was helpful for them.

We have noted however that some contributors have chosen to provide feedback in the form of reflective narratives. In examples that we have inspected, being enabled to provide feedback as a narrative has allowed a contributor to make a clear and contextualised statement about what worked for them, and why, at a point in time when they understand and felt ready to share these experiences, rather than at a time and in a form predetermined by health services. This is particularly critical for mental health services, where attitudes to treatment can continue to change in the years after treatment,24 perhaps underpinned by a developing personal understanding around mental health. Some approaches to reducing ambiguity, such as more structured data collection questions, might reduce the potential for narrative feedback to be contributed, perhaps precluding some valuable insights. Hence, service designers will need to examine how to negotiate trade-offs in the design and implementation of feedback collection services, between reducing ambiguity, and maximising narrative richness. Because of the challenges of processing ambiguous or narrative data, then groups tasked with making use of service user feedback should carefully plan to optimise the human resources needed within their resource constraints, for example by providing narrative analysis and interpretation training.

Conclusions

This paper has explored positive feedback presented in databases collected in the UK. Current feedback is diverse and heterogenous, ranging from a single sentence to substantial narratives containing more than 1000 words. Current positive feedback will require substantial human capacity to identify and comprehend, and ambiguity inherent in this feedback may limit its value for service improvement work. Future efforts may focus on enabling the contribution of feedback that is less ambiguous (supporting easier analysis), while avoiding compromising the ability of contributors to provide feedback containing rich and informative detail about their experiences of treatment, and its personal context.

Overview

Article metrics
Altmetricopen-url
Dimensionsopen-url