Article Text
PDF
Abstract
Introduction Patient-reported outcome measures (PROMs) allow patients to self-report the status of their health condition or experience independently. A key area for PROMs to contribute in building the evidence base is in understanding the effects of using person-generated health data (PGHD), and using PROMs to measure outcomes of using PGHD has been suggested in the literature. Key considerations inherent in the stroke rehabilitation context makes the measurement of PGHD outcomes in home-based poststroke rehabilitation, which uses body-tracking technologies, an important use case.
Objective This paper describes the development of a preliminary item bank of a PROM-PGHD for Kinect-based stroke rehabilitation systems (K-SRS), or PROM-PGHD for K-SRS.
Methods The authors designed a method to develop PROMs of using PGHD, or PROM-PGHD. The PROM-PGHD Development Method was designed by augmenting a key PROM development process, the Qualitative Item Review, and follows PROM development best practice. It has five steps, namely, literature review; binning and winnowing; initial item revision; eliciting patient input and final item Revision.
Results A preliminary item bank of the PROM-PGHD for K-SRS is presented. This is the result of implementing the first three steps of the PROM-PGHD Development Method within the domains of interest, that is, stroke and Kinect-based simulated rehabilitation.
Conclusions This paper has set out a case study of our method, showing what needs to be done to ensure that the PROM-PGHD items are suited to the health condition and technology category. We described it as a case study because we argue that it is possible for the PROM-PGHD method to be used by others to measure effects of PGHD utilisation in other cases of health conditions and technology categories. Hence, it offers generalisability and has broader clinical relevance for evidence-based practice with PGHD. This paper is the first to offer a case study of developing a PROM-PGHD.
- person-generated health data
- patient-reported outcome measures
- patient monitoring
- telemedicine
- questionnaire design
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- person-generated health data
- patient-reported outcome measures
- patient monitoring
- telemedicine
- questionnaire design
Summary
What is already known?
Patient-reported outcome measures (PROMs) offer a standardised approach to evaluating and improving healthcare services by enabling patients to contribute to more precise evaluation of the effects of various health interventions; and they contribute to improving the evidence base in various areas of clinical care.
Utilisation of person-generated health data (PGHD) by patients promotes participatory health, as it has been suggested to increase their engagement, improve health management coordination with their care providers and increase their sense of social support and connectedness.
Measuring PGHD outcomes in home-based poststroke rehabilitation, which uses body-tracking technologies, is an important use case due to key considerations inherent in the stroke rehabilitation context.
What does this paper add?
This paper has demonstrated a case study of our PROM-PGHD Development Method, using Kinect-based stroke rehabilitation systems (K-SRS) as the case study, resulting in a preliminary item bank of PROM-PGHD for K-SRS.
The PROM-PGHD method may be used by others to measure effects of PGHD utilisation in other cases of health conditions and technology categories, and therefore has broader clinical relevance for evidence-based practice with PGHD.
Introduction
Patient-reported outcome measures (PROMs) allow patients to self-report the status of their health condition or experience independently.1–3 As such, PROMs enable patients to contribute to more precise evaluation of the effects of various health interventions, and they contribute to improving the evidence base in various areas of clinical care.4 PROMs offer a standardised approach to evaluating and improving healthcare services, and this is highlighted by key national projects to develop suites of PROMs for various health conditions in the USA, Europe and Australia.5
A key area for PROMs to contribute in building the evidence base is in understanding the effects of using person-generated health data (PGHD).6 PGHD are created, recorded and analysed by people, who are monitoring their health outside of a clinical care setting. They include health, wellness and other biometric data produced from technologies such as mobile applications, activity tracking devices and simulated rehabilitation technologies.7–9 Utilisation of PGHD by patients promotes participatory health, as it has been suggested to increase their engagement, improve health management coordination with their care providers and increase their sense of social support and connectedness.10–15 When patients better understand their illness, it may make them more active in improving their health behaviour.16
Using PROMs to measure outcomes of using PGHD has been suggested.9 A PROM of using PGHD, or PROM-PGHD, would allow patients to directly self-report their health outcomes or status as result of accessing and using their own PGHD. This may generate a deeper understanding of how PGHDs may impact patients’ health status and quality of life, and has significance in an era of increasing remote wearable and mobile patient monitoring. Similar to PROMs being used as a complement to other health outcomes indicators,5 PROM-PGHDs could also be used to complement existing patient-reported, and clinician-reported, outcome measures. This could contribute to a more accurate and comprehensive assessment of patients’ experiences of using PGHD from existing and new health information technologies. Consequently, PROM-PGHDs may offer a deeper understanding of the health outcomes and related impacts of those technologies.
Measuring PGHD outcomes in home-based poststroke rehabilitation, which uses body-tracking technologies, is an important use case due to key considerations inherent in the stroke rehabilitation context.8 These may include its high cost over a long period of time; difficulties in access to therapy17; the complexity of care required18 19 and the need for patients to undertake frequent, repetitive movement exercises appropriate to their condition to support improved health outcomes.20 21 Therefore, more convenient, practical and effective options for patients are needed, which technology interventions may provide. Simulated rehabilitation systems, for example, using Kinect (Microsoft, Redmond, Washington, USA) provide patients with simulated activities of daily living.22 Stroke therapy benefits from such systems have been previously reported.22–25
As patients use such systems, they produce PGHD in the form of therapeutic progress data. Those PGHD have the potential to be used by clinicians and by patients themselves to monitor and evaluate patients’ recovery more consistently.22 24 Similar to how PROMs allow for a more holistic evaluation of the effects of various health services and interventions,4 a PROM-PGHD for simulated poststroke rehabilitation technologies could provide a more precise assessment of those systems, and also increase understanding of how those systems impact the health status of patients.6
Objective
This paper describes the development of a preliminary item bank of a PROM-PGHD for Kinect-based stroke rehabilitation systems (K-SRS), or PROM-PGHD for K-SRS.
Methods
In response to the lack of a systematic way for patients to measure and self-report health effects they experience from using their PGHD—whether those effects are positive, negative or nil—the authors designed a method to develop PROMs of using PGHD, or PROM-PGHD. The PROM-PGHD Development Method was designed by augmenting a key PROM development process, the Qualitative Item Review,26 and follows PROM development best practice. Table 1 presents the steps of the PROM-PGHD Development Method.
Activities of the PROM-PGHD Development Method
This paper describes the use of the first three steps of the method, to develop a preliminary item bank of a PROM-PGHD for K-SRS. Validation and subsequent revision of the preliminary PROM-PGHD through the methods’ last two steps, that is, eliciting patient input,6 and final item revision, are reported elsewhere. The following sections are organised accordingly.
Step 1: literature review
In this step, a literature review relating to the two domains of the focus area, the health condition and the technology category, was conducted. This is to consider the socio-technical system context of the focus area, necessary for designing a PROM-PGHD that is appropriate for the needs of the patient cohort. Based on the objectives of this paper, the target health condition of this paper is stroke, and the technology category is Kinect-based simulated rehabilitation. Through the literature review, existing PROM items within the domains of interest were identified.
A consideration in this step is that PROMs are not generally found in the literature reporting on research in the technology category. However, this research may report patient experience or satisfaction with a health intervention measured in ways that are similar to PROMs.27 Hence, a variety of measures of satisfaction or experience from the literature in the technology category are included in the identification of PROMs for PGHD.
The authors conducted an extensive literature review, which examined the extent of PGHD utilisation in 41 included studies of Kinect-based simulated rehabilitation systems for stroke; full details appear here.8 The review identified existing PROMs within the poststroke health condition, and self-reported measures within the Kinect-based simulated rehabilitation technology category. These are listed in table 2.
PROMs identified through the first step of the PROM-PGHD Development Method, the literature review
The end of step one resulted in a range of possible PROMs or similar instruments to capture both the post-stroke and the simulated rehabilitation domains of interest. The individual items of the PROMs were analysed for appropriate ‘binning’ or ‘winnowing’ as described in the next step.
Step 2: binning and winnowing
In this step, individual items of the identified PROMs were assessed for inclusion to the preliminary PROM-PGHD, and then categorised in a process called ‘binning’, explained next. ‘Winnowing’ is the process of assessing the PROM items and determining whether they should be ‘winnowed’, or removed. Many of the items were removed as they could not be used to measure the effects of using PGHD. The criteria for winnowing items are as follows: (1) item content was inconsistent with the PROM-PGHD objective of measuring effects of using PGHD; (2) the item content was too specific to be applicable elsewhere, for example, it was too disease-specific or (3) items were redundant or confusing.
Table 3 shows examples of items that were removed from the winnowing process, and the reasons for their removal. Meanwhile, table 4 shows the list of PROM items that were retained after winnowing, and their reasons for inclusion.
Examples of items removed from the winnowing process
Alignment of identified PROM items with PGHD effects, and reasons for their inclusion
The retained items after winnowing were categorised in a process called ‘binning’. Binning, a term used in statistics to mean grouping items together, is the process of aligning the retained PROM items with reported effects on patients who have used PGHD. As an efficient way of targeting reported effects for this purpose, articles from a major journal special issue on PGHD7 were analysed inductively28; to categorise ways used in them to describe reported effects of using PGHD. The derived themes are a representative sample of PGHD utilisation effects from a variety of health information technologies, for different health conditions. These effects are listed below:
Influence health-related behavioural or attitude changes in patients.29
Influence patient management of their own care, due to changes in feelings about their health status.30
Influence interest in their care processes.14 30 31
Facilitate personal care goals.30 32
Influence relationship with care providers.10 11 30 31
The PROM items retained after winnowing were matched against these categories of effects of PGHD utilisation. Table 4 shows the PROMs identified from step 1, and their outcome measure items, with corresponding response options that were retained after the winnowing process. It also lists the reasons for the items’ inclusion. The ‘Reason for inclusion’ column describes why the items may be appropriate in measuring self-reported outcomes of patients’ utilisation of PGHD. The final column shows the alignment of the retained items with the thematically derived PGHD utilisation effects after the binning process. Only effects 1, 2 and 4 had items binned, or aligned with them.
PGHD has been used to describe data that have been generated and recorded by people, and interpreted by them,7 that is, people are accessing and using their own health information. Thus, PROM items that use the terms data and information both were included.
Step 3: item revision
The retained PROM items from step 2 were selected from pre-existing PROMs identified in the literature; as such they were not worded consistently, and their response options differed. To ensure that the resulting PROM-PGHD can be presented as one coherent test, and to reduce cognitive burden on respondents, the retained PROM items were revised in this step.
They were revised, where necessary, to better match the target health condition and technology category. Further revisions may also occur to ensure that the different PROM item response options are consistent; their content has similar wording; are concise and simple; are able to stand alone separately from the other questions and are worded to encourage use of available response options.26
A consequence of collecting several existing items from PROM instruments is the resulting variability of response options present.26 However, there is a lack of empirical evidence that any particular set of response options is better than others. Optimal response options may vary based on the individual items in question.26 Thus, to ensure that the resulting PROM-PGHD is capable of measuring the experiences of future respondents within the target domains, some response options of the retained items also were revised to include additional response types. This is to provide patients with varying response options to comment on in the fourth step. The fourth step is the process of eliciting patient input on the preliminary PROM-PGHD item bank, which is out of scope for this paper, and is reported elsewhere.
Table 5 shows the included PROM items from table 4. It then depicts any revisions conducted on the PROM items and their corresponding response options. Furthermore, it provides the reason/s for the revision.
Revision of identified items
The final thing to do in this step is to group the revised items according to their alignment with a PGHD effect. For this item bank, they are also numbered as a group according to their response options, that is, true/false statements, rating scales and multiple choice questions. The result is a preliminary item bank of a PROM-PGHD for K-SRS.
Results
Figure 1 presents the preliminary item bank of the PROM-PGHD for K-SRS. This is the result of implementing the first three steps of the PROM-PGHD Development Method within the domains of interest, that is, stroke and Kinect-based simulated rehabilitation.
Preliminary PROM-PGHD item bank for K-SRS. This figure presents the preliminary item bank of the PROM-PGHD for K-SRS. This is the result of implementing the first three steps of the PROM-PGHD Development Method within the domains of interest, that is, stroke and Kinect-based simulated rehabilitation. The items were first grouped according to the PGHD effects they aligned with, and then grouped further according to their response types. K-SRS, Kinect-based stroke rehabilitation systems; N/A, not available; PGHD, person-generated health data; PROMs, patient-reported outcome measures.
The items were categorised into the three PGHD effects that they were aligned with, represented as ‘sections’. The items under the second section (aligned with the second PGHD utilisation effect) could indicate how PGHD influences patient decisions on the management of their own care, due to how they felt about their health status. Thus, this effect was rephrased as the ‘self-management of care’ section. The item under the third section, aligned with the fourth effect, could indicate whether patients have sufficient access to their PGHD in a way they prefer, to facilitate personalised self-care strategies. Hence, this effect was rephrased as the ‘personalisation’ section.
Nine items fell under the first section on ‘behavioural or attitude changes’, eight items under the second section on ‘self-management of care’ and one item under the third section ‘personalisation’. All items under each section were then grouped according to their response types, to improve the flow of items when read.
Discussion
This paper has demonstrated a case study of our PROM-PGHD Development Method, using K-SRS as the case study. The result is a preliminary item bank of PROM-PGHD for K-SRS. It demonstrated the implementation of each step of the PROM-PGHD development process.
The next step is to present the developed item bank to patients, to elicit their input on the items themselves and to discuss their experiences with accessing and using PGHD. This is to gather concepts that may not have been covered by the current item bank, and is the fourth step of the PROM-PGHD method. This step is reported elsewhere.6
An interesting consideration discovered in this case study is the necessity of including, in the identification of PROMs, measures from the technology category to self-report satisfaction or experience. This is to ensure coverage of relevant items within the socio-technical context of the domains being measured. It is still a disciplined approach to the selection of outcome measures within the technology category, as the process identifies—through the literature review—those measures that have been used in a K-SRS setting. This is a unique and valuable aspect to the PROM-PGHD development process that future developers of a PROM-PGHD will need to consider.
Using PROMs to measure outcomes of using PGHD has been suggested.9 PROMs allow for a more holistic evaluation of the effects of various health services and interventions.4 Similarly, a PROM-PGHD would allow for a more precise, patient-centred assessment of such systems; and may increase understanding on how they could impact the health status of patients. It promotes participatory health within the K-SRS domain as it recognises the value of the patient experience in the assessment and evaluation of PGHD, and the technologies that produce them.33 PROMs may be used to understand the impact healthcare services have on the status and quality of life of patients.2 Similarly, it is hoped that the item bank would, in the future, assist clinicians in selecting appropriate K-SRS based on PGHD utilisation effects on patients; and for patients to understand how certain K-SRS could affect their management of their own health. Moreover, the method’s applicability for a variety of health conditions and technology categories make it broadly relevant for evidence-based practice in clinical work with PGHD.
This paper is the first to offer a case study of developing a PROM-PGHD for a target health condition and technology category. While there are studies that present the development of PROMs of a health condition,3 34 and measures to self-report experience or satisfaction with health technologies,27 35 36 there have been no studies presenting the development of a PROM of using PGHD.
Limitations
It was necessary to revise the content of existing PROM items identified from the literature review, due to the collection of several, existing items from PROM instruments, resulting in a variety of response options present.26 Moreover, to elicit patient input on acceptability of response formats, some of the response option types were revised. While revisions of the response options for uniformity is considered minor and unlikely to alter the items substantially,26 we recognise that changing the content of the items may introduce changes to the items’ function. This revision is essential, however, to develop a preliminary item bank to measure self-reported outcomes of PGHD utilisation. This process still follows best practice of searching the literature for existing concepts and items, and eliciting patient input.1 26 37 38 We believe this is preferable to starting the item development completely from scratch. Nonetheless, the suitability of the items is expected to be improved in the next step, where patient perspectives on their PGHD utilisation experience are gathered.
As described, the method used to identify the PGHD utilisation effects in step 2 (binning and winnowing) was an inductive thematic analysis of a recent, authoritative source (JAMIA special issue on PGHD7). The special issue compiled a range of applications and effects of PGHD across a variety of health conditions and technology categories. However, the list may not have covered all effects reported in the literature. The effects that should be measured however, will be verified and/or supplemented by patients in the next step. Open-ended questions will be asked of the patients around their experience of accessing and using PGHD from a K-SRS, to elicit any effects that may not have been covered in the initial list in step 2. The authors have already reported on one such discussion with patients here.6
While the preliminary item bank of the PROM-PGHD for K-SRS was organised as described for the purpose of presentation here, this is not its final, or complete form. The objective of this paper was to describe a formal process of developing a preliminary item bank, which could then be presented to patients to elicit their input on the items’ readability, appropriateness of wording and relevance to their experiences of accessing and using their PGHD in a K-SRS. Moreover, because the items were categorised according to the effects they align with, the resulting item bank seems unbalanced in terms of the number of items under each section. The third section in particular has only one item. It should also be noted that only PGHD utilisation effects 1, 2, and 4 had items binned or aligned with them. Consequently, effects 3 and 5, which measure changes in patient engagement with formal care, were not represented in the items identified from the literature. This will be a key area of enquiry, which will be explored in the fourth step of the PROM-PGHD method.
Conclusion
This paper has set out a case study of our method, showing what needs to be done to ensure that the PROM-PGHD items are suited to the health condition and technology category. We described it as a case study because we argue that it is possible for the PROM-PGHD method to be used by others to measure effects of PGHD utilisation in other cases of health conditions and technology categories. Hence, it offers generalisability and has broader clinical relevance for evidence-based practice with PGHD.
Acknowledgments
GLD would like to acknowledge the support of his organisational sponsor, Newman College (University of Melbourne).
References
Footnotes
Contributors All authors contributed equally.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.