An informatics approach to inter-professional management of low back pain: a feasibility study using the Omaha System
=====================================================================================================================

* Robin R. Austin
* Craig Schulz
* Karen A. Monsen

## Abstract

**Background** Low back pain (LBP) is a complex health care issue that often involves multiple providers across various care settings. Health information technology holds promise to improve care delivery by providing infrastructure for communication, clinical documentation and management of patient data. Standardised terminology is essential for interoperability and enables evaluation of clinical data generated by documentation in an electronic health record (EHR).

**Objective** The purpose of this study was to demonstrate the feasibility of mapping evidence-based practice (EBP) for conservative management of LBP to the Omaha System and foster inter-professional communication and collaboration among diverse practitioners and patients.

**Methods** EBP guidelines for non-invasive treatment of LBP were mapped to the Omaha System using a clinical expert approach with attention to content feasibility, linguistic validity and granularity of terms.

**Results** A clinical guideline for LBP management was developed consisting of **13 interventions for** *pain* and *neuro-musculo-skeletal problems.* The most common intervention categories were *case management* followed by *treatments and procedures, teaching, guidance, and counselling* and *surveillance*. Scope of practice overlap was identified among primary care, chiropractic and physical therapy practice.

**Conclusion** Use of the guideline may facilitate clinical documentation using the Omaha System for LBP management and has potential to generate meaningful data to evaluate clinical effectiveness and promote quality research. The use of encoded EBP evidence within an EHR can increase the use of available evidence, enable interprofessional communication, improve quality of care, and enhance the usability of data across care settings.

*   informatics
*   low back pain (LBP)
*   evidence-based practice (EBP)
*   electronic health record (EHR)

## BACKGROUND

Pain conditions are well recognised as an enormous burden to individuals and society, with back pain being the most commonly reported (National Health Interview Survey, 2012). Low back pain (LBP) is the fifth most common reason for all physician visits in the United States where approximately one quarter of U.S. adults report having had LBP for at least one whole day in the past 3 months.1 LBP ranks as the sixth most-costly health condition with estimates of incremental health care expenditures for spine problems calculated at $86 billion per year.1–3

Non-specific LBP is characterised by the absence of an identifiable underlying disease process and is often complex and challenging to manage. Evidence-based guideline recommendations emphasise conservative management involving patient education and analgesic medications of acetaminophen and nonsterodial anti-inflammatory preparations, with the cautious use of opioids.4 However, physician adherence to guidelines may not be optimal,5,6 where opioids represent the most commonly prescribed drug class for back pain,6 and more than half of regular opioid users continue to report back pain.6,7

Research suggests LBP that has become chronic is the most difficult to manage.5,6 Multiple providers managing an individual’s LBP can lead to fragmented care and frustration for both patients and practitioners.5 Such siloed healthcare can compromise communication for the health care team, lead to poor care coordination, and decrease the continuity of care.8

The National Institute of Health Pain Consortium9 was established to foster collaboration across provider disciplines and develop new innovative multidisciplinary approaches to pain management. Health information technology (HIT) holds promise to improving care delivery across multiple care settings by providing infrastructure and support for communication, clinical documentation and management of patient data.8

The electronic health record (EHR) provides infrastructure and support for communication, collaboration and care coordination.8 In order to allow for information exchange and shareable data within an EHR, it is essential to use data standards such as a standardised terminology within a documentation system.10 Standards and interoperability within HIT, such as an EHR, combine comprehensive clinical information from a diverse set of data sources to support many healthcare processes including direct patient care, population health management, quality improvement and comparative effectiveness research.9 These demonstrable linkages enable patient-centered care planning and documentation, facilitate information exchange and research ready data. The true value and power of exchanging holistic and integrative patient data to support patient-centered care has yet to be fully realised.11 A majority of patient data is collected and stored in a variety of untouchable sources with differing codes, identifiers and terminology. In an era of data-driven healthcare, it is critical to use tools that can facilitate documentation, communication and evaluation of healthcare practices.9

## OBJECTIVES

The purpose of this study was to demonstrate the feasibility of encoding EBP guidelines for LBP using a standardised interface terminology.12

### Evidence-based practice and managing LBP

Evidence-based practice (EBP) is a patient care problem solving approach to clinical decision making within a healthcare organization that integrates the best available science and clinical evidence from both the patient and the provider.13 Practice recommendations drawn from guidelines and systematic reviews follow a rigorous development process and are based on high-quality scientific evidence. For the purposes of this study, evidence was identified to provide a complete and integrative best practice approach to non-invasive management of LBP. A key point of the evidence used in this study is the inclusion of multidisciplinary providers as an integral part of the patients’ holistic plan of care.

### The Omaha System: interface terminology and health ontology

The standardised terminology selected for this study is the Omaha System. The Omaha System is a research-based comprehensive practice and documentation standardised taxonomy designed to describe client care.14 The Omaha System is a multidisciplinary interface terminology that is used for encoding EBP and enabling data capture and exchange.14 The Omaha System includes an assessment component (Problem Classification Scheme), a care plan/services component (Intervention Scheme), and an evaluation component (Problem Rating Scale for Outcomes).12 For the purpose of this article, ontology refers to a framework that represents a body of knowledge.15 A taxonomy refers to the science of classification and arrangement of terms and concepts at levels from general to specific, according to established principles and rules.12 The Omaha System is a point-of-care terminology that avoids redundancy, is a patient-centered, and allows for data to be easily comparable for providers and patients.12,15 This system provides a systematic method to capture, document, monitor and measure patient-centered care.

## MATERIALS AND METHODS

### Clinical management of LBP

The literature evaluating non-invasive management of LBP is vast and variable in quality. For this study, the research team reviewed the literature and selected a well-constructed recent systematic review of LBP clinical practice guidelines4,16 supplemented by a comprehensive systematic review of non-invasive treatments for LBP.17 This high-quality evidence was identified to closely reflect optimal clinical practice. The specific reviews selected were: *Clinical practice guidelines for the non-invasive management of low back pain: The Institute for Clinical Systems Improvement (ICSI) Health Care Guideline for low back pain,*4 *Non-invasive treatments of low back pain,* 16 *and Clinical practice guidelines for the non-invasive management of low back pain: a systematic review by the Ontario Protocol for Traffic Injury Management (OPTIMa) collaboration.*17 The team selected the ICSI guideline for its detailed treatment algorithm for the initial mapping to the Omaha System. The additional sources were then reviewed, and information from the sources related to specific roles identified in the management of LBP was incorporated. Six specific roles identified from the guidelines include: 1) Primary Care (PC), which includes both Medical Doctor4 and Doctor of Osteopathic Medicine,4 2) Doctor of Chiropractic (DC),4,16,17 3) Physical Therapist (PT),16 4) Registered Nurse (RN),16 5) Massage Therapist,16 and 6) Acupuncturist.4,16,17

### Omaha System terms and definitions

The Omaha System has three relational, reliable and valid components designed to be used together: Problem Classification Scheme (client assessment), Intervention Scheme (care plans and services) and Problem Rating Scale for Outcomes (client change/evaluation).14 The Omaha System provides a framework for clinical documentation, explains provider interventions and enables the ability to measure health outcomes. The Omaha System also provides structure for data collection, sharing and analysis. Furthermore, the Omaha System meets Medicare & Medicaid, Joint Commission and Community Health Accreditation Program guidelines and regulations and has been integrated into Systematized Nomenclature of Medicine – Clinical Terms.12

#### Problem classification scheme

The Problem Classification Scheme is a comprehensive, nonexhaustive, mutally exclusive classification designed to identify health-related concerns.12 The Problem Classification Scheme enables practitioners to separate essential from non-essential data objectively and efficiently, organise data elements, and help identify relationships and trends within the data.12 There are 42 non-overlapping concepts or problems, and each problem is identified in a unique definition and a set of signs/symptoms. For example, neuro-musculo-skeletal function is defined as the “ability of nerves and muscles, and bones to perform or coordinate specific movement, sensation or regulation”.12 The neuro-musculo-skeletal function problem has the following signs and symptoms: “limited range of motion, decreased muscle strength, decreased coordination, decreased muscle tone, increased muscle tone, decreased sensation, increased sensation, decreased balance, gait/ambulation disturbance, difficulty transferring, fractures, tremors/seizures and difficulty with thermoregulation”.12

#### Problem rating scale for outcomes

The Problems Rating Scale for Outcomes is a standardised instrument that measures three dimensions of health for each problem: patient’s knowledge (K), behaviour (B) and status (S) relative to each of the problem concepts. The Knowledge Behavior Status (KBS) scales are Likert-type scales from one (lowest) to five (highest). KBS is a comprehensive, systematic, recurring evaluation framework designed to measure client progress in relation to specific health-related problems.12

#### Intervention scheme

The Intervention Scheme is a comprehensive, nonexhaustive, mutually exclusive classification used to describe practitioners’ actions and activities.12 There are four terms that are linked to form an intervention: problem, category, target and care description (PCTCD). The problem includes the 42 terms within the Problem Classification Scheme. For example, a problem includes conditions such as Pain, Cognition, or Nutrition. The category is the action component of the intervention. There are four categories: teaching, guidance and counselling, treatments and procedures, case management and surveillance. There are 75 target terms that define a component to further specify the intervention or needs of the patient.12 The last level is related to client specific information or care description. The care description term is not standardised and may be customised to describe current EBP related to a given EBP guideline.12

Problem, category, target (P–C–T) terms can be used in any combination together for a total of 12,600 possible interventions.12 This allows to specificity and granularity of interventions to allow for data aggregation and to facilitate conceptual modeling.15 The PCTCD data provides additional detail to interventions, deeper evaluation of data and opportunities for analysis of free text data.15

### Encoding methods

The encoding methods for EBP are similar to a provider coding process in private office settings, ambulatory care or outpatient facilities or hospital care systems. Medical coding of Current Procedural Terminology and International Statistical Classification of Diseases and Related Health Problems (ICD-9 or ICD-10) is performed by skilled coders that abstract the information from the documentation, assign the appropriate codes, create a claim to be processed by reviewing the patient health record and translate narrative terms into medical coding.18 While codes vary a great deal in breadth and specificity, ambiguous codes or confusing narratives in the coding process are resolved through discussion with peers and other professionals.18

#### Encoding team and process

The research team (the authors: health care experts with graduate degrees and extensive experience in clinical guidelines (KM, RA) and the Omaha System (KM, RA) and content experts in LBP and integrative health research (CS)) encoded the EBP evidence using the medical coding method. The EBP evidence was presented to the research team. A LBP and informatics expert (RA) and an expert with fifteen years of experience using the Omaha System in practice and research (KM) selected the content for the feasibility study and completed the first initial coding. The coding was reviewed and analysed by all team experts, resolving differences by consensus.

#### Feasibility criteria

Content feasibility was determined by three factors: content validity, linguistic validity and granularity of terms.15 The research team used these factors to determine feasibility criteria and increase rigor and consistency within the review process. Content validity was defined by “the extent to which a measurement reflects the specific intended domain of content”.19 Linguistic validity between the evidence and the terminology is defined as the equivalence of meaning.20 Granularity was defined as the degree of detail or precision contained in data relative to the degree of detail within the evidence itself.21 Granularity was decided by adhering to the granularity within the evidence. The guideline was categorised into the Core Treatment Plan and specific numbered steps as identified from the algorithm.4 The Core Treatment Plan was identified as common core elements that are standard treatment in the management of LBP.4 Specific numbered steps were identified as treatment recommendations outside the Core Treatment Plan.4 The general overview of the evidence, relevancy to clinical practice and a comprehensive treatment plan was adhered to and achieved through consensus of the research team.

## RESULTS

There were thirteen interventions for six problems in three domains: mental health (psychosocial domain), pain and neuro-musculo-skeletal function (physiological domain) and health care supervision, physical activity and medication regimen (health-related behaviours domain). The most frequent problem was pain followed by medication regimen, neuro-musculo-skeletal function, physical activity, healthcare supervision and mental health. All of the four intervention categories were represented with case management being the most frequent, followed by treatments and procedures, teaching, guidance, and counseling and surveillance. There were six targets, with signs/symptoms-physical being the most frequent, followed by medical/dental care, medication prescription, wellness, exercises and signs/symptoms mental/emotional (Table 1).

View this table:
[Table 1](http://informatics.bmj.com/content/24/3/268/T1)

Table 1 Guideline for LBP management mapped to the Omaha system with health care provider roles

Content validity was achieved using exact phrases from the Institue for Clinical Systems Improvement (ICSI) guideline and recommendations. For example, the PCTCD intervention regarding education in the Core Treatment Plan of the guideline is *Clinicians should educate patients as an adjunct to other treatment. No standardised form of education is suggested. *The PCTCD intervention for medication regimen in the core treatment plan recommends* Non-steroidal anti-inflammatory drugs may be used for short-term pain relief in patients with acute and sub-acute LBP and cautious and responsible use of opioids may be considered for a limited period of time *(Table 1).

Linguistic validity was achieved by reaching 100% consensus related to the interpretation of the meaning between the Omaha System term and the guidelines. The pain problem was them the most frequent problem selected, followed by neuro-musculo-skeletal function and medication regimen (Table 1).

Granularity was addressed by developing a drill-down approach and adhering to the levels of granularity in the guidelines and recommendations.15 The Core Treatment Plan from the guidelines consisted of 9 out of 13 interventions, with the remaining four interventions describing specific treatment options outside the Core Treatment Plan.

Analysis of role descriptions associated with the guideline interventions showed that the PC provider role was associated with 12 of 13 interventions (92.3%), followed by chiropractic and physical therapy with 11 of 13 interventions (85%), RN 8 of 13 interventions (62%), MT 7 of 13 (54%) and acupuncture 6 of 13 (46%). Five interventions were within the scope of all roles: initial assessment and identification of red flags, promote self-management, instruct patients on pain and activity management, address fear-avoidance beliefs and address mental health concerns (Figure 1).

![Figure 1](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/24/3/268/F1.medium.gif)

[Figure 1](http://informatics.bmj.com/content/24/3/268/F1)

Figure 1 Venn diagram of health care providers for LBP

## DISCUSSION

This study examined the feasibility of mapping EBP guidelines for LBP to the Omaha System. Evidence-based LBP guidelines were mapped to problems, categories and targets, and guidelines were incorporated verbatim as Care Descriptions. The study demonstrates the feasibility of using the Omaha System to describe multi-disciplinary LBP management and highlights the diversity of practitioners that assess for pain, physical function, activity level and mental health needs. Mapping the LBP guidelines to six problems in three domains demonstrates the multi-faceted nature of LBP and points to the need for a comprehensive holistic approach to care. Further research is needed to evaluate the use of the EBP to promote improvement in LBP management, communication and care coordination.

Precise coding is critical for all intervention terms (PCTCD to accurately convey the meaning of the evidence and to maintain the quality of the data. The research team addressed content validity using a care description that matches the language within the evidence. This allowed the research team to specifically match the intervention based on the evidence and reduce redundancy or misinterpretation of the plan of care. The research team addressed linguistic validity, equivalence of meaning20 through the collaborative efforts of the teams Omaha System experts and clinical content experts during the encoding process. Omaha System questions were addressed by KM and the LBP content was addressed by CS. This allowed for an open informed dialogue throughout the process. Both terminology and clinical areas of expertise are essential to ensure accurate and clinically meaningful encoding.15 Granularity was achieved at the level of the guideline recommendations through customization at the care description level of the Omaha System.

This feasibility study is an initial step in the process of implementing standardised terminology within specialty fields of health care. It enables generating and collecting standardised data that may be used to evaluate adherence and impact of EBP on patient outcomes in a broad scope of provider types and aligns with previous findings regarding the feasibility of using the Omaha System for diverse practices, populations and problems.15 Standardised terminology can thus be used to link practice settings with academic research opportunities specifically to examine the relative effectiveness of care across multiple care settings addressing LBP.

In this study, the research team adhered to the evidence as published.4,16,17 Despite serious concerns related to abuse potential of opioids,6 non-invasive LBP guidelines still recommend conservative use of opioids for LBP management.4,16,17 It is critical that the governing bodies involved in disseminating LBP guidelines review and revise their recommendations to reflect the current best practices in opioid use.22

Post hoc analysis of the LBP guideline found that while there are common elements shared between all providers and there were striking similarities between the provider roles, no single team member role includes all interventions. These findings align with the literature supporting a team approach to LBP care, affirm the need for a team approach, and highlight the unique roles of health care team members in the non-invasive management of LBP.5 Interestingly, team members from Chiropractic and Physical Therapy shared the same interventions and thus appeared to have a similar scope of practice. Overall, these findings affirm the need for a seamless health information system to support excellence in team based.5

The value of standardised terminology and interoperable data is enhanced when a holistic patient record is shared across multiple care settings.8 This study is an initial step to providing a framework for encoding EBP evidence for LBP management and the potential to organise data related to best practices. Clinical documentation based on encoded EBP generates meaningful data that are appropriate for quality research and provide data to support the EBP. This study outlines the structure needed to continue to build evidence to support health outcomes for quality, multi-disciplinary LBP care.

## CONCLUSION

Evidence-based LBP guidelines were mapped to the Omaha System. This study demonstrates the feasibility of using the Omaha System to describe the interventions and details the scope of practice for multidisciplinary teams treating LBP, thus translating guideline knowledge into practice. Furthermore, using the Omaha system to document LBP treatment has potential to enhance team communication and improve data quality. Further research is needed to evaluate the implementation of this clinical decision support design within an EHR. A pilot study applying the Omaha System encoded evidence in clinical practice will enable evaluation from the perspective of care providers in a clinical environment to evaluate acceptability and usability of the guideline to support practice and for use in EHRs to generate and exchange data.

## Acknowledgments

The authors acknowledge the Omaha System Partnership for Knowledge Discovery and Health Care Quality within the University of Minnesota Center for Nursing Informatics.

## Footnotes

*   Conflict of Interest The authors declare that they have no conflicts of interest related to this research.

*   Received August 1, 2017.


*   © 2017 The Author(s). Published by BCS, The Chartered Institute for IT under Creative

Commons license http://creativecommons.org/licenses/by/4.0/

## References

1.  1.Deyo RA, Mirza SK, Martin BI. Back pain prevalence and visit rates: estimates from U.S. national surveys, 2002. Spine 2006;31: 2724–27. pmid:17077742.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1097/01.brs.0000244618.06877.cd&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=17077742&link_type=MED&atom=%2Fbmjhci%2F24%2F3%2F268.atom) 
    
    [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000241857900013&link_type=ISI) 

2.  2.National Health Interview Survey. Accessed 9 September 2016. Available at: [http://www.cdc.gov/nchs/nhis/index.htm](http://www.cdc.gov/nchs/nhis/index.htm)
    
    

3.  3.Martin BI, Deyo RA, Mirza SK, Turner JA, Comstock BA, Hollingworth W, et al. Expenditures and health status among adults with back and neck problems. JAMA 2008;299(6):656–64. doi:10.1001/jama.299.6.656; pmid:18270354.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1001/jama.299.6.656&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=18270354&link_type=MED&atom=%2Fbmjhci%2F24%2F3%2F268.atom) 
    
    [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000253095200023&link_type=ISI) 

4.  4.Goertz M, Thorson D, Bonsell J, Bonte B, Campbell R, Haake B, et al. Institute for Clinical Systems Improvement. Adult Acute and Subacute Low Back Pain 2011. Bloomington, MN: Institute for Clinical Systems Improvement. Available at: [https://www.icsi.org/\_asset/bjvqrj/LBP.pdf](https://www.icsi.org/_asset/bjvqrj/LBP.pdf), Accessed 12 September 2016.
    
    

5.  5.Williams CM, Maher CG, Hancock MJ, McAuley J, McLachlan A, Britt H, et al. Low back pain and best practice care: a survey of general practice physicians. Archives of Internal Medicine 2016;170(3):271–277
    
    

6.  6.Ivanova JI, Birnbaum GH, Schiller M, Kantor E, Johnstone BM, Swindle RW. Real-world practice patterns, health-care utilization, and costs in patients with low back pain: the long road to guideline-concordant care. Spine Journal 2011;11: 622–32. doi:10.1016/j.spinee.2011.03.017; pmid:21601533.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.spinee.2011.03.017&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=21601533&link_type=MED&atom=%2Fbmjhci%2F24%2F3%2F268.atom) 

7.  7.Hudson TJ, Edlund MJ, Steffick DE, Tripathi SP, Sullivan MD. Epidemiology of regular prescribed opioid use: results from a national, population based survey. Journal of Pain and Symptom Management 2008;36: 280–88. doi:10.1016/j.jpainsymman.2007.10.003; pmid:18619768; pmcid:PMC4741098.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.jpainsymman.2007.10.003&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=18619768&link_type=MED&atom=%2Fbmjhci%2F24%2F3%2F268.atom) 
    
    [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000259211100007&link_type=ISI) 

8.  8.O’malley AS, Grossman JM, Cohen GR, Kemper NM, Pham HH. Are electronic medical records helpful for care coordination? Experiences of physician practices. Journal of General Internal Medicine 201025(3):177–85. doi:10.1007/s11606-009-1195-2; pmid:20033621; pmcid:PMC2839331.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/s11606-009-1195-2&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=20033621&link_type=MED&atom=%2Fbmjhci%2F24%2F3%2F268.atom) 

9.  9.Institute of Medicine. Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary 2011. Washington, DC: The National Academies Press
    
    

10. 10.American Health Information Management Association. Data Standards, Data Quality, and Interoperability 2013. Accessed 9 September 2016. Available at: [http://library.ahima.org/doc?oid=107104#.WH\_e661abwB](http://library.ahima.org/doc?oid=107104#.WH_e661abwB)
    
    

11. 11.Maizes V, Rakel D, Niemiec C. Integrative medicine and patient centered care. Explore: The Journal of Science and Healing 2009;5(5):277–89. doi:10.1016/j.explore.2009.06.008; pmid:19733814.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.explore.2009.06.008&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=19733814&link_type=MED&atom=%2Fbmjhci%2F24%2F3%2F268.atom) 

12. 12.Martin KS. The Omaha System: A Key to Practice, Documentation, and Information Management 2009. 2; Omaha, NE: Health Connections Press
    
    

13. 13.Newhouse RP, Dearholt S, Poe S, Pugh L, White K. Johns Hopkins Nursing Evidence-Based Practice: Model and Guidelines 2007. Indianapolis, IN: Sigma Theta Tau International Honor Society of Nursing
    
    

14. 14.Omaha System. The Omaha System: Solving the Clinical Data-Information Puzzle 2014. Accessed 9 September 2016. Available at: [http://www.omahasystem.org/](http://uri)
    
    

15. 15.Monsen KA, Neely C, Oftedahl G, Kerr MJ, Pietruszewski P, Farri O. Feasibility of encoding the Institute for Clinical Systems Improvement Depression Guideline using the Omaha System. Journal of Biomedical Informatics 2012;45(4):719–25. doi:10.1016/j.jbi.2012.06.004; pmid:22742937.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.jbi.2012.06.004&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=22742937&link_type=MED&atom=%2Fbmjhci%2F24%2F3%2F268.atom) 

16. 16.Chou R, Deyo R, Friedly J, Skelly A, Hashimoto R, Weimer M, et al. Noninvasive treatments for low back pain. Comparative Effectiveness Review No. 169 (Prepared by the Pacific Northwest Evidence-Based Practice Center under Contract 290-2012-00014-I.). AHRQ Publication No. 16-EHC004-EF 2016. Accessed 5 October 2016. Rockville, MD: Agency for Healthcare Research and Quality. Available at: [www.effectivehealthcare.ahrq.gov/reports/final.cfm](http://www.effectivehealthcare.ahrq.gov/reports/final.cfm)
    
    

17. 17.Wong JJ, Côté P, Sutton DA, Randhawa K, Yu H, Varatharajan S, et al. Clinical practice guidelines for the noninvasive management of low back pain: A systematic review by the Ontario Protocol for Traffic Injury Management (OPTIMa) collaboration. European Journal of Pain 2016. Accessed 5 October 2016. doi:10.1002/ejp.931. Available at: [http://onlinelibrary.wiley.com/doi/10.1002/ejp.931/epdf](http://onlinelibrary.wiley.com/doi/10.1002/ejp.931/epdf). doi:10.1002/ejp.931.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1002/ejp.931&link_type=DOI) 

18. 18.What Is Medical Coding?. Accessed 9 September 2016. Available at: [https://www.aapc.com/medical-coding/medical-coding.aspx](https://www.aapc.com/medical-coding/medical-coding.aspx)
    
    

19. 19.Carmines EG, Zeller RA. Reliability and Validity Assessment 1991. Newbury Park, CA: Sage
    
    

20. 20.Mellon J. Nuffeild’s Working Papers Series in Politics: Examining Survey Translation Validity Using Corpus Linguistics 2001. Accessed 9 September 2016. Oxford, UK: Nuffeild College. Available at: [https://www.nuffield.ox.ac.uk/politics/papers/2011/Jon%20Mellon\_working%20paper%202011\_08.pdf](https://www.nuffield.ox.ac.uk/politics/papers/2011/Jon%20Mellon_working%20paper%202011_08.pdf)
    
    

21. 21.The Granularity of Data and the Grain of Existence. Veryard Projects: The Granularity of Models. 9 September 2016. Available at: [http://www.users.globalnet.co.uk/~rxv/infomgt/grain.htm](http://www.users.globalnet.co.uk/~rxv/infomgt/grain.htm)
    
    

22. 22.Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain-United States. MMWR Recommendations and Reports 2016. Accessed 12 September 2016. 65(No. RR-1):1–49. Available at: [http://dx.doi.org/10.15585/mmwr.rr6501e1](http://dx.doi.org/10.15585/mmwr.rr6501e1). doi:10.15585/mmwr.rr6501e1.
    
    [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.15585/mmwr.rr6501e1&link_type=DOI) 
    
    [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=26987082&link_type=MED&atom=%2Fbmjhci%2F24%2F3%2F268.atom)