Article Text
PDF
Abstract
The study purpose was to conduct heuristic evaluation of an interactive, bilingual touchscreen-enabled breastfeeding educational programme for Hispanic women living in rural settings in Nebraska. Three raters conducted the evaluation during May 2013 using principles of Nielson’s heuristics. A total of 271 screens were evaluated and included: interface (n = 5), programme sections (n = 223) and educational content (n = 43). A total of 97 heuristic violations were identified and were mostly related to interface (8 violations/5 screens) and programme components (89 violations/266 screens). The most common heuristic violations reported were recognition rather than recall (62%, n = 60), consistency and standards (14%, n = 14) and match between the system and real world (9%, n = 9). Majority of the heuristic violations had minor usability issues (73%, n = 71). The only grade 4 heuristic violation reported was due to the visibility of system status in the assessment modules. The results demonstrated that the system was more consistent with Nielsen’s usability heuristics. With Nielsen’s usability heuristics, it is possible to identify problems in a timely manner, and help facilitate the identification and prioritisation of problems needing urgent attention at an earlier stage before the final deployment of the system.
- breastfeeding
- educational programme
- heuristics
- human–computer interaction
Commons license http://creativecommons.org/licenses/by/4.0/
Statistics from Altmetric.com
Introduction
Breastfeeding has been recognised as an optimal method to nourish an infant as it provides complete nutrition necessary for its full physical and mental development.1–3 Breastfeeding protects against common infections and diseases throughout childhood with effects lasting even into adulthood. Babies who are not fully breastfed for the first 3–4 months of age are at higher risk of health problems, such as gastroenteritis,1,4,5 respiratory infection,1 otitis media6 and urinary tract infection. 6 Benefits of breastfeeding for the mother include a rapid return of post-partum uterine tone and post-partum weight loss, higher bone mass density after menopause, delay of ovulation and decreased risk of breast and ovarian and endometrial cancers.7–9
Introduction of complementary foods should not happen before the sixth month and breastfeeding should continue at least until 12 months of age.3 According to the Centers of Disease Control and Prevention, 74.6% of breastfed babies are ever breastfed in the United States, 44.3% breastfed at 6 months and only 23.8% continue breastfeeding to 12 months.2 In Nebraska, where approximately 30% of the population lives in rural areas,10 the number of ever breastfed and exclusively breastfed babies at 6 months stay below the national average at 72.8% and 13.4%, respectively.2 Healthy People 2020 (HP2020) aim to increase the proportion of ever breastfed babies to 81.9%, 60.6% at 6 months and 34.1% at 12 months.4 The current outcomes are far behind public health goals.
Socio-economic, cultural, attitudinal and familial factors are associated with breastfeeding practices.11,12 Disparities in breastfeeding practices exist across different groups,7,8 and a recent review has indicated the need for more research on culturally tailored interventions to achieve the breastfeeding goals established by HP2020.13 Among Hispanic mothers, perceptions of infant rejection to breastfeeding and milk insufficiency are often reported,14 and formula supplementation is practised among one in three mothers.15 Previous research suggests that initiation of breastfeeding may be more frequent among urban women (59%) compared to that among rural women (49%).16 In urban settings, several longitudinal studies have prospectively examined breastfeeding initiation and discontinuation.17–22 Among urban women, breastfeeding initiation and continuation may be influenced by several factors including participation in the WIC programme,17 support from the health system,19–21 maternal depression21 and return to work or school.18,23
A number of health information technology (IT) solutions assist health care professionals in providing efficient quality care. Technology innovations facilitate presenting culturally relevant and tailored health information24,25 There has been increasing trend of using technologies for supporting health behaviour change such as enhancing physical activity,26–31 healthy diet,32–34 smoking35,36 and self-regulation of emotions.37
Health IT evaluation is complex as it intends to serve various functions. Evaluation involves hardware and information processes in a given environment.38,39 Lack of attention to health IT evaluation reflects an inability to achieve system efficiency, effectiveness and satisfaction. 38,39 There are unique methodological challenges in evaluating how populations use and navigate new technologies. Usability challenges have to be met in programme development and should meet all users’ needs. The errors generated due to complex nature and shortcomings of health care applications have been emphasised in various studies.38 Many health IT usability studies have been conducted to explore usability requirements, discover usability problems and design solutions.38,39 The poor design of interactive health care systems increase their complexity thereby hindering their utilisation and uptake by the users.38,39 Evaluating health technology interventions helps in identifying usability issues such as ease of navigation so as to design and develop solutions in a timely and an effective manner.40
Several methods are available to assess and improve the usability of interactive computer enabled applications. Among the expert-based methods, heuristic evaluation is commonly used.41 Nielson’s heuristics are a set of usability engineering principles developed to identify issues in user interface design and involves analysis of the interface. 42,43 Ten essential criteria constitute the Nielson’s heuristics and they include visibility of system status, match between system and the real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, recovery from errors and help and documentation. 42 Severity indexes ranging from negligible usability issues represented by zero (I don’t agree that this is a usability problem at all) to one (Cosmetic problem only: need not be fixed unless extra time is available on project), two (Minor usability problem: fixing this should be given low priority), three (Major usability problem: important to fix, so should be given high priority) and four (Usability catastrophe: imperative to fix this before product can be released) are utilised in assessing the extent to which such errors can hamper utilisation of the system.42 The majority of the errors can be identified in a systematic process, such as instructions and functionality problems, simplified representation and improved labeling.43 Heuristic evaluations require fewer resources than other methods to predict major usability problems.44,45 According to Nielson’s heuristics, a minimum of three and maximum of five raters are required to conduct a heuristic evaluation.45 Additional raters might not necessarily discover useful information.45
The objective of this study was to conduct a heuristic evaluation of an interactive, bilingual, touchscreen-enabled breastfeeding educational programme for Hispanic women living in rural settings.
Methods
Conceptual framework of Patient Education and Motivation Tool
We modified Patient Education and Motivation Tool (PEMT) to design and develop an interactive, bilingual, touchscreen computer based, breastfeeding educational support programme integrating a variety of cognitive-behavioural theories. PEMT facilitates health information and messages to be adapted depending on the psychosocial elements including attitude, self-efficacy, expectations, personal norms and social influences.46–49 The entire educational content is broken down into a series of modules, each module into sub-modules and each sub- module into a series of educational messages. Each message is then presented using various multimedia formats (such as audio, video, text and images). The primary objective of the PEMT is to present health information in an interactive tailored manner considering multiple factors influencing health status and health behaviours. The computer-based breastfeeding educational support programme aims to provide modular, culturally relevant, bi- lingual (English and Spanish) education tailored to the needs of the mothers (Figures 1 and 2). The details of the proposed breastfeeding educational support programme have been outlined in a prior study.
The breastfeeding educational support programme is delivered with a kiosk containing a touchscreen-enabled computer with the breast-feeding educational support programme on a mobile cart in an outpatient clinic.
Evaluation method
Three usability raters (AJ, DP and CA) were involved in the evaluation of the breastfeeding educational support programme. Two usability experts (DP and CA) used Nielsen’s heuristics while reviewing the user interface in May 2013. One usability expert (DP) was a research assistant with over one year of experience in public health informatics and evaluation of health technologies. The other (CA) was a biological chemist and also a research assistant in the area of public health informatics and consumer health technologies. Both individuals worked at the Mobile Emerging Technologies and Population Health Outcomes Research laboratory in the College of Public Health at the University of Nebraska Medical Center. Both evaluators individually reviewed the user interface of the breastfeeding support programme and generated a list of heuristic violations. A severity scale ranging from 0 (no usability problem) to 4 (usability catastrophe) was applied to all violations [18]. A third rater (AJ), who is an expert in the area of evaluation of health technologies, reviewed and resolved any disagreements for any differences that occurred to identify the potential heuristic violations. The heuristic evaluations were carried out in the interface, programme sections and interactive educational modules. The interface included the navigational features and the various buttons that were implemented for the users to self-pace an interactive computer-based breastfeeding programme (Figure 1). The programme sections included the introductory screen, table of contents and four different modules that gathered information about the user’s socio-demographics, self-efficacy, knowledge and Breastfeeding Attrition Prediction Tool (BAPT). The bi-lingual interactive breastfeeding educational content (English and Spanish) presented in the form of text, audio and images was also evaluated (Figure 2). The study was approved by the University of Nebraska Medical Center Institutional Review Board (IRB protocol #430-12-EP).
Statistical analysis
Descriptive analysis was performed to report means and standard deviations for the continuous variables, while frequency analysis was performed for the categorical variables. All results of the analysis were performed using SAS version 9.1.
Results
A total of 271 screens were evaluated, and 97 heuristics violations were observed. The various screen types included (a) interface (n = 5), (b) programme screens included socio- demographics (n = 9), self-efficacy assessment (n = 23), knowledge assessment (n = 90), BAPT (n = 60) and (c) educational content screens (n = 43). A total of 97 violations were identified, and majority of them were related to the interface (8 violations/5 screens) and programme modules (89 violations/266 screens). Other heuristic violations were related to socio-demographics (7 violations/9 screens), followed by knowledge component (64 violations/90 screens), self- efficacy (14 violations/23 screens) and BAPT (3 violations/60 screens). There was only violation reported for the interactive computer-based breastfeeding educational content (1 violation/43 screens; Table 1).
Overall heuristic violations
The most important heuristic violations included recognition rather than recall (62%, n = 60), followed by consistency and standards (14%, n = 14) and match between the system and real world (9%, n = 9; Figure 3). No violations were reported for ‘flexibility and efficiency of use’ and ‘aesthetic and minimalist design’.
Heuristic violations by programme components
The majority of the heuristic violations were reported for the assessment component (n = 88). For the assessment component, the most common heuristic violations reported were recognition rather than recall (67%; n = 60), while in the interface component, it was help and documentation (38%; n = 3), and for the educational component, it was user control and freedom (100%; n = 1). The number of heuristic violations for assessment, interface and education components is described in Figure 4.
Severity of heuristic violations
The average severity index of overall heuristic violations was 2 (SD = 0.88). Majority of the violations had minor usability issues (73%, n = 71). There were very few violations that were in the severity grade of 3 (n = 5) or 4 (n = 5). Majority of the violations were in the severity grade of 2 (n = 71; Table 1).
Severity of heuristic evaluations by programme components
The highest number of severe heuristic violations (grade 4) was seen among the assessment programme component (n = 5). No catastrophic issues were encountered for the interface. Sometimes clicking on the screen to receive more information would not achieve the desired result. The major usability violations were user control and freedom (60%, n = 3). Cosmetic issues represented 17% (n = 16) of the violations, and the most important categories included match between system and real world (44%, n = 7) and consistency and standards (31%, n = 5). Other suggestions included specifying a clearer indication in the title that more than one option can be selected. The programme was designed in English and Spanish, but the control panel does not offer the option to change languages to user. The assessment component had the majority of the catastrophic violations that were related to the visibility of system status (n = 5) compared to the educational module that had one catastrophic violation related to the user control and freedom (Table 2).
‘I’ represents interface, ‘A’ represents assessments and ‘E’ represents educational modules.
The common reasons of heuristic violations have been outlined in Table 3. The suggestions provided give an insight into the various possible changes that can be incorporated to the proposed interactive, bi-lingual touchscreen-enabled breastfeeding educational support programme for Hispanic women living in rural settings.
Discussion
The results of the heuristic evaluation reported a total of 97 violations based on the 271 different screen types evaluated in an interactive, bi-lingual touchscreen-enabled breastfeeding educational programme. The heuristic evaluation allows the software developers to identify and correct errors within each component of the IT system in a timely manner,40 and can be used to identify strengths in the design of the system. The violations identified in this study were related to the (a) interface (8 violations/5 screens), (b) socio-demographics (7 violations/9 screens), (c) self-efficacy (14 violations/23 screens), knowledge (64 violations/90 screens), (d) BAPT (3 violations/60 screens) and (e) educational content (1 violation/ 43 screens). The maximum number of heuristic violations included: recognition rather than recall (62%, n = 60) followed by consistency and standards (14%, n = 14), match between the system and real world (9%, n = 9), visibility of system status (5%, n = 5), help and documentation (4%, n = 4), user control and freedom (3%, n = 3), error prevention (2%, n = 2) and helping users recognise, diagnose and recover from errors (1%, n = 1). No violations were reported for ‘flexibility and efficiency of use’ and ‘aesthetic and minimalist design’. Identifying these heuristic violations will allow the conservation of resources compared to other methods to predict major usability problems.44,45 Additionally, heuristic evaluation is a better predictor of problems that are encountered by endusers and also identifies more severe usability problems.50,51
A severity rating ranging from 0 (no usability problem) to 4 (catastrophic) was utilised in classifying the degree of errors. By the identification of the severity rating, developers are able to prioritise problems needing urgent attention at an earlier stage before the final deployment of the system. The average severity index for the 97 violations identified was 2 (SD = 0.88). The violations due to recognition rather than recall were mainly restricted to the knowledge assessment module and related to the lack of specification of the expected answer to the questions asked before presenting the feedback. However, all the violations had a rating of 2 implying minor cosmetic problems. The raters felt that additional feedback should be provided on the same screen when a certain response was made to the question, as chosen by the user. This would help the user to make a clear distinction between the correct and the incorrect response. This is consistent with literature, where raters using Nielson’s heuristics are able to identify instructions and functionality problems, simplified representation and improved labeling.43 Majority of the violations due to consistency and standards were reported in the self-efficacy assessment module (8%, n = 8), followed by the knowledge module (4%, n = 4) and socio-demographics (1%, n = 1).
Identification of these violations in a systematic process using heuristic evaluation is suitable for a wide range of IT systems, and can be used in the implementation and development process. Some of the common heuristic violation findings included rephrasing of the words and the use of phrases or functions, which are more familiar to the user than system oriented and specifying a clearer indication in the title that more than one option can be selected. The violations due to match between the system and the real world were mainly based on the preference of users having a drop down menu for selecting age rather than having addition and subtraction symbols to alter their age. Violations due to visibility of system status were related to failure of the system to load the desired page when the suggested icon is clicked evaluation results encountered very few catastrophic (n = 5) and major usability violation (n = 1) in the breastfeeding educational support programme. The vast majority of the violations represented cosmetic issues, mainly regarding recognition than recall. In addition, some minor usability problems were identified related to match between system and real world, and consistency and standards. It would be important to correct these heuristic violations before the final deployment of the breastfeeding educational programme. Correcting the violations found in such evaluations can increase system efficiency, effectiveness and satisfaction among users.38,39
There are several limitations of using heuristic evaluation compared to other usability engineering methods. Evaluators detected different sets of problems, and exist for both novice and experienced evaluators.52 Therefore, it is beneficial to have an additional evaluator to resolve any disagreements between the raters, both in the detection of usability problems as well as the assessment of problem severity. The method heavily relies on the skills and expertise of the usability professionals, and these professionals might lack domain knowledge and may overlook domain related usability problems. One way to overcome this obstacle is to employ evaluators, known as double experts, who possess both usability and domain knowledge.53 It is essential to have a combination of these experts while evaluating computer-mediated patient education programmes in the health care environment, or else there is a risk of producing a mismatch between the system and the real world.
Conclusion
The results of the existing heuristic evaluation demonstrated that the system was more consistent with Nielsen’s usability heuristics. The ability to identify problems in a timely manner makes this method particularly well suited to the iterative design process. It is important to be considerate about the possible users of the system when conducting heuristic evaluations so that the system can speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms and information, and these words, phrases and concepts should appear in a natural and logical order. This will help us facilitate the identification and prioritisation of problems needing urgent attention at an earlier stage before the final deployment of the system.
Acknowledgments
This study was funded by the Agency for Health care Research Quality # HS021321-01A1.