The WHAAM application: a tool to support the evidence-based practice in the functional behaviour assessment

Theoretical background

The WHAAM application is based on the concepts and methods of functional behavioural assessment (FBA). The FBA is strongly linked to the above-mentioned ABA, bearing in mind that both are founded on the principle of Skinner’s operant conditioning.5^,6 Unlike the first stimulus-response (S-R) proposed by Watson7 at the beginning of behavioural analysis, the operant conditioning studies the observed human behaviour and considers the possibilities between antecedent, the behaviour itself, and its consequence. The old S-R model was therefore replaced by a stimulus, response, stimulus (S-R-S) model. According to this new perspective, the reasons that a behaviour is maintained can not only be derived from its external form or topography but also by discovering the conditions which trigger and maintain it. Carr,8 in a renowned scientific paper about self-injurious behaviour, proposed three general classes of possible functions sustained by behaviour: positive reinforcement, negative reinforcement and sensory or automatic consequences of the behaviour. In the last few decades, these functions have been deeply investigated and some researchers thought to consider new additional functions, such as tangible,9 control10 or access to stereotypy.11 FBA is defined by Gresham et al.12 as a collection of methods for gathering information about antecedents, behaviours and consequences in order to determine the reason (function) of the behaviour. It differs from experimental functional analysis because it is easily applied in natural settings (home, school, etc.), it needs less training to use it and it does not require direct manipulation of dependent variables to identify the main function sustained by a problem behaviour. The FBA is therefore a descriptive assessment involving indirect and direct observations and the measurement of the target behaviour.13 Questionnaires, rating scales and interviews are examples of indirect methods because they do not require a direct observation of the target behaviour and third party people, such as relatives or teachers, can administer them. The direct observation of behaviour is instead carried out in a natural setting by descriptive assessments and/or systematic recording. While descriptive methods like antecedent behaviour consequence (ABC) charts and narrative recording provide qualitative information about a targeted behaviour, identifying the variables that may trigger or maintain it,14 systematic direct observation of behaviour provides quantitative information about frequency, intensity or duration of a targeted behaviour during a defined time interval. The WHAAM application15^,16 is intended to promote the application of FBA methods to reduce the occurrence of problem behaviours in relation to attention deficit hyperactivity disorder (ADHD) or to replace them with positive ones. According to the Diagnostic and Statistical Manual of Mental Disorders-5,17 ADHD is a persistent pattern of inattention and/or hyperactivity–impulsivity that interferes with functioning or development, has symptoms presenting in two or more settings, and negatively impacts directly on social, academic or occupational functioning. For instance, children with ADHD are often rejected by peers and few are popular with their peers18^,19 and they are at greater risk of substance abuse, school dropout, delinquency, academic problems and other psychological disorders.20 In the FBA context, the WHAAM application is a successful example of how the most recent technologies can support FBA by adopting an EBP. In the following section, the system’s functions will be described, a case study will be presented highlighting the statistical relevance of the behavioural interventions carried out and the satisfaction of the users of the technology will be evaluated.

Overview of the WHAAM application features

Starting from the theoretical background of the FBA, the WHAAM application21 features are inspired by the multimodal treatment of ADHD. The most common approaches to manage ADHD symptoms includes behavioural interventions, the involvement of people included in the major settings of life in psycho-educational activities and the analysis of the medical history of the patient. In particular, the WHAAM application provides users with features for creating a network of stakeholders (parents, teachers and caregivers) close to the child with ADHD. This is to promote their collaboration during the whole functional behaviour process using both the web and the mobile sides of the application. The website allows users to sign in using their personal identification to access the following features:

• inclusion of data related to a patient (i.e. diagnosis, medical therapy, information about schools attended and crucial life events in his personal history);

• invitation of caregivers to become members of the child’s private network;

• use of instant messaging to promote collaboration and sharing data and information between the network’s member;

• identification and description of a child’s specific behavioural problem (target behaviour) which the network wants to change;

• planning the collection of baseline data;

• creating a hypothesis about the main function sustaining the target behaviour;

• creating an intervention plan, describing in detail the strategies to be applied and who will apply them within the network;

• the evaluation of the effect size of the intervention, applying statistical algorithms.

The mobile application is available for Android operating systems. It provides the following features:

• viewing the case data;

• viewing the target behaviour descriptions;

• access to the observation sessions assigned to the logged user;

• ongoing observation of the targeted behaviour through the recording of the behaviour frequencies or duration;

• collection of qualitative data about the target behaviour through ABC charts.

The WHAAM application supports the FBA process in six steps. The first step is the creation of a child’s record. In order to let the users choose what they want to disclose, the personal information of the child can be omitted, using only a nickname, gender and the date of birth as the required information. Additional data such as diagnoses, medications, school information and other general events characterizing the child’s life can be added in the section of ‘case data’. This section provides health professionals with information intended to support the understanding of the child target behaviours. According to the Italian Personal Data Protection Code,22 data encryption of personal records and a credentials’ authentication system guarantee the personal data protection from intrusive accesses. During the second step, people who have an important role in the life contexts of the child with ADHD can be invited to become a member of the child’s private networks. Thus, they will become active participants in all the phases of the FBA processes, putting into practice the tasks assigned to them, sharing knowledge and information. The third step is creating the accurate description of a target behaviour in order to start the functional assessment process. To help with the clarification of a target behaviour in a recognised, measurable way, the WHAAM application provides users with a set of predefined common ADHD behaviour definitions. It is however possible and helpful to write a specific behavioural definition in an operational, objective and concise way. It needs to be objective and concise because the reader has to have a very clear picture of the specific behaviour they are to observe. The description includes the place and the setting in which the target behaviour occurs. The fourth step is defining the settings for the data collection phases. The WHAAM application supports an AB single case research design, allowing users to compare baseline and intervention data. AB single-case design generally starts with a baseline (phase A) in order to observe the dependent variable as it appears. Once the baseline is established, the observation continues while implementing the intervention (phase B) in order to compare the two time series and testing the hypothesised change. The fifth step is collecting quantitative and qualitative data using the WHAAM mobile app. The user, who plays the role of observer, gathers the frequency or the duration of a targeted behaviour according to the agreed data collection settings. Every member of the child’s network can contribute qualitative data of a behaviour by filling in ABC charts. Data collected during this step is available to the child’s network directly through the WHAAM web application. Health professionals analyse all the gathered observations to formulate a hypothesis about the main function that is sustaining the target behaviour and then to create an intervention plan intended to reduce or eliminate the inappropriate behaviour. The intervention plan consists of a set of behavioural strategies defined by health professionals and assigned to specific members of the child’s network. During the intervention phase, new data are collected for further comparisons. The final step is in the evaluation of intervention. Health professionals can produce a visual analysis of the gathered data accessing the scatter plots representing the observation sessions. Moreover, the Parker et al.23 TAU-U test is automatically calculated by the application to show the effect of the treatment on the occurrences of the target behaviour in the AB designs. TAU-U is a non-parametric statistic, which estimates the treatment effect considering both the non-overlap between phases and the trend change in phase B. This effect is estimated controlling the natural trend of phase A. The use of Kendall TAU permits us to estimate the phases monotonicity without assuming any trend function (e.g. linear). TAU-U showed to be a robust statistic with autocorrelated data overcoming some limitations of other nonoverlap techniques. The formula r = sin (0.5 πτ) is used to convert TAU-U in Pearson’s r effect size index.24 Following Cohen,25 absolute value of r between 0 and 0.09, 0.10 and 0.29, 0.30 and 0.49 and greater or equal to 0.50 is evidence of a null, small, medium and large effect size, respectively.

The WHAAM application has been extensively tested in Italy, Portugal and United Kingdom during its development. The next section will describe the methodological constraints surrounding the use of the WHAAM application and two illustrative case studies supported by it.

Introducing the case: methodological considerations

The development of the WHAAM application adopted a user-centred design process including the final users from the early stages of developing and prototyping to the real context use of the application in a pilot test across Europe. As mentioned in the previous section, the WHAAM application involves many different user strategies in the management of unwanted behaviours of children with ADHD. In the process of management, each role has different duties and responsibilities and takes advantage of a subset of the features provided by the application. In order to get the best possible results in the FBA process using the new technology, users need to be aware of both the theoretical and methodological assumptions behind the FBA and the opportunities offered by the technology. For these reasons, the authors included a psycho educational content in the application and provides an integrated training course for teachers, health professionals and parents of children with ADHD. As underlined by Sanches-Ferreira et al.26 ‘the integrated model of training aims to promote community-based interventions in which every element of the school-family system has a key role in the identification and management of pupils’ problematic behaviours’ (p. 22). The training courses focused on three main learning outcomes: the comprehension of the FBA principles, the understanding of a set of specific behavioural techniques and the use of the WHAAM application to support the behavioural assessment and intervention on children’s undesired behaviours. The cases of Matteo and Anthony, which are described in detail in the next paragraphs, were examined during training courses held in Italy and in Portugal. It is worth noting that the names of children and some identifying details have been changed to protect the privacy of individuals. The authors measured the trainees’ thoughts on the usefulness of training, the usability and the overall satisfaction associated to the use of the WHAAM application during the training courses. In general, the course was considered substantial, while the application was seen as useful and easy to be use.27

Matteo’s case

Matteo is 17 and lives with his family, has one brother aged 22 and one sister aged 23. He attends an Italian secondary school. His mother is employed in a financial loans agency and his father is an employee of the Ministry of the Interior. The family is characterised by high internal conflict and dynamics of manipulation by the father. Matteo received the first diagnosis of ADHD when he was 11 years old and he is currently being treated with methylphenidate. By the time he was diagnosed, he already had a history of serious scholastic difficulties due to being oppositional toward teachers. He often misses instructions given by them and engages in significant off-task behaviour for most subjects, except math and physical education – in which he excels. Parents are very worried because Matteo is struggling daily with his schoolwork: he is disorganised when planning activities and he usually goes to school unprepared and for this reason collects many negative marks. He appears to be having problems making and keeping friends, especially in the school context. This is because his classmates consider him a preferred student and they cannot stand the fact that the professors never question him as they have an understanding of his condition. Matteo will graduate next year and has never received special education help or an individualised education program. Matteo has never been allowed to fail an assessment, according to his parents who think that teachers prefer not to deal with Matteo’s negative behaviours. The behaviour analysed with the support of the WHAAM application is his difficulty to get out of bed in time to go to school. His mother tries to wake up him initially at 06.50 am. Despite her many subsequent attempts, Matteo ignores her and continues to stay in bed. As a consequence of this, he is often so late that he skips the first hour of classes at school or the school day. In order to identify the function of Matteo’s behaviour, the health professional planned a ‘baseline phase’ in which quantitative data were gathered through five, 40-minute observation sessions taken by the mother using a duration recording system. In the baseline, the mean time needed by Matteo to get out of bed was 25.8 minutes [standard deviation (SD) = 13.62] with a range from 7 to 40 and a strong increasing trend (r = 0.95). After a clinical interview with the parents (indirect assessment) and from the analysis of collected baseline data, the health professional hypothesised that the primary function of the Matteo’s challenging behaviour was to escape school; second, it was gaining his mother’s attention. Starting from this assessment data, the intervention was aimed to help Matteo in overcoming his difficulty to get-out-of-bed. The intervention was outlined with parents using the following strategies:

• Negative reinforcement: an alarm clock was placed away from Matteo in order to train him to get out of bed. The clock rang every 5 minutes after the planned wake up time. In this procedure, the individual is motivated to exhibit a desired behaviour to avoid the aversive stimuli (the shrill sound of the alarm clock).

• Extinction: being that Matteo’s behaviour is maintained by positive reinforcement of his mother who cuddled him whenever he got up, the therapist invited the mother to eliminate the connection between the problem behaviour (get-out-of-bed) and the positive reinforcement (mother’s attention).

• Plan a positive reinforcement program by means of a token economy procedure: a paperboard football was given to Matteo every time he got out of bed by himself. Matteo’s participation in a football tournament was linked to the achievement of gaining 10 paperboard Footballs.

Once the intervention was defined, a new phase of systematic observation started with the same modalities and observer of the baseline. Five new observation sessions were carried out. During the intervention, the mean time employed by Matteo to get out of bed was 12.8 minutes (SD = 16.16), with a range from 0 to 40 and a strong decreasing trend (r = −0.81). Based on the data obtained during both the baseline and the intervention (Figure 1), the treatment had a strong effect on the decrease of the duration of the behaviour (r = −0.79, TAU-U = −0.58, p < 0.05).

Request permission

Matteo’s noncompliance and lack of independence in getting ready in the morning and going to school was tackled actively involving his own parents. According to the intervention approach suggested by Kearney et al.28 the key components of the parent training were (a) restructuring parent commands to clearer, simpler and more direct ways of giving instructions, (b) establishing a set of routines for the early part of the day and throughout the day, (c) negotiate rewards for compliance and school attendance and punishments for noncompliance and school refusal behaviour and (d) forced the school attendance under certain condition.

Anthony’s case

Anthony is a 9-year-old boy and the only child of a two-parent family. He is enrolled in the third grade of a local elementary school, in which he attends a special education program. Since the first grade, he has been identified as performing below expected academic achievements. Along with the diagnosis of ADHD, at the age of 7, he was referred to special education services in the second grade. The restrictions of following the educational curriculum were mainly identified as difficulties of concentrating his attention, understanding instructions and controlling his psychomotor activity. The inattentiveness and hyperactivity behaviours were also seen at home, not finishing the home chores and engaging in dangerous activities – engaging in excessively rough play. In the last assessment, it also showed him as having dyspraxia, having difficulties in planning complex sequence of movements. Currently, he is medicated with methylphenidate and improvements were reported in his concentration, his attention and remaining in his seat during classes. Nevertheless, teachers and parents still had great concerns with his lack of social relationships. Peers start to reject him for not following the rules of play – not waiting his turn – and seeking excessive physical contact (pushing and hugging tightly the others). At home, the interaction with his parents was also challenging, always seeking their attention by interrupting their conversations or tasks. Parents react differently to his problematic behaviour – the father strongly reprimands him and the mother makes light of his behaviour and gives him lots of attention – this is a source of constant conflict in the house. The problematic behaviour to be monitored by the parents using the WHAAM application was Anthony interrupting their conversation and making irrelevant and repetitive comments during a mealtime. When confronted with Anthony’s excessive and repetitive talk, parents ask him to stop and sometimes they leave the table. That behaviour tends to be even more noticeable when they have guests for dinner. As a consequence of this, there is often a lack of dialogue between parents during the mealtime (directing their full attention at Anthony) or a discussion between them on each other reactions to Anthony’s behaviour. Together with a health professional, it was decided to collect a baseline of data, where the father registered the frequency of Anthony’s interruptions of their conversation or when he engages in repetitive and excessive talk at the dinner table. Observation sessions of 25 minutes were collected. Additionally, in order to gain a complete pattern of the problematic behaviour a brief description of the antecedent and consequent events was collected for the application. The average of the occurrence of the target behaviour during the four baseline sessions was of 10.5 (SD = 3.10), with a range from 2 to 15 and a substantial increasing trend (r = 0.50, Figure 2). From the analysis of the frequencies graph of and the ABC descriptions, it was hypothesised that the main function of Anthony’s behaviour was to seek social attention. Using this interpretation, the intervention was outlined with parents using the following strategies:

• Establish rules: both before and during the dinner. Anthony was reminded that he should ask permission to talk. The use of a ‘talking object’ (an object to passes between each family member and is used as a way of identifying who has permission to talk at that moment).

• Reinforce appropriate behaviour: a period of time was identified to play family games such as board games and/or storytelling, every time Anthony followed the rules.

• Ignore inappropriate behaviour: when Anthony interrupts anybody that holds the talking object and/or makes repetitive comments, the parents were instructed to keep the conversation going without looking to Anthony.

Request permission

Starting with the implementation of these parent strategies, they collected data relative to the intervention phase. Four new observations were executed. At this period, the average of the occurrence of the target behaviour was of 3.25 (SD = 2.06), with a range from 1 to 5 and a strong decreasing trend (r = −0.97). As shown in Figure 2, there was a strong effect of the intervention successfully decreasing the frequency of the behaviour (r = −0.96, TAU-U = −0.82, p < 0.01).