Why focus on the user experience of psychological therapy?

User experience reflects the extent to which an intervention is perceived by a person as useful in meeting their needs and is enjoyable and easy to use.²¹ Ease of use, or usability, has been defined as ‘a quality attribute that assesses how easy interfaces are to use’.⁸⁶ It relates to the ease with which a person can become competent in using a product or service, achieve their objectives for use and recall how to use the product or service during future interactions. User experience, therefore, determines how likely people are to engage with a design and continue to use it. This has been relatively neglected in psychological therapy, for which the focus has instead been on efficacy by developing interventions that target evidence-based mechanisms to improve mental health outcomes.⁸⁷ These interventionist-causal approaches have shown promise over traditional psychological therapies for psychosis.⁸⁸

However, there are significant barriers to the effective implementation of these targeted therapies for psychosis and efficacious interventions will be limited in their impact if stakeholders are not sufficiently willing and able to use them in routine care.¹ Obstacles include therapy being difficult to access owing to resource constraints, uptake being low even when therapy is offered, and people struggling to adhere to therapy and apply it to their problems in daily life.^5,89–91 Optimising the user experience of therapy provides a means of addressing implementation barriers and improving uptake and adherence. Psychological concepts and techniques can be ‘reframed’ by redesigning conventional means of supporting behaviour change.^92,93 Digital technology affords unique opportunities to address the user experience of therapy because the user interface (i.e. the digital artefacts through which therapy is delivered) can be modified and personalised to meet people’s needs.⁹⁴

Digital therapeutics for psychosis

Digital therapeutics for psychosis are in their infancy, with encouraging findings for mobile apps, virtual reality and web-based support.^95–102 The use of digital therapeutics requires access, willingness to engage with technology and sufficient competency or support. Promisingly, people with psychosis appear to have comparable access and use of technology to the general population.^103–106 However, people with psychosis have higher rates of digital exclusion if they are older, are from an ethnic minortiy background, experience cognitive difficulties or experience persisting symptoms, resulting in a ‘digital divide’.^104,107,108 Being female and of white ethnicity is associated with a higher rate of digital therapy completion.¹⁰⁹ Nonetheless, digitally excluded people with psychosis are willing to access technology.^107,110 Torous et al.¹¹¹ found that an interest in mental health apps does not translate to high use, as only 10% of outpatients had a mental health app downloaded on their telephone, and privacy and economic concerns were common. This is consistent with findings across digital health, for which the level of implementation of therapeutics in real-world settings is poor and rates of attrition are high, especially in the absence of interpersonal support.^112–114

Poor user experience has been highlighted as a critical barrier to engagement with digital therapeutics, particularly for marginalised groups.^115,116 Digital designs are often ‘skeuomorphic’, replicating analogue versions of therapy artefacts and, therefore, failing to address barriers to use.¹¹⁷ For example, a commonly used tool for identifying and modifying distressing cognitions in CBT, a thought record, is often digitally reproduced with the same interface as paper versions: usually text prompts and response boxes presented as a form. A skeuomorphic digital thought record does not address obstacles to its use, for example being cognitively demanding and having an unappealing interface. Graham et al.⁹⁴ propose that human-centred design should underpin the development of digital therapeutics because improved user experience is expected to mediate better clinical outcomes. Human-centred design involves developing a rich understanding of the problem area and its context, from a range of stakeholder perspectives, to identify valued outcomes.^{92–94,118,119} Therefore, participatory design, or co-design, is inherent to this approach and entails direct user involvement.^120,121 However, participatory design in digital mental health has tended to neglect design-thinking methodology, which can constrain innovation so that new designs are variations of the status quo.^92,122 In addition, a risk inherent in participatory design is that the most willing, able and vocal users are more likely to be involved, neglecting the needs of the marginalised people, whom the design should address. To reduce health inequalities, attention needs to be paid to a diverse range of people with psychosis, particularly those who are from a minority ethnic background, have cognitive difficulties and who experience severe symptoms.^{109,123–127}

The development of SlowMo therapy: inclusive human-centred design

The SlowMo therapy is an exemplar of an inclusive human-centred design approach to developing digital therapeutics for psychosis.⁷ Prior to the SlowMo trial, a multidisciplinary team of people with lived experience, clinicians, researchers, industrial designers and software developers integrated the best practice principles of design thinking and participatory design to create the therapy. The Design Council’s¹²⁸ double-diamond method was used, which consisted of ethnographic investigation of the problem context (the discover phase), and using insights from this phase to reframe the problem and generate a design brief (the define phase). Solutions to the brief were generated and iteratively tested with users (the develop phase), with feedback determining the optimal design for development (the deliver phase). Our strategy for involving people in the design process, inclusive human-centred design, was different from conventional human-centred design. It involved purposive sampling of people from the extreme ends of the distributions of relevant variables (i.e. gender, age, ethnicity, cognitive abilities, use of technology and attitudes to therapy) to increase the likelihood that the design met the needs of the widest range of people.²⁰ The inclusive, human-centred design research identified the importance of therapy being usable, trustworthy, enjoyable, personalised and normalising, and of it offering flexible interpersonal support, in line with other recommendations for improving implementation of digital therapeutics for psychosis.^129–131

This iterative process and feedback led to the development of the SlowMo therapy, a blended digital therapy consisting of an intuitive web app to augment the experience of face-to-face individual therapy sessions, which is synchronised with a native mobile app for use in daily life. SlowMo therapy is presented as a journey that supports people to notice the large, fast-spinning and grey worry bubbles that fuel distress, and to make use of the slow-spinning and coloured bubbles to shrink fears and feel safer. The use of personalisation, ambient information and, particularly, visual rather than verbal metaphors aimed to provide a step change in therapy delivery by enhancing appeal and reducing cognitive demands.

The mobile app consisted of a redesigned CBT thought record for managing paranoia that attempted to overcome the aforementioned limitations of paper versions. This incorporated an attractive visual representation of thoughts and their attributes; simple interactions to support monitoring and modifying thoughts; easy access to previously identified helpful suggestions and thoughts; positive reinforcement for engaging in slowing down; and a flexible interface that afforded several ways of slowing down fast thoughts, depending on a person’s needs and preference (e.g. quick access to safer thoughts on the home screen or working through all stages of slowing down a thought over multiple screens). Concerns about privacy were addressed by developing a native app with opt-in data transfer. The mobile app also relied on user-initiated interaction and optional push notifications to accommodate those who might find notifications intrusive.^95,132

The SlowMo therapy design has now been tested in a large sample of people with psychosis in a multicentre RCT, as described in Chapter 1. Therefore, it provides an opportunity to validate the inclusive, human-centred design of our digital therapeutic for psychosis and to evaluate whether or not the design was successful in achieving its aims.

Evaluating user experience: validation of the SlowMo therapy design

Evaluating the user experience of digital therapeutics requires moving beyond the usual focus on efficacy and effectiveness outcomes in intervention research. User experience assessment can include subjective measures of usefulness, usability and satisfaction, as well as objective means, such as system analytics of passive or active interactions with technology.⁹⁴ However, there is little consensus regarding how best to define and measure user experience, and studies often have no theory or data to support the criteria employed.¹³³ A recent review of studies evaluating the usage of digital therapy found that more frequent and prolonged use was assumed to be desirable. This assumption risks conflating engagement with adherence and not recognising that disengagement may reflect e-attainment [i.e. technology-assisted achievement of goal(s)] of personal goals if skills acquisition has been sufficiently supported.^115,134 Therefore, adopting multiple metrices of engagement, reflecting the goals of both the technology and the individual, is recommended.

A further concern is whether or not digital therapeutic use is impeded by technical problems with the hardware or software, and whether or not adverse events that are related to technology occur.^135,136 This chapter will describe a multidimensional assessment of the SlowMo therapy user experience to evaluate whether or not its inclusive, human-centred design is likely to support implementation for a diverse range of people.^7,94,118 Excellent adherence rates for the SlowMo web app sessions and therapy fidelity have been reported in Chapter 1; therefore, the mobile app adherence will be the focus here. The therapy sample will be characterised in relation to their digital literacy, followed by presentation of the SlowMo mobile app adherence based on self-reported and system analytics, a survey evaluation of user experience, and rates of technical problems and technology-related adverse events. Chapter 4 will build on this with a co-produced qualitative study of the trial participants’ verbal accounts of their experience of SlowMo therapy.

Digital literacy

Digital literacy was investigated at the beginning of therapy for all participants who attended at least one session in relation to (1) self-reported ownership of smartphones or access to a computer, (2) frequency of use of smartphones (excluding telephone calls) and computers, and (3) confidence in using smartphones and computers. Frequency and confidence of use were assessed on scales from 0 to 100, with the anchors of ‘never’ and ‘all the time’, and ‘not at all’ and ‘totally’ for frequency and confidence, respectively. These digital literacy variables were selected because they were the most relevant to the user experience of SlowMo therapy, given that the therapy involved using a laptop computer (in sessions) and a smartphone (outside sessions). Given that reported health inequalities related to demographic factors, we planned to examine digital literacy in relation to gender, age and ethnicity.

Self-reported and system analytics of adherence to the SlowMo mobile app

Adherence to the SlowMo mobile app was assessed subjectively and objectively to validate whether or not the design had the intended effects on the user experience and subsequent usage. Participants were asked to report at the end of therapy how much they were using the mobile app and if they intended to use it in the future (rated from ‘0 – never’ to ‘100 – all the time’). Objective adherence was assessed according to analytic data for mobile app use. We operationalised adherence as at least one out-of-session interaction for a minimum of three of the therapy sessions. This was based on seven therapy sessions because session eight data were not valid; mobile app data syncing did not occur following the end of therapy (the mobile app was a native app and we did not have informed consent for ongoing data collection after therapy had ended). The adherence criteria were based on the assumption that engagement with the mobile app would be indicative of its usefulness, usability and appeal; however, sustained use throughout therapy was not necessary given that the aim was to support internalisation of the skill of slowing down in response to fast thinking.^15,115 Home screen views were selected as the target interaction given that slowing down with the mobile app is undertaken through viewing the home screen (to access safer thoughts) or subsequent screens that provide multiple routes to slowing down.

User experience survey for the SlowMo mobile app

User experience was assessed by a 12-item user experience survey (UES) (see Appendix 3) that was adapted from a 26-item self-reported measure employed by Ben-Zeev et al.,¹³⁷ in a study of a mobile app, FOCUS, that supports self-management of psychosis. The UES consisted of four items assessing usefulness, four items assessing usability and four items assessing enjoyment. Each item was rated on a scale from 0 to 100, with anchors of ‘totally disagree’ and ‘totally agree’. Ratings for each item were summed (with four items reverse scored) (range from 0 to 400 for each category) and a percentage score calculated. This exercise was undertaken at the end of therapy for participants who had completed all eight therapy sessions. We also examined the impact of service users’ characteristics on self-reported user experience on the survey.

Technical problems related to the SlowMo web app and mobile app

The therapists completed a survey at the end of each therapy session to document whether or not in the sessions there were any technical problems with internet connectivity, any technical problems with the web app software, any technical problems with data syncing between the web app and the mobile app and any other participant-reported technical problems. These were all recorded as ‘yes’ or ‘no’, with a brief description of the nature of the problem, if any.

Adverse events related to the SlowMo therapy hardware and software

As noted in Chapter 1, adverse events were actively monitored for the duration of the trial and were categorised by severity and relatedness to trial participation recorded. In addition, for any adverse events related to trial procedures in the therapy group, it was documented whether or not there was any evidence indicating that the event was related to the SlowMo software (i.e. the web app and mobile app) and hardware (i.e. the mobile phone provided to participants). Any events were rated from 1 to 5: 1, definitely related; 2, probably related; 3, possibly related; 4, unlikely to be related; and 5, not related. This information was then reviewed by the chairperson of the DMEC and the DMEC.

Statistical methods

Summary statistics were calculated for all variables for the entire SlowMo therapy group and split by site. To investigate the impact of participant characteristics on user experience, we performed independent group t-tests (gender and GPTS paranoia severity) or one-way analyses of variance (ethnicity and age) for the continuous dependent variables of digital literacy, self-reported app adherence and the UES, and chi-squared tests for smartphone ownership, computer access and system analytics app adherence (rated adherent/non-adherent). Independent group t-tests were also conducted to examine the association between system analytics adherence and pre-therapy smartphone literacy. Categories for the participant characteristics were gender (male and female), age (< 35, 35–49 and ≥ 50 years), ethnicity (white, black and other ethnicity – consisting of Asian people and people from other ethnic backgrounds) and paranoia severity (low and high, dichotomised by a median split of < 61 and ≥ 62 on the GPTS).

Digital literacy

Smartphone ownership and computer access in the SlowMo therapy group among those participants who attended at least one session, together with the frequency of use and confidence, are displayed in Table 13 by site and overall. This indicates that just over three-quarters of the sample owned a smartphone, which was consistent across all sites. For smartphone owners, the frequency of use was comparable in Sussex and Oxford and lower in London. A similar pattern was found for smartphone confidence. Computer access, frequency of use and confidence were the highest in Sussex, followed by Oxford, and then London. The impact of gender, age, ethnicity and paranoia severity on smartphone and computer ownership and on smartphone use and confidence is shown in Figures 5 and 6, respectively, with inferential statistics presented in Appendix 3, Table 22. There were significant age differences in smartphone literacy, with older people being less likely to report ownership and confidence in using a smartphone. Older people and women were also significantly less confident in using computers. Ethnicity had a significant impact on computer access and smartphone and computer confidence, with people from a black ethnic group reporting less access and less confidence than those from white and other ethnic groups. Paranoia severity did not have a significant relationship to digital literacy.

TABLE 13

Smartphone and computer access, use and confidence in the SlowMo therapy group (N = 168)

FIGURE 5

Smartphone ownership and computer access among people attending at least one therapy session (n = 168). (a) Smartphone ownership by gender, (b) computer access by gender, (c) smartphone ownership by age, (d) computer access by age, (e) (more...)

FIGURE 6

Smartphone and computer use and confidence among people attending at least one therapy session (n = 168). (a) Smartphone and computer use by gender, (b) smartphone and computer confidence by gender, (c) smartphone and computer use by age, (more...)

Self-reported and system analytics adherence to the SlowMo mobile app

Self-reported current and intended future use of the mobile app are reported in Table 14. This assessment was not offered to the first 45 therapy cases, and completion rates were 80% and 78% for current and intended future use, respectively, for the remaining cases. The data indicate that the rate of current use varied from never to all of the time, with participants, on average, reporting using the mobile app just under half of the time. The current reported use was highest in Oxford and lowest in Sussex. By contrast, all participants reported at least some intention to use the mobile app again in the future, and the average frequency of intended use was also higher than current use, at just over half of the time. Self-reported adherence was compared with participants’ characteristics of age, gender, ethnicity and paranoia severity, as shown in Appendix 3, Table 23. Female participants reported significantly higher current and future intended use of the mobile app than male participants. There were no significant differences in current and intended use for age, ethnicity or paranoia severity.

TABLE 14

Self-reported current and future usage of the SlowMo mobile app among people completing all eight sessions who were offered the assessment

The system analytics adherence for the mobile app had some data lost at the beginning of the trial owing to a bug in the code. Once rectified, analytics data were stored when the participant had the version of the mobile app with the analytics coded installed; for individuals in therapy when the analytics issue was resolved, the mobile app could be updated to this version at any stage of therapy (sessions 1–8). Participants were defined as having missing analytics when there were insufficient data points to determine mobile app adherence according to our a priori criterion of at least one home screen view for at least three sessions.

For participants in the therapy group, 65.4% met the mobile app adherence criterion. This increased to 71.4% for participants who attended at least one session (and were, therefore, provided with a mobile phone with the mobile app installed). Among those participants attending all eight sessions, the adherence rate was 80.7%, suggesting a high rate of adherence. One-fifth of participants (21.4%) used the mobile app outside every recorded session. System analytics adherence was compared with participants’ characteristics of age, gender, ethnicity and paranoia severity, as well as pre-therapy smartphone use and confidence, as shown in Appendix 3, Table 24. There were no significant differences in the analytics adherence to the mobile app according to age, gender, ethnicity or paranoia severity. However, adherence rates were higher among those who attended all eight sessions, reported using smartphones more frequently and were confident in smartphone use prior to therapy.

User experience scale for the SlowMo mobile app

The UES findings for each subscale and the total score are presented in Table 15. The UES was not offered to the participants who completed therapy at the beginning of the trial (n = 45). A further three participants were not eligible to complete the UES, as they declined any engagement with the SlowMo mobile app. For the remaining sample, the completion rate was 85%. UES ratings were comparable across all subscales, with the majority of people providing positive ratings for enjoyment, usability and usefulness. However, there was a large range of scores, suggesting that the mobile app was positively received by most but not all participants. Figure 7 shows the UES ratings in relation to gender, age, ethnicity and paranoia severity.

TABLE 15

User experience scale ratings in therapy completers (N = 82)

FIGURE 7

User experience survey subscales and total scores among therapy completers (n = 82). (a) Gender, (b) age, (c) ethnicity and (d) paranoia severity.

The UES ratings were compared with participant characteristics, as shown in Appendix 3, Table 25. There were significant differences depending on gender, with women reporting higher rates of enjoyment and usefulness; however, rates of usability were similar for male and female participants. The significant differences in smartphone confidence prior to therapy did not appear to affect the self-reported user experience, as there were no significant differences depending on age and ethnicity. There were also no differences in UES ratings in relation to paranoia severity.

Technical problems

The technical problems with the SlowMo therapy connectivity, data syncing and software are shown in Appendix 3, Table 26. This demonstrates that technical problems occurred, although these were for a minority of sessions only. The most common technical problems were internet connectivity and data syncing.

Adverse events related to the SlowMo software (web app and mobile app) and hardware (mobile phone)

None of the 54 adverse events reported over the course of the trial was assessed as being related to the SlowMo mobile app software. There was one non-serious adverse events that was judged as ‘definitely’ related to the mobile phone that was provided to a participant so that they could access the mobile app. This involved a concern raised by a carer that the participant was using the SlowMo mobile phone to access a dating site using the internet connection at their home, which they viewed as inappropriate and reported to the trial therapist.