Overall rationale

We performed an effectiveness review following conventional systematic review methodology [i.e. Centre for Reviews and Dissemination (CRD) guidelines] utilising systematic reviews and randomised controlled trials (RCTs).²¹ As the scoping review revealed an overall pattern that reminder interventions are effective in many circumstances, albeit to varying degrees, we complemented this with a review informed by realist principles supplementing effectiveness information with non-randomised and qualitative research and engaging with specific and general theory. This review was informed by a conceptual review of relevant theories; thus, the overall project comprised three components:

1. conceptual review of relevant theories (review 1)
2. systematic review of effectiveness studies (systematic reviews and RCTs) (review 2)
3. review informed by realist principles (review 3).

Figure 1 outlines the chronological process by which the three reviews were conducted, their relationship to each other and their overall contribution to the project.

FIGURE 1

Flow chart summarising the review process.

Literature searches (reviews 2 and 3)

Sifting/categorisation of search results (reviews 2 and 3)

Production of draft conceptual framework and draft propositions (review 1)

The multidimensional nature of health problems, health service settings and their interactions with individual service users implies that the mechanisms supporting or undermining patient attendance at outpatient appointments following receipt of a reminder are likely to be varied, complex and context dependent. Therefore, we sought to develop a comprehensive conceptual framework that could guide our review of the available literature systematically in order to explore the wide range of contextual and mechanistic factors that may influence reminder effectiveness. To construct this framework, we initially conducted a rapid review of our available literature base to identify any prior conceptual models or frameworks that had been employed to explain why and how reminder systems do, or do not, work. It was established early in this process that the majority of the reminder-focused literature was theory light, with a noticeable lack of attention within the identified RCTs to process evaluation that could explain why, how and in what circumstances reminders are effective and there were few and limited theories being advanced by authors to explain reminder system functioning. Indeed, it was apparent that much of the literature was framed very narrowly around the notion that reminders simply remind forgetful patients, with little consideration of the broader range of factors that could be at play.

We identified only one conceptual model specifically about appointment reminder systems. This model, developed by Coomes et al.,²² proposes that communication functionality of SMS reminders (e.g. single or multicomponent, interactivity, frequency of reminder, timing of reminder and tailoring of the message) and patient psychosocial factors (e.g. patient involvement, social support, medication adherence, risk behaviours, and health and well-being) could mediate the impact of SMS reminders on health-care quality and health outcomes for people living with a human immunodeficiency virus (HIV) infection.²² For the purposes of the current project, this model was limited in two regards: first, it dealt only with SMS reminders rather than the spectrum of available reminders; and, second, it dealt with mechanisms leading to health-care outcomes rather than considering the mechanisms leading to appointment attendance. However, it did identify features of SMS functionality that were potentially useful to consider within our own framework, which aimed to be more comprehensive.

Given the limited theoretical understanding available from the prior literature on reminder systems, it was decided to conduct a focused review of the conceptually close topic area of medical compliance and adherence. It was known to the team that at least some of the research in this area had been theoretically driven and it was decided that this literature could therefore yield conceptual insight of relevance to the topic of the current review. From this literature, we identified a number of conceptual models specifying factors that can explain medical adherence (including appointment attendance behaviour). These were:

• theory of planned behaviour,²³ which had been adapted to explain attendance at screening appointments,²⁴ adherence with psychological therapies²⁵ and attendance at a cardiac rehabilitation programme²⁶
• transtheoretical model (also known as ‘stages of change’),²⁷ which had been adapted to explain adherence with psychological therapies²⁵ and attendance at dental check-ups²⁸
• self-determination theory,²⁹ which had been used to explain adherence with psychological therapies²⁵ and dental clinic attendance³⁰
• health-care utilisation model,³¹ which has been adapted for use to explain medication adherence.³²

A further three models which appeared to have some relevance to our topic of enquiry were also identified during this stage of our work, although they had not been used to guide empirical work on adherence/attendance specifically. These were:

• protection motivation theory,³³ which contends that how a person behaves will be broadly explained by a person appraisal of the threat and an appraisal of how to cope with that threat and can, therefore, potentially be related to health behaviours such as making and attending appointments
• rational choice theory, a development of William Glasser’s choice theory³⁴ that has been discussed in relation to the role of message framing to motivate healthy behaviours³⁵
• complexity theory,³⁶ which provides a scientific framework for understanding complex phenomena in the natural, biological and human sciences.

Although none of the above models specifically dealt with our topic, they provided a useful starting point for mapping out the contextual factors, mechanisms and outcomes that were important to consider in any attempt to explain how reminder systems work, for whom and in which circumstances. In addition to these behavioural models, which were predominantly informed by psychological theory, we also engaged with relevant sociological literature. This approach was in keeping with our desire to develop a comprehensive conceptual framework that mapped the range of influences operating at different levels, thereby seeking to understand the functioning of reminder systems within a broad psychosocial-systems perspective, rather than locating the phenomenon entirely at the individual level. Although our review did not identify any sociologically driven research papers that dealt specifically with responses to appointment reminders, a number of contributions were felt to be useful in relation to expanding our conceptual framework to encompass a broader range of potentially important factors. First, we noted earlier work on the uptake of screening interventions that highlighted the way in which attendance can be understood as a response to normative expectations about what constitutes responsible and legitimate action – a form of moral obligation – rather than an individual choice or decision (e.g. Griffiths et al.³⁷ on breast screening).³⁸ This strand of work can be seen to be influenced by Pierre Bourdieu’s concept of habitus:³⁹ the embodiment of social rules, values and dispositions. Individuals, he argues, acquire a ‘sense of one’s place’ within hierarchically structured society and include/exclude themselves from goods, persons and places that are inside/outside their social group.³⁹ Understanding differential patterns of appointment behaviour between ‘groups’ may be enhanced by considering the way in which such behaviours may be supported or undermined by other taken-for-granted ‘ways of being’ or ‘sticky habits’ that exist within particular groups or communities.³⁹ Second, we identified work that seeks to highlight the roles of concrete, situated contingencies that shape and constrain behaviours, such as the practical demands of daily living, including health-related adherence/attendance. For instance, Rosenfield and Weinberg³⁸ alert us to the need to counter the tendency for adherence research to focus on thought processes, knowledge and beliefs, thereby overlooking the importance of ‘the contours and rhythms of situated domestic practices’.³⁷ Understanding more about the day-to-day lives of patients can potentially throw light on how people respond to appointment reminders. Finally, we saw value in critical sociological contributions that take a more macro focus seeking to explore and expose the linkages between the health-care system and the broader political, economical and social systems of society in historical perspective. Critical theory has enriched various bodies of work exploring patient experiences of health care, revealing the ways in which medical ideology helps to maintain and reproduce class structure (as well as other social divisions, e.g. gender, race/ethnicity). These perspectives tend to challenge the way in which health-related behaviours and their promotion are frequently presented as unproblematic by health professionals and health researchers, while downplaying the complex costs and benefits that may be involved for the patient, particularly patients from more marginalised groups. These sociological contributions complemented the more individualistic focus of the psychological models and allowed us to usefully expand our conceptual framework.

At an operational level, team members drew on this wide-ranging theoretical literature together with evidence gathered during early data extraction and synthesis to develop a series of internal concept notes that were then discussed in team meetings. The multidisciplinary make-up of the project team meant that different members were well placed to engage with different literatures and to produce summaries for the rest of the team to consider. Deliberative discussions were combined with visual tools to iteratively consider alternative ways of representing reminder system functioning, considering:

• Process maps of attendance charting patient decisions, appointment attendance/cancellation/rescheduling behaviour and consequent outcomes, looking at the day of initially receiving the appointment, the day of receiving a reminder closer to the appointment (if applicable) and, finally, the day of the appointment itself.
• Balance-sheets focusing on patient decision(s) to attend or not, in terms of a weighing-up of the advantages of attending compared with the advantages of not attending. It was felt that, overall, the balance would tend to be weighed heavily in favour of not attending (as this is the easier option for the patient in the short term), but that various factors, including reminder interventions, may help tip the balance the other way.
• A ‘tumblers in a safe’ model whereby various factors need to align if a patient is to attend the appointment. Such factors include individual patient characteristics, the wider social system (including norms, expectations and enablers such as transport) and the health service system (including flexibility of the appointment system and the effectiveness of the reminder system). Similarly, for an appointment reminder system to be effective, various elements must ‘line up’, namely the reminder should be received in time and its content should be accurate, the patient should be receptive and genuinely not forget and the service should be sufficiently flexible to enable cancellation/rebooking.

This process enabled the development of our conceptual framework, which is depicted in its final iteration in Figure 2. In this framework, propositions A–F identify various important contextual elements that are hypothesised to contribute to attendance outcomes. The individual (proposition A) is placed at the centre to convey the centrality of the patient role in deciding whether or not he or she will attend (or at least intends to attend). Each patient can be perceived as mentally constructing a balance sheet by which they weigh up perceived obstacles to attendance against enablers. The centrality of the patient interaction with the reminder system further indicates its central importance within this review. There are different factors that will influence the ‘baseline’ of where the patient is ‘at’ with respect to attendance obstacles or enablers when he or she receives an appointment notification. These include various social factors (proposition D) and health-care setting factors (proposition C), with the appointment system located at the heart of the latter. The model shows some possible ‘distal/proxy attributes’ (proposition F) – attributes that could be used to characterise a patient (or patient group) that may tend to give that patient or group a certain ‘baseline’ or which may predict the effectiveness of different types of reminder system. The reminder system (proposition B), depending on its characteristics and depending on where the patient weights the balance of obstacles/enablers before receiving the reminder, may tip the balance in favour of enablers so that the patient intends to attend. Other factors outside the reminder system per se, but within the control of the health-care system, may be modifiable in order to tip the balance in favour of intention to attend. Finally, the model recognises that there may be individual- and system-level obstacles and enablers to cancellation and rebooking (proposition E) that warrant consideration. An accompanying articulation document expands/explains the elements of the conceptual framework and highlights possible causal pathways between them (see Appendix 2). These propositions form the basis of our evidence statements and discussions in Chapters 4 and 5.

FIGURE 2

Targeting the Use of Reminders and Notifications for Uptake by Populations (TURNUP) conceptual framework.

Preliminary data extraction: randomised controlled trials (review 2)

Full-text screening took place in conjunction with data extraction. The team prioritised RCT studies of appointment reminder interventions in order to draw out effectiveness data to address the objective concerning which types of reminder systems are most effective in improving the uptake of health service appointments. These data fed into review 2, to identify whether or not any studies conducted subgroup analyses, and to start to identify (if possible) any trends in terms of ‘what works’ and in ‘which context’. We mapped the studies to the draft propositions derived from the conceptual framework (Table 1 and see Figure 2). A data extraction tool was designed, piloted, refined and ultimately used in Google Forms (Googleplex, Mountain View, CA, USA) to extract data from included RCTs. Google Forms allowed the five reviewers to use the extraction tool concurrently and to view interim patterns from the data as soon as each study was submitted. Preliminary analysis of these data was used formatively by the review team to identify contextual patterns that might influence the effectiveness of reminder systems and, therefore, to support the development of the preliminary conceptual framework [see Production of draft conceptual framework and draft propositions (review 1)].

TABLE 1

Draft propositions from a preliminary conceptual framework identifying potentially important influences on patient attendance

Testing the draft propositions: further data extraction (reviews 2 and 3)

Testing of the draft propositions (see Table 1) and populating them with evidence for the review informed by realist principles required further exploration of the included studies. Previously extracted data from the included RCTs about reminder systems were augmented with fields for data to be extracted directly against the contextual factors and potential explanatory variables identified in the preliminary conceptual framework (propositions A–F in Table 1). Other fields included gender, age, other population categories, description of study population, country, setting, stated underlying purpose of the study and, for each reminder intervention described, description of reminder intervention(s), reminder timing, reminder medium, reminder interactivity, reminder content, reminder source/bearer, reminder intensity, comparison/control, reported outcomes, economic analysis and disadvantages of introducing reminder systems. This process was also extended to papers reporting other study designs (both qualitative and quantitative), again utilising the same fields, to allow mapping against the research questions addressed by each study (see Review questions). The propositions are expanded in the results section (from Proposition A: the reminder–patient interaction onwards), with evidence statements demonstrating the strength of the findings for each proposition.

The extraction forms also provided a facility for memoing by each reviewer, including identification of any relevant references for follow-up and a field for noting any quality issues. See Quality appraisal (reviews 2 and 3) for a discussion about this.

From the outset, the team recognised that they would not be extracting from all eligible papers; therefore, the papers were prioritised for extraction, based on study type and likely relevance to a UK NHS context. All RCTs about reminder systems were prioritised for full extraction, as they could be seen to contribute to both the systematic review (review 2) and the review informed by realist principles (review 3). We also prioritised all reviews (systematic and otherwise) about reminder systems and appointment systems. We then turned to qualitative, mixed-methods and non-RCT quantitative studies about reminders and appointments for the UK, Ireland, Europe, Australia and New Zealand. In accordance with realist principles, we did not extract data from all possible studies. Having extracted from a sizeable number of the above categories of paper, we found that we were not routinely uncovering new ideas and concepts. Thus, during synthesis we focused on identifying potentially ‘rich’ sources of data from the remaining studies. The methodological justifications for this approach, predicated on considerations of data saturation, were subsequently endorsed by our project steering group (see Involvement of patient steering group).

Data extraction was carried out independently by six reviewers (AB, MG, SB, SMc, SN and SS). Although the team did not undertake double data extraction, individual reviewers were able to refer back to the original articles when interpreting their assigned section, thus corroborating the initial extraction process.

Synthesis (reviews 2 and 3)

Thematic synthesis

The extracted data from the RCTs and other study designs were exported from Google Forms into a single Microsoft Excel spreadsheet that was shared between four of the reviewers (AB, MG, SMc and SS). Each reviewer examined different propositions from the conceptual framework, in order to perform a thematic analysis of the evidence in respect of the different propositions. As described above [see Preliminary data extraction: RCTs (review 2)], the data extraction forms included fields for extracting directly to the different framework factors. A complete list of the different framework factors can be found in Appendix 3. One example of the different framework factors is item B.4 which refers to ‘the format of the reminder may or may not compromise the delivery of the content to the patient’; however, in most instances, it was necessary for the reviewer to ‘read across’ the extraction row in order to contextualise the excerpt or observation. The reviewers formulated evidence statements for each of the six areas of the model using the summary categories for bodies of evidence laid out in Table 2.

TABLE 2

Summary categories for bodies of evidence

Quality appraisal (reviews 2 and 3)

The data extraction forms included a field for the reviewer to memo any methodological quality issues noted, flagged up either by the author of the paper being reviewed or by the reviewer. Quality appraisal was not used as a mechanism for study inclusion or exclusion, as realist review principles (review 3) recognise that it is perfectly acceptable to use informative ‘nuggets’ from a methodologically flawed paper.⁴⁰ For the RCTs included in the systematic review (review 2), full formal quality appraisal was carried out using the Critical Appraisal Skills Programme (CASP) appraisal tool for those RCTs that had not already been quality assessed by a previous systematic review.⁴¹

Review complexity

The sequential process outlined in this chapter belies the complexity of the project. Two specific areas of complexity were identified: complexity at a macro level, evident in the inter-relationship and interdependency of the three reviews with their methodologies; and complexity at a micro level, relating to the numerous factors that interact to impact upon appointment behaviour, a complexity acknowledged in the choice of a methodology informed by realist principles.

This complexity was explored, and largely resolved, through team discussions and by drawing on advice from the project steering group. The steering group drew on its experience of realist synthesis methodology, and of systematic reviews more generally, to help refine details of the review procedures. Group members facilitated fruitful discussion on the complementary strengths of the systematic review and the review informed by realist principles. They advised on strategies for managing and stratifying large volumes of qualitative literature and deciding when to stop pooling literature. The steering group confirmed the team’s perception that theoretical saturation was likely to occur with fewer new ideas and concepts being likely to emerge as extractions progressed. This served to inform the subsequent prioritisation and process of the remaining identified articles. The review team were, therefore, able to proceed with their more selective approaches utilising theoretical sampling, as mandated by realist synthesis methodologies. As a consequence, the team decided to scan the remaining papers for conceptual richness and to prioritise papers with particular richness or that offered unique perspectives on an issue, while proceeding with the thematic synthesis described above.

Furthermore, the steering group acted as a sounding board for theories identified from review 1, which sought to explain why patients do not attend appointments and the mechanisms by which reminder systems work to improve appointment attendance. Such discussions informed the subsequent development of the candidate propositions.

Development of materials for health service users

The findings of reviews 1, 2 and 3 were finally synthesised to support the development of materials for use by health service managers to support decision-making regarding the management of non-attendance at outpatient departments. The materials have been developed in three parts:

1. an outline of all known reasons for patient non-attendance and incorporating potential reminder and wider solutions for managing those reasons for not attending
2. clinical scenarios that focus on the managing attendance, cancellations and rescheduling
3. advantages and disadvantages of different reminder systems.

Involvement of patient steering group

In February 2013, two members of the project team (SMc and MC) met with a group of six patient representatives at Sheffield Teaching Hospital. These representatives were invited from a group of public governors from Sheffield Teaching Hospitals. They were presented with a draft of the clinical scenarios for managing attendance, cancellations and rescheduling, and were asked to review this from a patient perspective (see Chapter 6, Other emergent interpretations for a fuller explanation). Each scenario was discussed openly and frankly, with the patient representatives providing comments about the draft document and additional information around each scenario and suggesting additional clinical scenarios for inclusion in the document. These comments were utilised to refine our final clinical scenarios document.