Methods
This was a pragmatic multicentre randomised external (rehearsal) pilot trial to assess patient recruitment and willingness to be randomised, rehearse trial methods and processes, and provide outcome data to inform sample size calculations for a future definitive trial. All of these were considered important elements of the determination of feasibility.
Units recruiting to the trial
Recruitment to the pilot trial was initially limited to six specified units; these were a mix of specialist urogynaecology (Newcastle upon Tyne and Leicester) and female urology (Sheffield and Swansea) departments in university teaching hospitals providing secondary- and tertiary-level care, and general gynaecology units in district general hospitals providing secondary care services (Wansbeck Hospital, Northumberland and Queen Elizabeth Hospital, Gateshead).
In order to improve adherence with recruitment targets and to test the processes for possible future use, two Patient Identification Centre (PIC) sites (Sunderland Royal Hospital and South Tyneside District General Hospital) and one additional full recruiting site (South Tees Hospitals NHS Foundation Trust) were introduced during 2012.
Inclusion and exclusion criteria
Inclusion criteria
Inclusion criteria for the pilot trial (and currently anticipated inclusion criteria for the future definitive trial) were as follows:
Women were required to fulfil ALL criteria to be eligible:
Clinical diagnosis of SUI or stress predominant MUI.
Women must state that their family is complete.
Women should have undergone a course of PFMT (± other non-surgical treatments for their urge symptoms) with inadequate resolution of their symptoms.
Both the woman herself and her treating clinician should agree that surgery is an appropriate and acceptable next line of treatment.
Exclusion criteria
For the pilot trial (and currently anticipated for a future definitive trial), the following situations excluded eligibility:
Symptomatic uterovaginal prolapse requiring treatment.
Previous surgery for UI or pelvic organ prolapse (POP).
Urodynamic investigation within the last 3 years.
Neurological disease causing UI.
Current involvement in competing research studies (e.g. studies of investigation or treatment of UI).
Unable to give competent informed consent.
Withdrawal options
There were two trial withdrawal options:
Withdrawing completely, that is withdrawal from the allocated investigation protocol and provision of follow-up data. Consent would be sought to retain data collected up to the point of withdrawal and to complete an ‘end of study’ visit at the time of withdrawal.
Withdrawing partially, that is withdrawal from the allocated investigation protocol (including a request to move to the alternative investigation arm) but continuing to provide follow-up data by attending clinic and completing questionnaires.
Participants’ reasons for withdrawal were recorded where possible, as the information might be relevant to the protocol for a future definitive study.
Recruitment
Potential trial recruits were identified by the study research nurses prior to attending new or follow-up appointments for SUI or MUI in the clinics run by the unit clinical leads. The Patient Information Sheet (PIS) was available in two forms: a short (one-page) introduction to the study and a more detailed (six-page) description of the trial and the implications of involvement (see Appendix 5 and 6). The short PIS was sent out along with a letter of invitation (see Appendix 2), with new appointments or with a reminder letter to attend follow-up appointments; this allowed any questions that the woman may have about the study to be addressed at the one visit; the full PIS was provided on request. Those declining to take part underwent further investigation and/or treatment as appropriate at the same visit. Those agreeing to take part signed a study consent form (see Appendix 10); with the patient’s agreement, the general practitioner (GP) was notified of their involvement in the trial (see Appendix 3).
Where other potential recruits became apparent only at the time of a clinic visit, they were invited to take part in the study and given verbal and written information. After a period of at least 24 hours to read, consider and discuss the information with family and/or friends, the research nurse contacted the patient by telephone to respond to any further outstanding questions and review their decision regarding involvement.
Patient and public involvement
In order to ensure that issues of importance to women undergoing IUTs would be addressed by a future definitive trial, advice and opinions were sought from patients and patient advocates at all stages of the INVESTIGATE-I study, particularly at the time of its conception, design and initiation. One of the trial grant holders (BSB), a clinical researcher, was the past chair of the Bladder and Bowel Foundation (B&BF), a patient-led support and advocacy organisation. B&BF members, staff and trustees were involved at the early stages of trial development in co-ordinating the involvement of patients in reviewing the protocol, materials and grant applications. The B&BF was also involved in identifying patient members for the Trial Steering Committee (TSC). Incontinence is a sensitive issue that is seldom discussed or acknowledged in public, so identifying women who were willing to participate in this capacity was less straightforward than may be the case in other areas of health care.
A particular challenge for the trial was the design of materials such as the PISs. In addition to explaining clearly the trial’s purpose and what involvement would mean for participants, these had to address two issues specific to the trial that are not common to many studies.
First, a diagnostic test that is routinely used, even an invasive test, is often accepted without question by patients in the belief that it will serve to inform treatment decisions. In this context, explaining the absence of good evidence of its value and the equipoise that exists between a diagnostic test and no test is more challenging than explaining equipoise between two treatments. It was important that participants understood that they were not being denied an effective element of the care process.
Second, a feasibility study may not be perceived to be as important as a definitive trial by potential participants. The PIS had to outline the potential importance of a feasibility study in making best use of public funds by informing the design of a definitive trial that could ultimately result in less invasive but equally effective patient care pathways.
Lay members of the TSC and a previous service user (trial participant) were involved in reviewing the plain English summary.
In a future definitive trial, a broader spread of patient and public representation could be sought. This might include, women’s network members from professional organisations or research support structures (e.g. Royal College of Obstetricians and Gynaecologists and Research Design Services); ex-patients; and, ex-trial participants. A Patient Advisory Group facilitated by one of the research team could serve to increase the level of engagement from patient and public representatives. As a result of our experiences in these feasibility studies, it would be intended to extend patient and public involvement (PPI) throughout the whole development and implementation of a definitive trial, including, design of the research (through contribution to proposal and protocol development); formulation of patient information materials (through consultation with PPI representatives); and, trial management (through membership of TSC), reporting and dissemination (through contribution to trial publication and presentation to lay audiences).
Randomisation
To ensure concealment of allocation, randomisation was undertaken by an internet-accessed computer randomisation system held by the Newcastle Clinical Trials Unit (NCTU); randomisation between intervention and control was 1 : 1 and was stratified by centre using random block length. The recruiter logged into the system by password and site identification code and then entered the date of birth and initials of the patient they were randomising. The system responded with a unique randomisation number and the trial arm to which the patient had been randomised. This was viewed on the screen and backed up with an e-mail confirmation to the individual carrying out the randomisation and also copied to the central trial office.
Sample size
The sample size for the external pilot trial was determined pragmatically, using the recommended minimum of 30 participants per arm.55 We aimed to recruit 60 participants per trial arm to investigate both the distribution and key parameters of the outcome measures. Previous trials in the area of pelvic floor dysfunction, including investigation,29 surgical44,56,57 and non-surgical treatments58 suggested average attrition rates of 13% (7–20%) between identification and randomisation, 16% (6–20%) between randomisation and treatment, and 13% (9–20%) between treatment and follow-up at 6 months. Taking the more pessimistic figure in each case, we estimated that a total of 240 eligible patients should be approached allowing for 50% overall attrition. The recruiting units collectively undertook 540 relevant procedures per year; therefore, identifying 240 eligible women within the originally planned 9-month recruitment period should not have presented undue difficulty.
Blinding
It was neither feasible nor appropriate to blind participants or clinicians (investigating and operating) as to the allocation of investigation strategy.
Interventions
Patients were randomised [documented on case report form (CRF) – ‘visit 1′ – see Appendix 19c] to receive either:
no IUT: basic clinical assessment supplemented by non-invasive tests as directed by the clinician; these included frequency/volume charting or bladder diary, mid-stream urine culture, urine flow rate and residual urine volume measurement (by ultrasound), or
IUT: basic clinical and non-invasive tests as above, plus invasive urodynamic testing. Dual-channel subtracted cystometry with simultaneous pressure/flow voiding studies is the most commonly applied technique in the evaluation of patients prior to surgery for SUI in most centres. Videourodynamics and ambulatory bladder pressure monitoring are used as alternative or additional invasive tests in some units; these tests were also permissible within the pilot trial, at the discretion of the clinician.
Given the pragmatic nature of the pilot trial, we were not prescriptive about which tests were carried out, nor indeed about exactly how they were carried out, save for the expectation that they would conform to good urodynamic practices.59,60 For this reason, we do not feel it appropriate to give a detailed description of the interventions in accordance with the TIDieR guidelines.61 Readers wishing to understand more about the interventions might refer to standard texts,62,63 or to standardisation documents.59,60
Further investigation was undertaken, where appropriate, at the same visit or a later one, as per local custom, and the treatment plan formulated.
Outcome measures
In INVESTIGATE-I, we were primarily concerned with determining the number of eligible patients in each unit, and the rates of patient recruitment, randomisation, retention and response. We also piloted the collection of the outcome measures for a future definitive trial, to assess data yield (e.g. percentage of recruited participants returning completed questionnaires) and quality (e.g. completeness and consistency of responses within returned questionnaires). This information was collected to guide the choice and mode of administration of questionnaires and data collection tools in a future definitive trial.
In a definitive trial, we would intend to use patient reported outcome measures as opposed to the more traditional methods for the quantification of leakage as the primary outcome. Our preferred primary outcome, rehearsed in the pilot trial, was:
the combined symptom score of the ICIQ Female Lower Urinary Tract Symptoms (ICIQ-FLUTS) questionnaire at 6 months after treatment.
43
Secondary outcomes for the future trial, also rehearsed in the pilot, comprise:
Further details of the scoring systems applied to the ICIQs and UDI are given in Appendix 16.
Thus, within INVESTIGATE-I, we piloted the collection of the above outcome measures, to assess data yield (e.g. percentage of recruited participants returning completed questionnaires) and quality (e.g. completeness and consistency of responses within returned questionnaires). This information can then be used to guide the choice and mode of administration of questionnaires and data collection tools in a future definitive trial.
Baseline assessment of study outcomes
Following consent and randomisation, patients were given a pack of baseline study outcome questionnaires; these were presented in the order ICIQ-FLUTS, ICIQ-LUTSqol, ICIQ-UI SF, UDI, EQ-5D and SF-12 (see Appendix 17). Participants were asked to complete the questionnaires at home, within 2 weeks of receipt, and to post their responses, using the addressed prepaid envelope provided, to the Trial Manager at the NCTU.
Subsequent treatment within the trial
Following investigation, it would be expected that women randomised to the control (no IUT) arm of the study, i.e. those treated on the basis of clinical assessment and non-invasive tests (documented on CRF – ‘visit 2’ – see Appendix 19e), would undergo surgical treatment (documented on CRF – ‘visit 4′ – see Appendix 19g) (). Given the pragmatic nature of the study, the choice of operation was left to the individual surgeon and patient; as only primary cases were included, it was anticipated that this would be either a retropubic or transobturator foramen mid-urethral tape procedure in most cases. Those randomised to the intervention (IUT) arm, i.e. undergoing invasive urodynamic testing (documented on CRF – ‘visit 3′ – see Appendix 19f), had similar surgical treatment when urodynamic stress incontinence (USI) was confirmed (documented on CRF – ‘visit 4’). Where other diagnoses were identified following investigation, alternative treatments might be offered (documented on CRF – ‘visit 5′ – see Appendix 19i); these included bladder retraining, anti-muscarinic drug treatments, neuromodulation, botulinum toxin injections (where DO was diagnosed), or clean intermittent self-catheterisation (where a VD was identified). Exactly which of these interventions was chosen depended on what conservative treatments had been used before entry into the trial; for example, if a woman had tried PFMT plus bladder retraining before entry, she was likely to be offered anti-muscarinic drug treatment if DO was shown on invasive urodynamic testing. In all centres the treatment algorithm employed was in keeping with the then current NICE recommendations (2006).17 In some cases where mixed abnormalities were reported, women would first undergo one or more of these interventions (to stabilise bladder overactivity, or improve voiding efficiency) and then proceed to surgery for SUI. After the participant entered the study the clinician remained free to recommend alternative investigation or treatment to that specified in the protocol at any stage if they felt it to be in the participant’s best interest. In these cases the participant remained in the study for the purposes of follow-up and data analysis.
Diagram of the study design and the flow of participants. a, The choice of non-surgical treatments is left to the clinician and patient, but may include bladder retraining, drugs, neuromodulation, botulinum toxin injections, and clean intermittent catheterisation, (more...)
Any adverse events (AEs) or serious adverse events (SAEs) were documented in the CRF (see Appendices 20 and 21); SAE notification was faxed to the NCTU within 24 hours.
Follow-up
Clinicians arranged postoperative follow-up or other outpatient review, as per their normal practice and timing (documented on CRF – ‘visit 6’ – see Appendix 19h). Patients were sent a pack of follow-up study outcome questionnaires along with a prepaid envelope by the NCTU at 6 months after the start of treatment (i.e. 6 months after the date of surgery, or the start of any non-surgical intervention, or period of ‘watchful waiting’). This applied in all cases, even where surgery was undertaken as a secondary intervention in those women initially treated non-surgically. They were asked to complete the questionnaires at home and return them to the NCTU. Those failing to return questionnaires within 1 month of the initial request were contacted by the appropriate research nurse by telephone, to encourage responses. In the last 9 months of the study, the option of completing the questionnaire over the telephone with the research nurse was also given to participants during the reminder telephone call. If the questionnaires were not returned after the telephone reminder, a further copy of the questionnaires was mailed to the participant with a reminder letter. The patients withdrawal or completion of study follow-up was documented on CRF – ‘visit 7’ (see Appendix 19k).
Governance and regulatory arrangements
Ethics and research and development approval
The conduct of this study was in accordance with the ethical principles set out in the Declaration of Helsinki (2008)69 and the Research Governance Framework for Health and Social Care (second edition, 2005).70 Application for ethical approval was made through the Integrated Research Application System, and a letter of favourable ethical opinion was obtained from Newcastle & North Tyneside 1 Research Ethics Committee (REC) on 6 January 2011 – reference number 10/H0906/76. Application for research and development (R&D) approval was made via the NIHR Co-ordinated System for gaining NHS Permissions (CSP) – reference number 62776. Global sign-off for R&D approval was received on 15 March 2011, with local R&D approvals of the protocol between 28 March 2011 and 9 August 2011.
Changes to the original protocol
Two amendments were made to the original protocol. The first (v1.1; dated 1 July 2011) added detail to the protocol on the collection of health economics outcomes from the study, and included the Participant Costs Questionnaire (PCQ) and the trial management plan as appendices. The second (v1.2; dated 12 September 2012) related to a change in the method for follow-up reminders; as in the original protocol, a telephone reminder would be undertaken by the local site research nurse if the questionnaire had not been returned after 4 weeks; in addition, if after the telephone reminder, the questionnaires were not returned within a further 2 weeks, a further copy of the questionnaires would be mailed to the participant with a reminder letter. Both amendments were approved by the study sponsor and by Newcastle & North Tyneside 1 REC.
Clinical trials agreements
Clinical trials agreements (CTAs), using the model for non-commercial research within the health service, were established for the various study sites with sponsor Newcastle upon Tyne Hospitals NHS Foundation Trust (NuTH) between 25 May and 15 August 2011. Site initiation visits took place between 30 March and 17 June 2011, with the start to recruitment permitted (‘green light’ to proceed) only after completion of all regulatory approvals and site initiation, between 14 June and 15 August 2011 (for the primary sites) ().
Dates of R&D approval, CTA and site initiation visit on the primary and later study sites
Following approval of an extension to recruitment, one additional recruiting site and two PIC sites were approved.
Consent
Women were informed about the detail of the study with the brief and more detailed PIS, and by discussion with the local research nurse independently of the clinician responsible for ongoing care, and of staff undertaking investigations. Patients provided written informed consent. Separate written consent to take part in the qualitative patient interview substudy was sought, and it was made clear to trial participants that they were under no obligation to take part in the qualitative substudy (see Chapter 7).
To inform the design of a future definitive trial, those who declined to participate in the trial or who withdrew prematurely were asked for their reasons for withdrawal, but the right to refuse to participate without giving reasons was also respected.
Other regulatory arrangements
Other regulatory arrangements for the study, relating to confidentiality, indemnity, on-site monitoring and internal audit, day-to-day management by the Trial Management Group (TMG), and oversight by the TSC and Data Monitoring and Ethics Committee (DMEC) are described in detail in the study protocol, the latest version of which is available on the NIHR Journals Library website.
Encouraging participant recruitment
It is unclear why some trials appear to recruit more easily to target than others.71 Factors related to the research question itself (e.g. being a cancer or drug trial), related to trial organisation (e.g. having a dedicated trial manager) and related to treatment access (e.g. involving a treatment only available within the trial) have been shown to be associated with more successful recruitment. Other strategies have been employed to encourage recruitment for example, newsletters and mailshots, although it has not been shown unequivocally that these are causally linked to changes in recruitment.72,73 One of the aims of a feasibility study is to investigate how well units are able to identify eligible trial participants and recruit them. A number of additional strategies were employed within INVESTIGATE-I, partly to encourage recruitment in the pilot itself, but more particularly to rehearse them as possible strategies within a future definitive trial. These included the establishment of additional study sites, and strategies to facilitate communication and staff engagement.
Additional study sites
Following approval by HTA of a 9-month extension to recruitment (initially 2 months, then a further 7 months), one additional full recruiting site (South Tees Hospitals NHS Foundation Trust) and two PIC sites (Sunderland Royal Hospital and South Tyneside District General Hospital) were established.
Communication and staff engagement
Study acronym and logo
The full study title incorporated the underlying clinical question addressed, the overall study methodology, and identified the trial element as having a randomised design. The short title (INVasive Evaluation before Surgical Treatment of Incontinence Gives Added Therapeutic Effect?), study acronym (INVESTIGATE-I), and logo (incorporating a graphic image of dripping and calmed water) did not simply provide a random selection of letters from the full title to give a snappier sound bite. They each serve to complement and ‘stand for’ the full title, add to the effectiveness and understanding of the message, by a representational name and image. They were used in all communications to trial staff, regulatory authorities, other clinicians, patients (other than when site specific stationery was appropriate) and the trial website, and as such provided a constant identity for the INVESTIGATE studies. The importance of such study ‘branding’ is emphasised in the STEPS study.72
Basecamp
Basecamp© (developed by www.37signals.com, Chicago, IL, USA) is a web-based project management application; this was used for communication and document sharing between members of the TMG and between the TMG and other members of the research team, particularly those based outwith Newcastle.
Trial website
A trial website (www.investigate-trial.com) was developed early during the project as a means of increasing awareness of the INVESTIGATE studies within the research team, for other staff at the various study sites, for clinical colleagues who might be interested to learn more and perhaps to collaborate in a future trial, and for the general public. It includes information about the current study (INVESTIGATE-I), including the justification, methodology, and recruitment progress; reference is also made to a possible future definitive study; trial governance arrangements are included, with appropriate links; PISs and study newsletters (v.i.) are available for download, and there are links to open-access publications from the INVESTIGATE studies; contact details for the research team and site clinical staff are also provided. All sections are updated as necessary, and a ‘latest news’ section on the home page gives topical issues regarding trial progress and staff development (see Appendix 22).
Study newsletters
Study newsletters were circulated to the research team every 2 to 3 months during the trial period. These covered, information about the study, including protocol amendments; progress with the trial and interview studies; feedback from the TSC and DMEC meetings, and from the trial funder; details of study presentations and publications; personal news from the trial team (see Appendix 23). Progress with recruitment against target was included using the ‘Recruitment to Target’ (RtT) thermometer (©Newcastle upon Tyne Hospitals NHS Foundation Trust and Newcastle University) (v.i.).
Recruitment updates
At times when recruitment was a particularly acute concern, a weekly progress update was distributed to the research team. These were employed in particular during the 2-month provisional extension (during which 50% recruitment had to be completed in order to secure a further extension) and in the final weeks of recruitment. These updates were limited to information on recruitment, but showed this by centre, with a competitive edge to encourage peer rivalry; progress was illustrated in a variety of ways [e.g. using a ‘league table’; the RtT thermometer; black, red, amber, green (‘BRAG’) flag status (black = zero recruits, red = > 24% off target, amber = 15–24% off target, green = 0–14% off target); and countdown clock and filmstrip graphics (see Appendix 24)].
‘Recruitment to Target’ thermometer
During the construction of the study website, a graphic device described as the ‘RtT (Recruitment to Target) thermometer’ was developed to help trial staff visualise progress against recruitment target numbers and timing. This was initially formatted in Microsoft PowerPoint (Microsoft Corporation, Redmond, WA, USA), as a simple graphic image illustrating actual recruitment against recruitment target (including a BRAG status pennant, colour-coded as above), and time expired of the available study recruiting time, in the form of a ‘maximum and minimum thermometer’. It was then converted into hypertext markup language (HTML) code that can easily be adapted for use in any trial, and added into a website (see Appendix 25). The use of the device was subsequently disseminated for use in other studies via the NCTU trial managers and Comprehensive Local Research Network (CLRN).
Results
Screening
Overall, 771 patients were identified by research nurses from clinic notes and correspondence as being potential recruits into the study, and were sent the PISs. Of those screened, 284 were deemed eligible for the trial, giving a ‘screen positive’ rate of 37%. The reasons for non-eligibility of screened patients are shown in ; most commonly these were patients not having undergone supervised PFMT prior to referral (14%), urgency or urgency predominant MUI (12%), failure to attend clinic appointments (11%), patients not wishing to participate (8%), patients with prolapse requiring treatment (5%), or clinicians feeling that surgery was not appropriate (5%). Although the reasons for non-eligibility varied between centres, the overall figures were obviously heavily weighted by the centre screening the highest number of patients. In some units, patients not wishing to participate made up a larger proportion of screening failures; overall however, 78% of eligible women identified were recruited into the study.
Screening and recruitment numbers by centre including screening codes (1–15), for those women not randomised, sorted by overall frequency of reporting of codes
The numbers screened at individual centres varied between 14 and 399; the percentage of eligible women recruited varied between 55% and 100%, but did not show an obvious trend with the number screened (see ). Although a single code was assigned to each patient, it is possible that codes were used variably in the different centres, and that there may have been some inconsistency or overlap in the use of codes. For example, it is possible that ‘patient does not wish to participate’ could overlap with ‘patient does not wish surgery’. While the centres screening larger numbers of women also recruited larger numbers (), the conversion from screening to recruitment decreased as the screening number increased ().
Numbers screened and recruited at individual centres.
Number and percentage recruited to trial by number screened (shown on log scale) at each centre. The graph plots the percentage and number of participants recruited. The number of participants is indicated by black filled circles and the percentage of (more...)
Quality assurance of screening processes
In view of the variations seen in screening and recruitment between centres, a quality assurance check was made with PIs and recruiting staff in each unit, confirming that all employed a similar practice in relation to screening; this was stated in the study protocol as follows:
Potential trial recruits will be identified by the study research nurses prior to attending new or follow-up appointments for SUI or MUI in the clinics run by the unit clinical leads. The Patient Information Sheet (PIS) will be sent out with new appointments or with a reminder letter to attend follow-up appointments; this will allow any questions that the woman may have about the study to be addressed at the one visit. Those declining to take part would undergo further investigation and or treatment as appropriate at the same visit. Those agreeing to take part will sign a study consent form.
If other potential recruits become apparent only at the time of a clinic visit, they will be invited to take part in the study, and will be given verbal and written information. After a period of at least 24 hours to read, consider and discuss the information with family and/or friends, the research nurse will contact the patient by telephone to respond to any further outstanding questions, and review their decision regarding involvement.
It is possible that women referred to the various centres were in some way different, although the workload and nature of the units would have made this unlikely. The number of women screened in individual centres would therefore be expected to be determined by the ease with which PIs or research nurses were able to identify eligible women from referral letters or hospital notes. It might also be a reflection of their individual position on the spectrum of sensitivity versus specificity, that is whether they perceived the priority as being only to screen those women who were very likely to be eligible, or saw the importance of ‘broadening the net’ to include all potential recruits. In view of the pragmatic intention of the pilot trial, we did not give a strict definition to the term ‘stress predominant MUI’, preferring to leave it to clinicians to determine this within their own practices. It is possible that individual screeners or PIs may have interpreted the term variably, such that this also could have contributed to variation in recruitment rates.
In order to explore these issues further, a series of 20 identical vignettes were distributed to screeners via the trial Basecamp site. These were mainly based on actual GP referral letters, although in some cases with modifications to cover the range of inclusion and exclusion criteria. Sixteen vignettes mentioned one or more definite inclusion criteria (SUI, stress predominant MUI, PFMT, family complete); the other four had possible inclusions (UI but not specified as to stress or urgency related; ‘wet all the time’; PFMT mentioned but level of supervision not specified). Four had definite exclusions (previous pelvic floor surgery; neurological disease; urgency predominant MUI) and 15 contained possible exclusions (unsupervised PFMT). The vignettes are shown in Appendix 26.
Each of the 11 screeners from the seven full recruiting units graded the vignettes independently, on the basis of the following instructions:
What we want to know is whether you would have considered each of the women described in the letters to be a potential recruit for the INVESTIGATE-I trial. In other words, if you had reviewed the letter at the time that we were looking for recruits into the trial would you, or would you not, have sent out a Patient Information Leaflet (PIL) to the woman described (please tick either ‘Yes’ or ‘No’ in the blue boxes on the score sheet). It would also be helpful to know whether you feel the decision is clear-cut, or borderline (by ticking in the appropriate green box), and something of why you made that decision (by ticking the orange boxes and adding comments as appropriate), on the score sheet provided.
The possible responses were, therefore, clear cut ‘Yes’ (Y); borderline ‘Yes’ (?Y); borderline ‘No’ (?N), or clear cut ‘No’ (N). Each screener’s grading for the various vignettes is shown in . For six vignettes, everyone agreed that the patient was eligible; for one, all agreed that the patient was not eligible; the grade breakdown for the remainder was mixed.
Screener responses to the 20 vignettes. Data are sorted vertically by the rate of positive screening (% yes) for each vignette, and horizontally by % yes for individual screeners
Assuming a majority decision was one in which the ‘%Yes’ grading was above or below 50% (irrespective of whether the decisions were considered to be clear-cut or borderline), in other words that the majority felt that the patient described in the vignette was (or was not) eligible for screening, then there were 34 occasions on which one or more individual screeners ‘disagreed’ with the majority. The number of ‘disagreements’ varied across the 11 screeners; this ranged from one screener who dissented from the majority decision for 1/20 vignettes to another who dissented in 7/20 vignettes. reports these separately as occasions on which the screener said ‘Yes’ when the majority said ‘No’, and those on which the screener said ‘No’ when the majority said ‘Yes’. The former judgement would lead some patients being deemed eligible and sent the PIS when they might be found to be ineligible at a later appointment (i.e. erring on the side of over-inclusiveness at the screening stage). The latter judgement would lead to some potential recruits not being invited to take part in the trial when they would have been eligible. Given the difficulty in recruiting patients in some centres, it is the latter judgement that should be minimised within trials.
Free-text comments were sought to help clarify the screeners’ decisions. These included:
Vignette 2 (majority view – clear-cut ‘yes’) Four comments, all along the same line,that is the letter did not specifically mention PFMT; they appeared, therefore, to have taken the view that it had not been done rather than ‘might have been done’.
Vignette 3 (majority view – clear-cut ‘yes’) One comment: ‘Need to check notes and if documented that pt [patient] has stress incontinence and received PFMT then would be eligible but if it is only on patient’s say so then further investigations would be beneficial to give a diagnosis.’ The vignette did specifically state: ‘Complaining of stress incontinence. She denies any urgency and says that when she coughs and laughs she passes small amounts of urine.’ As well as: ‘She has tried pelvic floor exercises including an internal pelvic toner to no avail’.
Vignette 5 (majority view – borderline ‘yes’) Five comments, essentially taking the view that the vaginal laxity was the greater problem and the incontinence less of an issue. Physiotherapy report (included with referral) states: ‘She has attended on 3 occasions in total and reports that her continence symptoms have become more manageable but not completely resolved’ and ‘on examination there was no significant vaginal or uterine vaginal or uterine descent’.
Vignette 6 (majority view – borderline ‘yes’) One comment, essentially same as vignette 3 (same screener).
Vignette 9 (majority view – borderline ‘yes’) Three comments, all along the same lines – no supervised physiotherapy, and best assess later.
Vignette 10 (majority view – borderline ‘no’) One comment, highlighted the patient had previous surgery and may not have done PFMT, but indicated ‘yes’ to screening.
Vignette 11 (majority view – borderline ‘yes’) Four comments, indicating need for PFMT (this was not mentioned in the letter, although it did state that the patient wished to consider surgery); two also referred to young age and therefore uncertainty of family plans.
Vignette 12 (majority view – borderline ‘yes’) Two comments, one relating to complaint of ‘dragging sensation’, one to need for PFMT (not mentioned in letter).
Vignette 13 (majority view – clear-cut ‘no’) One comment on definition of ‘repair operation’.
Vignette 16 (majority view – borderline ‘yes’) Three comments both relating to the history of OAB. Letter states:
She has been treated in the past for urinary problems, and has had a number of medications, and says that she even had Botox injections to her bladder. Since these latter interventions her symptoms have changed somewhat; previously she reported both urge and stress incontinence, but now she is left with only the stress element, with leakage occurring particularly on coughing or sneezing, or when she is at the gym.
Vignette 18 (majority view – clear-cut ‘no’) Most referred to lack of supervised PFMT specifically indicated in letter. One commented that ‘Patient may feel she has done 6 months physio and it may be agreed that surgery is an appropriate treatment now’.
Vignette 19 (majority view – clear-cut ‘yes) One comment related to treatment for rectal (not uterovaginal) prolapse.
Vignette 20 (majority view – borderline ‘yes’) One comment referred to need for pad at night and that this could represent OAB or fistula.
Hence the majority of the explanatory comments related to missing information, most commonly whether or not PFMT had been undertaken at all, or whether or not it had been supervised. A number also related to reports of vaginal laxity or dragging sensation, although information about clinical findings in relation to POP either was not present or was negative. There were also uncertainties or misinterpretations of the significance of descriptions of rectal prolapse and repair surgery.
Differences between units were not clearly apparent and the relationship between disparity in screening categorisation in this exercise and screening to recruitment ratios in the trial itself was also not obvious.
In a future trial it would be appropriate to:
ensure that definitions in inclusion and exclusion criteria are clarified (e.g. prolapse symptoms vs. clinical findings vs. need for treatment; rectal vs. uterovaginal prolapse, etc.)
suggest that where information is missing from referral letters it is assumed the patient might be eligible, and therefore that the default action should be to send out the PIS, unless obvious exclusions are specified
arrange group training/standards setting sessions for PIs and research nurses to agree a consistent approach to the screening and recruitment process across sites.
Recruitment
Monthly recruitment by centre is shown in and , for the initial recruitment period (up to the end of March 2012) and for the period of extension (from April to December 2012). Regulatory requirements took approximately 3 months longer than anticipated and, as a result, recruitment targets were revised. Even once all approvals were in place, and all sites in a position to start recruitment, the rate of accrual was significantly less than required with some sites unable to identify any patients for some weeks after opening to recruitment. Although proposed in 2011,74 the NIHR 70-day benchmark for recruitment was not published until 2012 and was not a required of CLRNs until after 2013.75 Nevertheless, several steps were introduced to improve recruitment, including the incorporation of additional clinicians on two of the existing sites and the establishment of an additional full recruiting site and two PIC sites. A request was made for a 9-month unfunded extension to the recruitment period.
Monthly recruitment numbers by centre. Original and revised predictions of recruitment are shown to the left of the table and actual recruitment by centre to the right; horizontal bars in individual site columns denote the date at which recruitment could (more...)
Monthly total recruitment numbers. The original and revised predictions of overall recruitment are shown as continuous and dashed lines, and actual recruitment in histogram; the overall ‘BRAG’ flag or ‘RtT’ status is also (more...)
The number of participants recruited per recruiting month (i.e. between the completion of all site-specific regulatory requirements and the end of the study) varied between 0.4 and 3.9 per month at the original sites (mean 1.9); at the additional full recruiting site this figure was 2.5 per month; the PICs did not identify any potentially eligible patients for referral to a recruiting site in the 8 months that they were active.
Randomisation
Of the 284 women screened positive, 222 agreed to randomisation into the trial, giving a trial consent rate of 78%. This recruitment total (222) represented 93% of the planned sample size (240) for the pilot trial. Overall, 110 women were randomised to the control or no IUT arm and 112 to the intervention or IUT arm. Immediately after randomisation it became apparent that one woman in the no arm was ineligible for the trial and she was withdrawn leaving a total of 221 eligible patients randomised (109 in the no IUT arm and 112 in the IUT arm).
The screening, recruitment, randomisation and trial follow-up are summarised in the CONSORT diagram shown as .
Trial CONSORT flow diagram. DNA, did not attend.
Retention
Demographic data and details of any subsequent treatment for incontinence were collected from hospital notes and CRFs (see Appendices 19a–k), and women were asked to complete questionnaires on clinical outcomes (see Appendix 17) and a 3-day bladder diary (see Appendix 18) at baseline and 6 months after the start of treatment (i.e. 6 months after the date of surgery, or the start of any non-surgical intervention, or period of ‘watchful waiting’). Baseline questionnaires were sent to 219 women and returned by 165; this represented a response rate of 75% overall, 72% in the IUT arm and 79% in the no IUT arm. At 6 months after treatment, questionnaires were returned by 63% (125/200) of those who were sent questionnaires at follow-up; 56% (54/97) in the IUT arm and 69% (71/103) in the no IUT arm.
Six women returned a completely blank questionnaire booklet (three in each study arm); one further woman in the IUT arm completed only the EQ-5D and SF-12 questionnaires, but for the purpose of return of primary outcomes this was categorised as returning a blank questionnaire, as the ICIQ-FLUTS was not completed. This information is summarised in the trial CONSORT diagram . Six of the seven women who returned blank questionnaires reported ‘no significant urinary symptoms’ on the follow-up CRF (visit 6). The same six either annotated the front of their questionnaire or bladder diary, or in one case telephoned the NCTU indicating that they had not had urinary problems since their surgery. One of the women who returned a blank questionnaire reported ‘significant urinary symptoms’ on the follow-up CRF; she also annotated her diary to indicate that there had been little change in her urinary symptoms following her surgery, although she improved slightly with subsequent drug treatment.
The progress of recruitment and follow-up is shown in . It also shows the anticipated follow-up at 6 months, although these predictions do not make allowance for individual centre waiting times for investigation and surgery; this was certainly an error that would require attention in planning a future definitive trial. The actual times at which follow-up questionnaires were posted out to participants (at 6 months after surgery or start of treatment) do reflect these waits, and illustrate an average additional delay to follow-up of approximately 4 months. also illustrates the actual rate at which follow-up questionnaires were received back at the NCTU. At the time of closure of the database for final analysis, 125 follow-up questionnaires had been received (exceeding the target of 120), although as per the CONSORT diagram, seven of these omitted primary outcome data – ICIQ-FLUTS total score.
Actual recruitment with anticipated and actual receipt of follow-up questionnaires. Lines illustrate actual monthly recruitments, anticipated follow-up (based on 6 months after recruitment), anticipated follow-up (based on questionnaires posted – (more...)
Demographic data
provides the demographic data by trial arm; the consistency of these variables between IUT and no IUT arms confirms the validity of the randomisation process.
Summary of demographic data at baseline by trial arm
Completeness of data collection
The questionnaire packs contained four condition-specific scales (ICIQ-FLUTS, ICIQ-UI SF, ICIQ –LUTSqol and UDI), two general health scales (EQ-5D and SF-12) and a 3-day bladder diary. When the questionnaire packs were reviewed, it was evident that not all patients had completed all scales in their entirety, although missing values within individual scales were few. The columns to the right-hand side of show the proportion of each questionnaire or subscale that could be calculated from the data provided.
Summary of numeric outcome measures by trial arm and data collection time point
At baseline, the ICIQ-FLUTS overall score could be calculated for 98% of subjects who had returned the questionnaire pack and was partially completed by only 2%. No patients provided an incomplete submission for all subscales of this instrument, and the completion rates were therefore slightly higher for individual ICIQ-FLUTS subscales than for the overall score. The completion rates for the ICIQ-UI SF and ICIQ-LUTSqol scales were 99% and 95%, respectively, and for the UDI scale was 84%. For the latter three scales, there were occasional questionnaire packs in which the whole scale had not been completed at baseline.
At 6 months after treatment for incontinence, the ICIQ-FLUTS overall score could be calculated for 90% subjects who had returned the questionnaire pack and was only partially completed for 4%. The completion rates for the ICIQ-UI SF and ICIQ-LUTSqol scales were 91% and 87%, respectively, and for the overall UDI scale was 81%. For all four scales, there were occasional questionnaire packs in which the whole scale had not been completed at 6 months. The distribution of missing data on these scales and subscales is described in . It was found that 6% of all items making up the ICIQ-FLUTS overall score were missing. Most women had no missing items, but there were seven women who failed to complete any item in this scale. There were similar low percentages of missing items in the other three scales, and the numbers of women who failed to complete any item on a scale were two for ICIQ-UI SF, four for ICIQ-LUTSqol and one for UDI. These high completion rates suggest that there were few problems with individual items on a scale for women in the pilot trial.
Descriptive statistics on missing data on questionnaires returned at 6 months
The right-hand columns of show how many items on the 3-day bladder diary were available. Only 148 women returned the diary at baseline (68% of those women sent baseline questionnaires). Data were available in 99% of the returned diaries to compute the average number of visits to the bathroom during the day and night, although the average number of pads used in 24 hours was only available on 65% of returned diaries. This latter variable was not completed at all in 30% of diaries and was partially available in 5%.
Summary of average 3-day bladder diary outcomes by trial arm and data collection time point
At 6 months after treatment, 105 diaries were returned (53% of those sent the 6-month questionnaire pack). Data were available on the average number of visits to the bathroom on all of these, but only 40% of the 105 diaries that were returned completed the diary for the number of pads used; 12% partially completed it and 48% provided no data on pad use at all. Additionally, 10 women returned blank bladder diaries, five in each study arm. Five of these women annotated the diaries to indicate that they did not have current symptoms (four in the no IUT arm and one in the IUT arm).
The response rate at both time points for the bladder diary was low, and data on the number of pads used was a particular problem using this diary format. It should be noted that ‘pad use’ was recorded in a single box at the bottom of the diary sheets (see Appendix 18) and may have been more easily overlooked by patients than other items on the diary.
Comparison of responders and non-responders to the six-month questionnaire
In view of the unexpectedly high rate of non-response to the 6-month questionnaires, a limited comparison of responders and non-responders was made on the basis of their clinical follow-up. A total of 135 women had a postoperative follow-up visit documented on the study database; 93 actually attended an outpatient clinic and 42 had a review by telephone (routine practice in three of the centres).
During clinical follow-up, 17 women reported significant urinary symptoms, and 13 had significant clinical findings on examination (including four tape extrusions); none of those with positive examination findings reported symptoms. The symptoms specified by 13 of these 17 women included, one to three episodes of UTI (three women); OAB symptoms (five women); other incontinence symptoms (three women); suprapubic pain (one woman); and only one woman reported persistence of SUI.
Of the 125 women who returned follow-up questionnaires at 6 months after treatment, 83 had clinical follow-up, of whom 12/83 (14.5%) described significant urinary symptoms, and 9/83 (10.8%) had significant examination findings, at the clinical review. Of the 81 who failed to return follow-up questionnaires at 6 months following treatment, 52 had clinical follow-up, of whom 5/52 (9.6%) described significant urinary symptoms, and 4/52 (7.7%) had significant examination findings. While those women returning the 6-month questionnaires somewhat more often had significant symptoms or examination findings at earlier clinical review than those failing to do so, the numbers do not allow meaningful statistical comparison.
Questionnaire data
Baseline
shows the distribution of the questionnaire scales at baseline by trial arm. The ICIQ-FLUTS total score has a possible range of 0–48. The distribution of ICIQ-FLUTS total score at baseline was fairly symmetrical with a mean of 16.9 (SD 5.7) in the IUT arm and 16.4 (SD 6.3) in the no IUT arm. The distributions of the other scales and subscales were similarly well matched between the IUT and no IUT arms and were fairly symmetrical.
Six-month follow-up
also shows the distribution of the questionnaire scales at 6-month follow-up by trial arm. The distribution of ICIQ-FLUTS total score at follow-up had a mean of 9.2 in the IUT arm and 6.9 in the no IUT arm. The distribution of ICIQ-UI SF (possible values 0–21) had a mean of 5.3 in the IUT arm and 3.3 in the no IUT arm. The distribution of ICIQ-LUTSqol (possible values 19–76) had a mean of 26.7 in the IUT arm and 25.3 in the no IUT arm. The distribution of UDI overall score (possible values 0–300) had a mean of 49.1 in the IUT arm and 33.9 in the no IUT arm. For all scales, typical scores were much lower than at baseline. The distribution of the ICIQ-FLUTS total scores at 6-month follow-up by trial arm is shown in the upper part of . The shape of these distributions at 6 months was generally positively skewed, which reflects the fact that many women had experienced considerable relief from their initial symptoms, but some had not.
Distribution of ICIQ-FLUTS total score at 6 months and changes between baseline and 6 months by trial arm. Graphs by randomisation group.
It is difficult to interpret any difference in mean scores between baseline and 6 months follow-up from , because many of the women who provided baseline data failed to do so at 6 months. shows the distribution of the paired changes in scale scores for those women who had completed both questionnaires. It can be seen that the mean change in ICIQ-FLUTS total score was 7.8 in the IUT arm and 9.3 in the no IUT arm. The distribution of the change scores for the ICIQ-FLUTS total scores is shown in the lower part of . Typically, there was a marked drop in these scores over 6 months, but little difference in the mean changes between the trial arms. This pattern was also seen in the other four scales. However, no formal comparison between arms is appropriate in a pilot study.
Summary statistics for paired changes in scale scores (baseline – 6 month)
Bladder-diary data
shows the results from the 3-day bladder diaries by trial arm.
Baseline
The mean number of daytime bathroom visits was 7.4 in the IUT arm and 7.6 in the no IUT arm. The average number of night-time bathroom visits was 0.9 in the IUT arm and 0.8 in the no IUT arm. The average number of pads used in 24 hours was 2.8 in the IUT arm and 2.7 in the no IUT arm. The two arms were well balanced at baseline.
Six-month follow-up
The mean number of daytime bathroom visits was 6.8 in the IUT arm and 6.2 in the no IUT arm. The average number of night-time bathroom visits was 1.3 in the IUT arm and 1.1 in the no IUT arm. The average number of pads used in 24 hours was 1.7 in the IUT arm and 0.5 in the no IUT arm. The two arms at had similar distributions at this time point.
Treatment data
gives details of the surgical treatment received by the trial subjects for their UI. In the IUT arm, 80% received surgery, compared with 95% in the no IUT arm. For those undergoing surgery, additional details are given further down the table. It can be seen that the distributions of operation type, grade of surgeon, anaesthetic technique and use of antibiotic prophylaxis were similar between the trial arms.
Summary of surgical treatments received for UI by trial arm
Details of the non-surgical treatments are given in and . One woman in the no IUT arm and four (4%) in the IUT arm decided to defer any treatment initially (designated as ‘watchful waiting’). A further 15 women (15%) in the IUT arm underwent lifestyle changes or other non-surgical treatments. As routine incontinence management, more than one lifestyle change was commonly documented, and other non-surgical treatments were often used in combination; 28 treatments were applied in these 15 women. Despite (unsuccessful) completion of a course of supervised PFMT being an inclusion criterion for the trial, six women underwent further PFMT alone (two) or in combination with other non-surgical treatments (four).
Summary of non-surgical treatments received for UI
Non-surgical treatment combinations used in individual patients by study arm
