Table 132, Philips checklist for immunomodulatory agents - Cystic Fibrosis

Section	Pass/fail	Comments
Structure
Statement of decision problem / objective	P	Decision problem stated clearly in title of review question, and clarified in the model structure section
Justification of modelling approach	P	Model structure justified with reference to clinical expert opinion
Statement of scope / perspective	P	No statement of scope, but table of contents makes scope explicitly clear so there is no risk of ambiguity
Structural assumptions	P	Assumptions justified in section ‘Model structure’, and clinical relevance confirmed with Guideline committee. Various structural assumptions relating to particular treatments or exacerbations explained in relevant sections.
Strategies / Comparators	P	Very nonstandard approach to transition probabilities (see below), but otherwise structure is highly consistent with other similar models in the area
Model type	P	Choice of Markov Model obvious. Limitations of this model type discussed and sensible attempts to address these limitations have been made
Time horizon	P	Lifetime time horizon, in keeping with Reference Case
Health states/disease pathways	P	Health states carefully considered - especially choice of FEV states
Cycle Length	P	Cycle length unusual (first cycle is 9 months, subsequent cycles annual) but justified with reference to literature on short-term effects of treatment. Subsequent annual cycle length standard.
Parsimony	P	Structural components of the model carefully chosen to aid understanding, especially the number and extent of health states.
Data
Data Identification	P	Systematic review of published literature
Data Synthesis	P	Synthesis strategy well justified and explained. Significant difficulty with integrating rates occurring across cycles that didn’t match the model cycle length, but approach to this explained and defensible.
Discounting	P	3.5% as specified in Reference Case
Analysis of trial data	P	Data analysed at most appropriate level
Treatment effects	P	Odds ratios derived from trials and superimposed on population baseline risks generated through regression model
Transition probabilities	P	Derivation of transitions probabilities not standard as model attempts to map a continuous process onto a discrete-state model. Nevertheless the methodology employed here is well-described, and validated by NICE TSU
Mortality	P	Discussion of mortality in model write-up; CF has a very poor prognosis and so life tables not appropriate. Data from Vertex used to calculate ‘all cause CF’ mortality, and lung-transplant specific mortality appended to this. Model-specific mortality rates explained and justified.
Extrapolation	P	Significant extrapolation, but well justified in text with reference to committee expert opinion. Not possible to validate with reference to literature, as such literature does not exist
Risk factors	P	Evidence of nonlinear effect of risk factors on mortality sought and incorporated into model, for example by considering lung transplant as a separate state
Utilities	P	Utilities described in section on health-related quality of life, and justified with reference to literature. Model clear on baseline QoL and subsequent decrements
Charges and costs	P	Charges and costs described in section 1.3, and come from standard sources such as NHS Reference Costs and PSSRU
Adjustment over time / between countries	P	Costs inflated from historic values using standard sources
Half-cycle correction	F	No half-cycle correction undertaken. Defensible as cycle length much shorter than model time horizon, but half-cycle correction would be preferred
Data incorporation	P	Choice of data to incorporate and how the data are used is clear and well-justified
Uncertainty
General statement regarding sensitivity analysis	P	Described fully in sections on sensitivity analysis methods and results
Structural	N/A	Model structure not varied as committee opinion was that the structure was an effective one for investigating the review question
Methodological	P	Although discount rate not varied as per Philips (2004), substantial methodological variation examined and discussed
Parameter	P	Parameter uncertainty well investigated - Table 31 considers all relevant OWSAs that the committee suggested, and a PSA is further undertaken to reflect probabilistic uncertainty
Consistency
Internal	P	Model is highly robust to ‘stress testing’ such as putting extreme values into cells. Model behaves in an intuitive way, for example recommending treatments to which a substantial discount has been applied
External	P	Face validity confirmed with reference to Guideline committee. Additionally, values appear congruent with general clinical practice.
Between-model	P	Results consistent with literature on the topic, although literature is extremely sparse.
Predictive	N/A	Model not intended to be used predictively, and such predictive work would be well outside NICE methods manual

Section

Pass/fail

Comments

Structure

Statement of decision problem / objective

Decision problem stated clearly in title of review question, and clarified in the model structure section

Justification of modelling approach

Model structure justified with reference to clinical expert opinion

Statement of scope / perspective

No statement of scope, but table of contents makes scope explicitly clear so there is no risk of ambiguity

Structural assumptions

Assumptions justified in section ‘Model structure’, and clinical relevance confirmed with Guideline committee. Various structural assumptions relating to particular treatments or exacerbations explained in relevant sections.

Strategies / Comparators

Very nonstandard approach to transition probabilities (see below), but otherwise structure is highly consistent with other similar models in the area

Model type

Choice of Markov Model obvious. Limitations of this model type discussed and sensible attempts to address these limitations have been made

Time horizon

Lifetime time horizon, in keeping with Reference Case

Health states/disease pathways

Health states carefully considered - especially choice of FEV states

Cycle Length

Cycle length unusual (first cycle is 9 months, subsequent cycles annual) but justified with reference to literature on short-term effects of treatment. Subsequent annual cycle length standard.

Parsimony

Structural components of the model carefully chosen to aid understanding, especially the number and extent of health states.

Data

Data Identification

Systematic review of published literature

Data Synthesis

Synthesis strategy well justified and explained. Significant difficulty with integrating rates occurring across cycles that didn’t match the model cycle length, but approach to this explained and defensible.

Discounting

3.5% as specified in Reference Case

Analysis of trial data

Data analysed at most appropriate level

Treatment effects

Odds ratios derived from trials and superimposed on population baseline risks generated through regression model

Transition probabilities

Derivation of transitions probabilities not standard as model attempts to map a continuous process onto a discrete-state model. Nevertheless the methodology employed here is well-described, and validated by NICE TSU

Mortality

Discussion of mortality in model write-up; CF has a very poor prognosis and so life tables not appropriate. Data from Vertex used to calculate ‘all cause CF’ mortality, and lung-transplant specific mortality appended to this. Model-specific mortality rates explained and justified.

Extrapolation

Significant extrapolation, but well justified in text with reference to committee expert opinion. Not possible to validate with reference to literature, as such literature does not exist

Risk factors

Evidence of nonlinear effect of risk factors on mortality sought and incorporated into model, for example by considering lung transplant as a separate state

Utilities

Utilities described in section on health-related quality of life, and justified with reference to literature. Model clear on baseline QoL and subsequent decrements

Charges and costs

Charges and costs described in section 1.3, and come from standard sources such as NHS Reference Costs and PSSRU

Adjustment over time / between countries

Costs inflated from historic values using standard sources

Half-cycle correction

No half-cycle correction undertaken. Defensible as cycle length much shorter than model time horizon, but half-cycle correction would be preferred

Data incorporation

Choice of data to incorporate and how the data are used is clear and well-justified

Uncertainty

General statement regarding sensitivity analysis

Described fully in sections on sensitivity analysis methods and results

Structural

N/A

Model structure not varied as committee opinion was that the structure was an effective one for investigating the review question

Methodological

Although discount rate not varied as per Philips (2004), substantial methodological variation examined and discussed

Parameter

Parameter uncertainty well investigated - Table 31 considers all relevant OWSAs that the committee suggested, and a PSA is further undertaken to reflect probabilistic uncertainty

Consistency

Internal

Model is highly robust to ‘stress testing’ such as putting extreme values into cells. Model behaves in an intuitive way, for example recommending treatments to which a substantial discount has been applied

External

Face validity confirmed with reference to Guideline committee. Additionally, values appear congruent with general clinical practice.

Between-model

Results consistent with literature on the topic, although literature is extremely sparse.

Predictive

N/A

Model not intended to be used predictively, and such predictive work would be well outside NICE methods manual

From: Appendix K, Health Economics

Cystic Fibrosis: Diagnosis and management.

NICE Guideline, No. 78.

National Guideline Alliance (UK).

London: National Institute for Health and Care Excellence (NICE); 2017 Oct 25.

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Table 132Philips checklist for immunomodulatory agents