Background

In the Philippines, demand for family planning (FP) among women of reproductive age (WRA) is high: the most-recent estimates from the 2017 National Demographic and Health Survey (NDHS) indicate that 17% of currently married women ages 15–49 years and 49% of sexually active unmarried women ages 15–49 years have an unmet need for FP.^*,1 Fortunately, the Philippines Department of Health (DOH) is committed to providing “information and access, without bias, to all methods of FP,” as indicated in the Responsible Parenthood and Reproductive Health (RPRH) Act of 2012.² In parallel, the Philippines’ DOH FP Program’s vision is “for Filipino women and men [to] achieve their desired family size and fulfill the reproductive health and rights for all through universal access to quality FP information and services.”³ The Philippines National FP Program and the RPRH Act of 2012 enumerate efforts to achieve this vision. They include strengthening FP commodity procurement and distribution, building demand for FP through public awareness–raising efforts, and facilitating coordination with nongovernmental organizations and the private sector to improve access. These efforts address the National FP Program objectives to (1) increase the modern contraceptive prevalence rate (mCPR)^† among women from 24.9% in 2017 to 30% by 2022 and (2) reduce the unmet need for modern FP from 10.8% in 2017 to 8% in 2022.³

This report conveys technical details of the model Reach Health Assessing Cost-Effectiveness for Family Planning (RACE-FP), a decision support tool developed for FP policy makers in the Philippines. The objective of RACE-FP is to estimate the impact single or combined FP interventions have on averting unintended pregnancies at national and regional levels and on downstream maternal and neonatal health (MNH)–related outcomes. A secondary RACE-FP objective is to estimate intervention and supply costs to inform key decision-makers of how to best allocate limited resources to achieve FP-related goals. RTI International, the lead implementing partner of the US Agency for International Development (USAID) Reach Health Project, initiated RACE-FP.

RACE-FP was based on the Maternal and Neonatal Directed Assessment of Technologies (MANDATE) model. MANDATE is an evidence-based, mathematical model designed to estimate the impact of interventions—such as preventive interventions, diagnostics, treatments, and transfers to different care settings—to reduce maternal, fetal, and neonatal mortality in sub-Saharan Africa and India.⁵ Developed in 2014 with funding from the Bill and Melinda Gates Foundation, the MANDATE web-based tool⁶ is freely available for public use, used by more than 100,000 unique users across 184 countries, and referenced in multiple publications.^5,7–14 MANDATE has been adapted and expanded to support decision-making on several topics in different contexts; adaptations include Model for Assessment of Pediatric Interventions for Tuberculosis (MAP-IT),^15,16 Assessing Diarrhea and Pneumonia Treatments (ADAPT),¹⁷ and Zimbabwe Averted Pregnancy (ZAP).¹⁸ RACE-FP is the most-recent adaptation of MANDATE.

Guide to This Report

This report describes the analytical framework behind the estimates in the RACE-FP model. First, we present information on the underlying structure of the model and how the population flows through the model to obtain estimates for the baseline scenario. Next, we present the baseline results, along with an example demonstrating how expanding a FP intervention changes results. Additional details on data sources, parameters, assumptions, calculations, and limitations can be found in supplementary materials, which we reference throughout the report. Readers can obtain a full picture of RACE-FP by reviewing both the report text and the supplementary materials (available at https://doi.org/10.5281/zenodo.6029905). This report contains the following sections:

▪

Model Design and Structure provides an overview of the RACE-FP model framework, including the conceptual model, modeled outcomes, time periods of analysis, and model inputs.

▪

Model Population describes the modeled population and how it flows through the model to obtain estimates for the baseline scenario. Details on how the main at-risk population was identified are in Supplement A (Model Population), and supplementary information on the parameters affecting the population is in Supplement B (Contraceptive Penetration, Utilization, and Effectiveness).

▪

Interventions outlines the different interventions in the model, intervention parameters, and the relationship between these intervention parameters and the main at-risk population. Supplement D (Intervention Parameters) further elucidates the intervention parameters.

▪

Baseline Results summarizes how RACE-FP uses the intervention parameters to compare baseline values to the user-created scenario. Baseline values for outcomes are also included in this section, and supplementary detail on how these values were calculated is in Supplement E (RACE-FP Outcomes). Information on how contraceptive commodity costs were calculated is in Supplement C (Contraceptive Commodity Costs).

▪

Model Sensitivity and Uncertainty Analyses presents the methods and partial baseline results of the sensitivity and uncertainty analyses. Complete baseline probabilistic sensitivity analysis (PSA) results are in Supplement F (Model Sensitivity Results).

Supplements are referenced throughout the report and include definitions, calculations, assumptions, limitations, and sources used to identify needed parameters for use in RACE-FP.

Model Framework

RACE-FP is a deterministic decision tree mathematical model that we initially designed as a modification of MANDATE. We developed the underlying RACE-FP model in Excel and made it available in a user-friendly, interactive web-based platform called UPmod.¹⁹ In 2021, RTI created UPmod to provide an intuitive interface for mathematical models developed in Excel or other programming languages. Hosting RACE-FP on UPmod facilitates use and discussion among key Philippines stakeholders; however, this methodology report focuses on the underlying Excel model.

Baseline and Scenario Runs

As a deterministic mathematical model, RACE-FP compares estimates from a baseline run to a user-developed counterfactual scenario. RACE-FP runs both the baseline and scenario models with identical starting populations (for details, see Supplement A: Model Population). While the baseline model assumes no change in the availability of FP interventions (i.e., the baseline model estimates status quo contraceptive use and downstream effects), the scenario model takes user changes to the intervention parameters into consideration. WRA who are at risk of unintended pregnancy and not using any method to prevent pregnancy are the main at-risk population. RACE-FP applies a series of probabilities, user-entered decisions, and assumptions around contraceptive access and use to this at-risk population to estimate unintended pregnancy and related downstream outcomes. By increasing intervention exposure parameters, the user is modifying the number of people exposed to the intervention which will, in turn, increase contraceptive access and utilization rates. RACE-FP results display how outcomes differ between the baseline and scenario to support FP policy decisions.

For example, if a policymaker in the Philippines is interested in expanding efforts to train public sector providers in FP service provision, they can use RACE-FP to examine how this expansion would affect FP outcomes, MNH outcomes, and cost. Typically, a user will estimate the impact of expanding an intervention; however, RACE-FP allows reductions as well, which may be useful for resource allocation. RACE-FP is not a stochastic model; therefore, the differences in outcomes between a baseline and scenario run are solely the result of changes in interventions.

Modeled Outcomes

The primary objective of RACE-FP is to estimate the reduction in unmet need and unintended pregnancy that results from FP interventions. RACE-FP also estimates the number of related MNH outcomes (e.g., maternal death, neonatal death), the number of users of each FP method, and the cost to support increased contraceptive use and intervention expansion. The Results page presents outcomes comparing baseline and scenario(s) for the geographic location specified when the user defines the population of interest. The user can toggle between age groups (adolescents, adults, or all women) when viewing results. Table 1 lists outcomes included in RACE-FP. For details on definitions, calculations, assumptions, and sources used to calculate outcomes, see Supplement E: RACE-FP Outcomes.

Table 1

Outcomes included in RACE-FP.

Time Period

RACE-FP is a single-time-period model with a baseline year of 2018 representing the status quo. However, the user can adjust the time period by updating baseline model parameters to reflect the year of interest (e.g., updating the parameters that use 2018 Philippines NDHS data when the next report is published). The model assumes all users initiate a method at the start of the modeled time period and are covered through the modeled year. We included a cost and inflation tool within the model so the user can convert costs to a different year by inserting the year of interest and corresponding Consumer Price Index with a base year of 2012 (the Philippines Consumer Price Index used for 2018 was 115.4 with a base year of 2012).

Model Inputs: Parameter Identification and Validation

Philippines-specific data parameters were used whenever possible, ideally from a Philippines government source. If Philippines-specific data were not available for a parameter, we used regional estimates (e.g., Asia, Southeast Asia) or countries as close geographically or culturally to the Philippines as were available.

If data from 2018—our baseline year—were not available, we used data collected as close to 2018 as possible and either assumed that the available data reflected 2018 values or adjusted the parameter to reflect the best estimate for 2018 (e.g., Philippines Statistical Authority 2015 Census data and multipliers were used to estimate disaggregated population parameters, and cost data were converted to 2018 Philippine pesos [Php]).

Although we developed the RACE-FP model to support regional decision-making, we scrupulously examined available regional- and provincial-level data and compared them with national-level data for the greatest model accuracy possible. National-level estimates were used if the regional-level data were deemed inadequate (e.g., the sample size was small and considered not generalizable). Currently, data in RACE-FP allow for national-level Philippines estimates and for estimates in the regions of NCR, Central Visayas, and Caraga; if users have access to specific regional-level data where national-level estimates were used, they can update parameters to be more specific as appropriate. Furthermore, if a user is interested in exploring the impact of FP interventions in a region not included in the model, RACE-FP allows the user to add a new geographic area along with relevant parameters specific to the location.

RTI researchers based in the United States collaborated with the RTI Reach Health team based in Manila to identify the most-appropriate parameters for the RACE-FP model. The Reach Health team in Manila coordinated with contacts at the DOH to obtain Philippines-specific data sources and review and validate parameters.

Inputs used to populate the model can be classified into three categories: population, contraceptive, and intervention parameters.

Interventions Included in RACE-FP

RACE-FP is organized by objectives of interest to the Philippines DOH, each of which include one or more interventions. Objectives were identified in collaboration with key stakeholders from the Philippines; many of the objectives aligned with national DOH objectives. Interventions within each objective were informed by available data. For example, although the ideal exposure parameter for an intervention to improve postpartum FP (PPFP) would be the proportion of WRA who received PPFP, those data were not available. Therefore, we used the proportion of WRA receiving postnatal care (PNC) as a proxy. Table 3 lists the objectives and interventions included in RACE-FP.

Table 3

Objectives and interventions to improve FP in the Philippines included in RACE-FP.

Intervention Parameters

For each intervention included in the model, we needed to obtain parameter values for intervention exposure, success, method distribution, and cost. Intervention parameter values are not specific to geographic location; however, the user can change values for current exposure, success, method distribution, and cost if—upon reviewing the definitions—the user determines that they have more-accurate estimates to represent their population. Intervention parameter values are used to calculate the number of women in the main at-risk population who are exposed to an intervention within the modeled time period and the number of women for whom the intervention is successful (i.e., those who begin using a contraceptive method within the modeled time period). Detail on all intervention parameters used in the model can be found in Supplement D: Intervention Parameters.

Intervention Population Adjustments

Although all interventions theoretically affect the main at-risk population, the intervention parameters are applied to a population specific to each intervention. For example, increasing the proportion of adolescent-friendly health facilities only affects adolescents, and increasing the proportion of public providers trained in FP service provision only affects those who receive care from public facilities. Therefore, each intervention has a corresponding set of population adjustments to create an “applicable population” that the intervention parameters affect. The Intervention Assumptions column in Supplement D: Intervention Parameters documents applicable populations for each intervention.

Baseline FP Outcomes

In this run, we estimated that 7.3 million WRA used a new contraceptive method in 2018, and nearly all (97.9%) method users were adults. We estimated that approximately 322,000 of all 5 million adolescents ages 15–19 (6.4%) had some level of need or demand for contraceptives by adding together adolescents with met need (170,610) and unmet need (152,149); this represents all adolescents except those with no need (i.e., abstinence, current pregnancy, and those trying to conceive). As nearly 90% of adolescents report they are abstinent in NDHS, the low rate of need among adolescents in RACE-FP is not surprising.

The proportion of demand satisfied is the number of WRA with met need divided by the demand (met need + unmet need). Adolescents in the baseline model see about half of demand satisfied (52.9%), whereas adults have nearly 70% of demand satisfied.

mCPR is also higher among adults, at 29.1%, compared with adolescents at just 2.7%. In addition, RACE-FP presents the number of users of each contraceptive, and these figures can be applied to the total number of users in the current time period to determine the method distribution. For example, we estimate that 131,682 adults were sterilized, representing just 1.8% of the 7,174,505 total adult non-continuing users in the modeled time period, implying that sterilization is a relatively uncommon new method among adults (RACE-FP assumes adolescents do not have access to sterilization). Note that the 131,682 adults represent new users in the modeled time period, so the total number sterilized—or using other long-acting methods—will be higher in aggregate. Oral contraceptives are the most popular modern method among both adolescents and adults (estimated total of 3.5 million users). TFP is the second-most popular method overall, with an estimated 2.4 million users in the modeled time period. The third most-popular method (or second-most popular modern method) among all WRA is injectables (834,809 users).

Baseline MNH Outcomes

Reducing the main at-risk population—or, inversely, increasing the number of FP users—is a key driver of MNH outcomes, so unintended pregnancy is the primary outcome RACE-FP measures. Other MNH outcomes (e.g., unsafe abortion, miscarriage, maternal deaths) are contingent on these estimates of unintended and intended pregnancies. At baseline, we estimate there were 1.4 million unintended pregnancies and 2.6 million total pregnancies in 2018. We estimate that there were 1.3 million live births in 2018, which is lower than the 1.7 million cited in the 2018 Philippine Health Statistics report.²¹ This discrepancy may be attributable to our parameter for the percentage of pregnancies that end in miscarriage (18.7%), which was recorded using data from 1980.²²

We estimate approximately 1,600 maternal deaths in 2018, which corresponds to the 1,600 estimate from the 2018 Philippine Health Statistics Report.²¹ Baseline results estimate that approximately 2 million unintended pregnancies were averted because of the use of contraceptives. Table 4 shows other downstream effects (e.g., unsafe abortions averted and maternal deaths averted) attributable to modern method use or all method use.

Table 4

Baseline outcomes for RACE-FP.

Baseline Cost Outcomes

Contraceptive costs represent the total indirect and direct cost of all commodities WRA used in the modeled year. In effect, this is the number of method users multiplied by the cost for each commodity. In total, we estimate that contraceptives cost approximately 2.9 billion Php in 2018. Total contraceptive cost is calculated by summing the estimated contraceptive costs for each method.

Sample Scenario Run

In the sample scenario run, we focused on one intervention: Increase the proportion of public sector providers trained in FP service provision. As detailed in Supplement D: Intervention Parameters, exposure is defined as the proportion of public facilities with a provider who has received FP competency-based training. We expanded the baseline exposure of 45.4% to 60.0%.

Table 5 illustrates how a few select indicators are affected compared with baseline results when the proportion of facilities with a trained provider is expanded from 45.4% to 60.0% for a 1-year period beginning in 2018. As a result of the expanded exposure, an estimated additional 105,000 WRA who are at risk of pregnancy and not covered by a preexisting method would begin using a contraceptive method, and mCPR would increase by 9 percentage points. This investment is estimated to prevent 32,700 unintended pregnancies and 19,300 unsafe abortions. Costs of this investment are disaggregated by the cost of the intervention (12 million Php) versus the cost of supplying additional contraceptives to support the increased number of FP users (62 million Php). The user can examine the additional cost associated with this expansion (72 million Php [2018]), examine the improvements in FP and MNH outcomes, and weigh the additional costs against additional benefits.

Table 5

Baseline vs. sample scenario outcomes.

Although this scenario is an example of expanding just one intervention, RACE-FP users can incorporate multiple intervention changes within their scenarios to compare with the baseline. For example, a user could create a scenario that, in addition to expanding training of public sector providers, expands FP training for private sector providers and reduces stockouts in all three possible delivery settings. RACE-FP also can compare results from multiple scenarios at once to help the user find the optimal set of interventions for their populations of interest (adolescents, adults, all WRA; Philippines national or regional level).

Parameter Selection

For each region (Philippines, NCR, Central Visayas, and Caraga), the model contains contraceptive PUE parameters for each of the 10 contraceptive methods (abstinence, sterilization, LAM, TFP, MNFP, male condoms, oral contraceptives, injectables, IUDs, and implants), two distinct age groups (adolescents, adults), and three delivery settings (public, private, and community). In total, for each region, the combination yields a total of 191 parameters: 180 PUE parameters plus 11 cost parameters (10 birth control method costs plus total intervention cost).

Because of the complexity of the underlying PUE parameters, for each region, we did not include all 180 PUE parameters in the default PSA. Because the initial population in RACE-FP includes all WRA, if several parameters at the top of the decision tree experience a maximum positive movement, it would choke out the available movement for the downstream sections of the tree. For example, abstinence among adolescents is 93.7%, and additional increases would leave little to no room for changes in FP interventions to affect outcomes. Instead, we identified a set of the most-impactful PUE parameters to include in the default PSA, as described below. However, a user can include or remove any of the 180 PUE parameters in the multivariate PSA. We used the overall Philippines region to select parameters as a proxy for the other regions.

After consulting with subject matter experts, we conducted the following process to select a set of 20 parameters to be used in the PSA out of the 191 total parameters (Figure 2):

1.

Exclude cost parameters (remaining N = 180)

2.

Exclude PUEs not relating to FP methods affected by interventions (i.e., remove abstinence and TFP) (remaining N = 144)

3.

Exclude effectiveness parameters,^§ assuming they are fixed (remaining N = 96)

4.

Run one-way sensitivity analyses, using the number of unplanned pregnancies as the outcome of interest, on each of the remaining 96 parameters to identify the 20 most impactful via the width of the range on the outcome.

Figure 2

Consort diagram of parameter selection for one-way sensitivity analysis.

One-way sensitivity analyses examine the impact of varying values of model parameters on selected outcomes, one at a time. Figure 3 presents results of the one-way sensitivity analysis in a tornado chart; the parameters at the top of the diagram had the greatest impact on the outcome (unintended pregnancy), and parameters at the bottom had the least impact.

Figure 3

One-way sensitivity analysis for unplanned pregnancies.

RACE-FP is very sensitive to penetration and utilization parameters related to oral contraceptive use in adults. The most-impactful parameter—the utilization rate of oral contraceptives among adults in community settings—led to a range of nearly 65,000 unintended pregnancies in the one-way sensitivity test. We expected this parameter to be impactful, considering oral contraceptives are the most popular method in the Philippines.¹ Not surprisingly, the 10 most-impactful parameters were five pairs of penetration and utilization rates (Figure 3).

The PSA tool used the 20 parameters highlighted in Figure 3 to generate the full output of a 500-iteration multivariate PSA for the baseline, including minimum, maximum, median, and mean values for all modeled outcomes, as shown in Supplement F: Model Sensitivity Results. Table 6 shows an excerpt of results from Supplement F. The number of contraceptive users had a range of approximately 470,000, varying between 7.10 million and 7.57 million, with a mean of 7.33 million. mCPR was between 23.4% and 25.1%, with a mean and median of 24.3%. The number of unintended pregnancies had a range of roughly 140,000, varying between 1.35 million and 1.49 million, with a mean of 1.42 million.

Table 6

Excerpt of PSA results.

RACE-FP Limitations

All models are based on several assumptions and contain abstractions and simplifications. As such, users should interpret results with some degree of skepticism.²⁴ It is important for users to understand the limitations of a model and the implications of those limitations for the results of interest. RACE-FP is subject to the same limitations as other mathematical models, and we made every effort to transparently document our definitions, sources, and assumptions to facilitate informed decision-making based on the model results.

The primary challenge of developing the RACE-FP intervention was data availability for model parameterization. For many of the parameters related to the impact of the interventions, we lacked Philippines-specific data and instead used published data from studies conducted in other countries. For some parameters, we were able to use Reach Health data (e.g., Usapan intervention^** or mobile outreach intervention parameters), but those data may be specific to subpopulations targeted in the Reach Health program and not representative of the full population. In addition, because reliable parameters at the regional level were not always available, we used national-level estimates. Although we generally assumed that intervention parameters were the same across regions, there may be considerable differences within the Philippines in the uptake of different types of FP interventions. We mitigated these challenges by developing a flexible model that allows users to revise parameters if they have more-recent or local data. We also transparently documented parameter values, assumptions, and sources on a resource page that is intended to support users’ understanding of how data limitations and assumptions about parameter values may affect model findings (these details can also be found in the supplementary materials referenced throughout this report). Such information can help users avoid unintentional misuse of the model. As with any model, RACE-FP will require maintenance, and parameters will need to be updated regularly to avoid estimates becoming obsolete. Although this upkeep will require time and resources, it will contribute to sustainability.

Another limitation is that RACE-FP uses a decision tree model structure. Decision tree models are useful for modeling acute conditions and situations, like pregnancy, that have a short, fixed time horizon; however, these models cannot capture the complexity of longer-term FP decision-making.^25,26 An alternative model structure, such as an individual-level simulation model, would allow users to assess the impact of changing FP decisions over a longer time horizon, including over the lifecycle of women of childbearing age. This type of model would allow users to assess the impact of increasing uptake of longer-term FP options, such as sterilization and long-acting reversible contraceptives. Individual-level simulation models can also reflect differences in individual characteristics, such as age, race/ethnicity, and prior pregnancy outcomes, and allow for different parameter values for people with these different characteristics. However, such models require detailed data to parameterize and validate the model, and such granular data are currently unavailable. Additionally, more-complex model structures can be computationally intensive, and it is more difficult for a general user to understand the model calculations than a decision tree model, especially for models like RACE-FP that include a wide range of potential interventions.

Finally, a key limitation is that most of the RACE-FP development work, especially in obtaining local stakeholder input on the model’s parameter values, occurred during the COVID-19 pandemic. During this time, international travel from the United States to the Philippines was not permitted, and we obtained all project staff input on model parameters virtually. Moreover, COVID-19 strained the public health system and health professionals, who, as a result of their attention to the pandemic, had little time available to provide input on model parameters, share relevant data, and review assumptions about parameter values in the model. We therefore parameterized the model using the best data available and developed comprehensive documentation of our methods, which we share in this report, to support future collaboration with the relevant officials and project team members to further validate and update the model.

Future Directions

RACE-FP is a useful tool to support FP decision-making, as it estimates how investments in specific FP interventions in the Philippines may affect FP and MNH outcomes in that country. In the future, we plan to conduct additional validations of RACE-FP, incorporate user feedback from initial testing of the model with intended users, and adapt the model for use in other countries. First, efforts to continue validating and obtaining buy-in from key stakeholders in the Philippines are crucial to ensure familiarity and comfort using RACE-FP. Capacity-building activities and collaborative detailed review of parameters will contribute to identifying parameters that need updating and appropriate values for them, especially at the subnational level. Initiating RACE-FP with one region before bringing to scale, if desired, will further allow for model strengthening. With greater understanding of how the model operates, users will know how best to interpret results.

Second, RACE-FP functionalities can be upgraded. to best suit user needs. We can expand the number of FP interventions of interest; build on the number of geographic regions included beyond NCR, Central Visayas, and Caraga; modify the results to be presented by setting (public, private, community); account for population growth beyond 2018 or allow the model to be run for multiple years at a time; and incorporate other variables, such as disability-adjusted life years or cost-effectiveness outcomes. We can also explore the option to change the pathway of the model algorithm. For example, we can input desired FP outcomes and format results to present areas of potential investment needed to accomplish these goals.

Third, the established RACE-FP framework allows for direct adaptation for other countries, so this tool can support policymakers worldwide. Although initially developed for the Philippines, RACE-FP has broad applicability and can be tailored to support FP decision-making wherever country- and regional-level data are available.