Models and Curve Fitting Guidelines

For competitive binding assays and functional antagonist assays the most common summary measure of the dose-response curve is the IC₅₀, the concentration of substance that provides 50% inhibition. For agonist/stimulator assays the most common summary measure is the EC₅₀, the concentration giving 50% of that compound’s maximal response. Substantial variation in the methodology used to derive these values exists, and this variation has been shown to substantially impact overall assay variability. This section discusses important issues to consider and provides some guidelines on how to proceed. They are a based on the Data Standardization for Results Management chapter of this book. Consult that document for the specifics for each assay type. Consult a statistician to see if these guidelines are appropriate for your assay, and if other outcomes such as AUC or a threshold dose should be used.

Before fitting a concentration -response curve to obtain the EC₅₀/IC₅₀, each well should be converted to either percent activity or percent inhibition with respect to positive and negative controls (note: for simplicity all text below is stated for determining IC₅₀’s; determining EC₅₀’s is identical). Percent activity of all replicate wells from a given run, (including multiple plates per run) for a given concentration, should be averaged either by taking the mean, or preferably, taking the median. Outliers less influence the median values when there are 3 or more replicates. Thus only one averaged point per concentration per run is used to fit the dose-response equation to the data. This is because replicate wells on either the same or different plates are often correlated with each other and, thus, do not provide true replication of the experiment.

The four parameter logistic model (4PL), also called the Hill-Slope model, is the most common equation fit to in vitro concentration-response data. One form of the equation is

where, y is the percent activity and x is the corresponding concentration. The fitted IC₅₀ parameter is the relative IC₅₀, and is defined as the concentration giving a response half way between the fitted top and bottom of the curve. Some software, such as ActivityBase, also provides the absolute IC₅₀, which is defined as the concentration giving exactly a 50% response. The relative IC₅₀ is recommended for most assays (see notes below and Glossary for definitions). You should also report the fitting error, which is usually called the standard error by most software packages (we use the term fitting error to differentiate it from the standard error of the mean [SEM] derived from multiple determinations of a compound).

The 4PL model is the best model for concentration -response data, but there are cases where it should not be used. In some cases, due to the potency of the compound falling outside the concentration range, the data may not fully describe the bottom or top asymptote of the curve. In those cases, respectively, the bottom (3PLFB) or top (3PLFT) can be fixed to improve the curve fit. If you observe a substantial reduction in the % Fitting Error, and a better concentration-response plot of the fitted curve with respect to the actual data then you should switch to either the 3PLFB or 3PLFT model as appropriate.

Examples

All examples below are from receptor binding data fitting % Activity versus concentration (expressed by ActivityBase as log-concentration in the plots). For this type of assay, the top, bottom and slope parameters should in theory by 100, 0 and –1, respectively.

Figure 1 is a concentration-response best fit by the 4PL model. Both asymptotes are defined by the data, and the fitting error is approximately the same with all 3 models. Note that even though the fitting error is smallest with the top fixed (8.63% versus 9.51%), the reduction is not small enough to warrant the fixed top model, nor is there any material change in the IC₅₀. The fixed bottom model is clearly inappropriate as the data clearly defines a bottom >0.

Figure 1:

Curve fit results for a dose-response best fit by a 4PL model.

The fitting error is expressed here as a percentage of the fitted parameter value. For example, if the IC₅₀ is 0.061 and its fitting error is 0.0058, then the % Fit Error is 9.51%.

Figure 2 is best fit by the fixed top (3PLFT) model. The data does not define a top asymptote, and the fitted top (128.32) and slope (-0.58) from the 4PL model are inappropriate for this (binding) data. By fixing the top at 100% the fitting error is reduced from 57.54 to 21.55%, and the IC₅₀ increases by more than two-fold. Thus the 3PLFT model should be selected over the 4PL.

Figure 2:

Curve fit results for a dose-response best fit a by a 3PLFT model.

Figure 3 is best fit by a fixed bottom (3PLFB) model. Note that the data does not define the bottom asymptote, and the fitted bottom (41.54) and fitted slope (-1.83) from the 4PL are inappropriate for binding data. The fixed bottom model reduces the fitting error from 80.19% to 20.85%, while the IC₅₀ increases by more than two-fold. The fitted IC₅₀ (20.88 nM) is inside the dose-range (0.001-25 nM), and so it is appropriate to report this value. Note in this case Activity Base was unable to fit a fixed top model.

Figure 3:

Curve fit results for a dose-response best fit by a 3PLFB model.

Figure 4 illustrates the definition and effect of outliers (Figure 4 A). Outliers are single, vertically isolated points that are clearly inappropriate. The point is “obviously” erroneous. The effect of the outlier in this case is to bias the estimate of the bottom upwards, pulling it away from the other points of the data. In general, outliers can bias either the top, bottom or slope parameter depending upon where they occur in the concentration -response curve. It is appropriate to remove the outlier (Figure 4 B) and refit the points. Fixing top or bottom did not materially improve the curve fit (not shown).

Figure 4:

Curve fit results for a dose-response containing an outlier. (A) The effect of the outlier in this case is to bias the estimate of the bottom, pulling it away from the other points of the data. (B) Results once the outlier is removed and the curve refit (more...)

Figure 5 illustrates the effect of high assay variation. No single point stands out as “obviously erroneous”, and therefore it would be inappropriate to remove any points from the curve fit. Fixing top or bottom does not materially improve the curve fit, and so the 4PL model should be used. Note that the estimates themselves are not implausible, but the fitting error is 33.83%, which is caused by the relatively high assay variation.

Figure 5:

Curve fit results for a dose-response with high assay variability, but no outliers.

Notes:

1.

This equation can be fit to the data using Activity Base, Bravo Curve fit, JMP, Graphpad Prism or Sigma Plot. Note that the form of the equation varies from one software package to the next. Some, such as Graphpad Prism, fit Log-IC₅₀ instead of IC₅₀, and the equation looks quite different, but the results are the same as that shown above.

2.

The terms absolute and relative IC₅₀ are not universal. Both are usually just called the “IC₅₀”, and it’s left unstated which value is actually used.

3.

If your software toll reports Log-IC₅₀ then convert both the estimate and the % fitting error (%FE) according to the formulas
and

4.

There should be at least one point on both sides of the reported IC₅₀, i.e. the reported IC₅₀ should lie inside the concentration -range used in the assay. The intent of this rule is to make the IC₅₀ estimate an interpolation of generated data and not an extrapolation of generated data. Cases not satisfying this rule should not have an IC₅₀ reported or reported with a comment that indicates the value is extrapolated. If a value is reported, it should be “<X_min” or “>X_max”, as appropriate, where X_min is the lowest concentration and X_max is the largest concentration included in the analysis.

5.

It is a good idea to remove obvious outliers and then refit the curve without the outliers. Note that if it isn’t obvious, it isn’t an outlier. See examples 4 and 5 above to distinguish high variability from outliers.

6.

For competition assays, such as radioligand binding assays and competitive inhibition assays, the fitted slope should be within 2 (slope) fitting errors of the value 1, and slope estimates outside this range indicate assay problems that need to be investigated.

Control Compounds

Key assays in a project and assays where problems are suspected should have two control compounds, a primary and a secondary (this is referred to as Close Monitoring). All other assays should have at least a primary control (Regular Monitoring). Both compounds need to be run once per run, unless plate variability is suspected. In that case the primary control compound needs to be run once per plate. The purpose of the primary control is to ensure that there isn’t any “assay drift”, i.e. that the same compound has a stable K_i/K_b/EC₅₀ over time, and that the assay reproducibility (MSR, is stable over time. (MSR: Minimum Significant Ratio; see HTS Assay Validation).

The purpose of the secondary control is to examine the stability of results over a concentration -range. If problems do develop, then it is important to examine whether the entire concentration -range is equally affected (a small problem) or whether the concentration -range is differentially affected (a big problem). Also, two controls permit direct calculation of both the within-run and overall MSR’s, and a check that the MSR is consistent over a range of potencies.

The activity of the primary control should be at or near the most potent compound available, and ideally should be the Lead compound. There should also be sufficient stock of a single lot of the compound so that it can be run on a continuous basis for some period of time. Since the control compound is supposed to be representative of the test compounds, it should receive the same sample handling as all the test compounds, and not be specifically prepared and added to the assay outside of normal test compound procedures.

For the secondary control, IC₅₀ should be >100 fold less potent than the primary control. Otherwise it has the same requirements as the primary control. As the SAR develops the potency traditionally improves. So when the “best” compounds are more than 100-fold more potent than the primary control then select a new primary control. If the assay has a secondary control then the old primary control becomes the new secondary control, and the existing secondary control is dropped. If there is no secondary control, then it is suggested to run both primary controls over the first 6 runs of the new primary control.

A scatter plot for control compound for the values of log-K_i/K_b/EC₅₀ versus run date should be updated after every run and checked for problems. For assays with two control compounds the difference in log-K_i/K_b/EC₅₀ versus run date should be plotted, and for agonist and non-competitive antagonist assays the efficacy versus run date should also be plotted. Outliers and trends in the values of log-K_i/K_b/EC₅₀, either up or down (assay drift) should be checked visually, and problems investigated and corrected as they occur. Runs with significant numbers of outliers should be repeated.

After 6 runs compute the overall MSR of the assay based on the control compounds according to formula:

where s is the standard deviation of the log-K_i/K_b/EC₅₀ values. This MSR is the total or overall MSR (whereas the one computed in a test-retest study encompasses only the within-run variability), and should be less than or equal to 7.5. This standard comes from practical experience obtained thus far with assays in the company, and not theoretical statistical considerations. Note that this is a minimum standard that all assays should meet, and in practice chemistry requirements may indicate a smaller MSR (as low as 2-3) is required for some or all assays. The Project/Program Team should discuss this issue with a statistician to set appropriate MSR’s for their assays.

After each run, a running MSR plot should be maintained (i.e. computed from the last 6 runs) and checked to ensure the continued good reproducibility of the assay. The attached template (available from the online eBook) can be used to generate this with MSR values calculated from the last 6 runs.

Examples

Figure 6 illustrates results for an assay with a single control. Figure 6A shows the potency versus run date scatter plot, Figure 6B shows the moving MSR chart. The MSR points are based on the last 6 runs of the assay, i.e. the first point is computed using runs 1-6, the second point uses runs 2-7, etc. The Mean Summary section indicates the highest/lowest/last IC₅₀’s in the period were 22.63, 4.42 and 11.25 µM respectively (chart units are in nM). The overall average was 10.17 µM. The potency has no apparent temporal trends, and no unusual observations. Figure 6B shows the trends in MSR over time, which appears to increase until mid Feb-2002, and then decrease. However, the magnitude of the increase trends is quite small and well within the variation of an estimate based on a sample of size 6. The highest/lowest/latest MSR’s are 6.8, 2.7 and 2.7 respectively. The overall MSR is 4.4, which is not the average of the 6-run MSR’s but instead is a single estimate derived using the entire sample (18 data points in this case). This is a stable assay with moderate assay variation (3 < MSR < 5).

Figure 6:

Potency, MSR chart, and summary statistics for an assay with one control.

Figure 7 illustrates an assay with two controls. In Figure 7A, the red and blue lines represent the two compounds, and are positioned using the left axis. The green line is the potency ratio between the two compounds and is positioned using the right axis. Figure 7B shows the moving MSR values both within run and overall. The Overall-Overall MSR is the value to be reported. The within-run MSR’s are only for comparison backwards to the test-retest study results, and for times when compounds are compared within the same run of an assay. As with Figure 6, there are no apparent temporal problems, i.e. this is a stable assay with an overall MSR of 2.0. This assay is less variable than the assay in Figure 6.

Figure 7:

Potency, MSR chart, and summary statistics for an assay with two controls.

Figures 8 and 9 illustrate problems with a shift in compound potency. Figure 8 illustrates a steady degradation in potency over time, whereas Figure 9 illustrates a more sudden shift in potency at a particular point in time. In Figure 8 the assay variability appears to be shrinking, while in Figure 9 it appears to be stationary. Repetitive freeze-thaw cycles of a compound may cause a slow degradation in potency whereas a change in lot of a key assay ingredient may result in a sudden potency shift.

Figure 8:

Potency and MSR chart illustrating assay drift.

Figure 9:

Potency and MSR chart illustrating sudden change in potency.

In both cases it is important to identify the cause and correct it as soon as possible.

Figure 10 illustrates an assay with stable potency, but in June the assay variability increased. The moving MSR was stable around 3, but after June increased to over 10, and remained there. This also is most likely caused by a change in the assay process around that time. Again it is important to identify and correct the cause as soon as possible. Note however that a single outlier will cause the MSR chart to increase for the next 6 runs, and so it usually takes more time to correctly distinguish a change in assay variability from a single outlier result.

Figure 10:

Potency and MSR chart for change in assay variability.

Retrospective Studies

During the course of project/program development numerous compounds are repeatedly evaluated and stored in archival databases. This data can be mined to examine the reproducibility of assay results. This work should always be done by a statistician as the repeated compounds are not a random selection of all compounds, and may be biased with respect to time of evaluation, potency, structure and “assayability” (the latter term is meant to reflect conditions such as solubility, quenching, stickiness to plastic and other practical problems). In spite of these potential problems retrospective studies can be a very useful exercise, particularly in establishing the acceptability of older assays that have never been formally evaluated for reproducibility. In addition, the MSR can be examined over various subsets such as potency range, structure and run date to check that the control compound MSR’s are representative of the test compounds with respect to potency range, structure and run date.

Bridging Studies

If a key aspect of an assay changes, such as an equipment change or lot of a reagent, then a test-retest study should be conducted to verify equivalence of the two protocols. A judgment should be made on a case-by-case basis of whether the full protocol outlined in the HTS Assay Validation chapter needs to be made, or only a single run under old and new conditions (i.e. one might do just Step 4 of the procedure, or one might do both Steps 3 and 4 depending upon the severity of the protocol change). Also in cases of specific modifications such as replacing equipment for a particular step in the assay an experiment can be designed to validate that the replacement is equivalent to the original in the conduct of that step of the assay.

Dimethylsulfoxide: biological compatibility and compound storage.

Dimethylsulfoxide (DMSO) is a universal solvent for all compounds tested in high, medium and low throughput screens (HTS, MTS and LTS). Compounds are initially dissolved in 100% DMSO and further diluted in 100% DMSO screening and IC₅₀ or K_i determinations. So manyassays may require an additional dilution step in water or assay buffer to reduce the DMSO concentration to a level that is acceptable for the assay, depending upon the specific capabilities of the equipment being used. It is extremely important that the DMSO compatibility of biological reagents such as enzymes, receptors, protein/peptide reagents and cells be established to ensure that the screening assays are not adversely affected. In general, the final DMSO concentrations in cell-based assays are <0.2% and are <1% in biochemical assays. It is highly recommended that the tolerable DMSO concentration be determined individually for each validated assay.

DMSO is also used as a cryoprotectant in the freezing of cell cultures at ATCC. The product is cell culture grade and has been tested to ensure cell viability. Each lot is also tested for the absence of bacteria, fungi, and endotoxin.

When solubilized compounds are stored in DMSO, it is important to understand the stability of these compounds under various storage conditions and freeze-thaw cycles. A detailed study of these effects was published recently (1). It is believed that the degradation of DMSO solubilized compounds is mainly due to moisture absorbed from the air. This can happen during frequent freeze-thaw cycles of compounds stored frozen in DMSO, or frequent exposure to air during repeated access for biological testing (cherry-picking).

Recommended storage conditions for DMSO solubilized compounds:

• Polypropylene plates.
• Storage temperature: 10 degree C or room temperature.
• Inert gas atmosphere: argon flush.
• Minimal exposure to moist environments