Section2:Bridging Studies for Assay Upgrades and Minor Changes

From Assay Guidance Wiki

Sections C and D cover the validation of entirely new assays, or assays that are intended to replace existing assays. The replacement assays are “different” from the original assay, either because of facility changes, personnel differences, or substantively different detection and automation equipment. Assay upgrades and changes occur as a natural part of the assay life cycle. Requiring a full validation for every conceivable change is impractical and would serve as a barrier to implementing assay improvements. Hence full validation following every assay change is not recommended. Instead bridging studies or “mini-validation” studies are recommended to document that the change does not degrade the quality of the data generated by the new assay.

The level of validation recommended has 3 tiers, from a small plate uniformity study (Tier I), to just the assay comparison portion of the Replicate-Experiment study (Tier II) to the full validation package of Sections C and D (Tier III). Examples of changes within each Tier are given below, along with the recommended validation study for that tier. Note that if the study indicates the change will have an adverse impact on assay quality (i.e. the study indicates there are problems), then the cause should be investigated and a full (Tier III) validation should be done. If the results from that study indicate the assays are not equivalent, but the new assay has to be implemented, then a the results should not be combined into one set.

The following applies principally to changes in biological components of the protocol. If changes are made to the data analysis protocol then these can ordinarily be validated without generating any new data, by comparing the results using the original and new data analysis protocols on a set of existing data. Discuss any changes with a statistician. If changes are made to both the data analysis and biological components of the protocol then the appropriate tier should be selected according to the severity of the biological change as discussed below. The data analysis changes should be validated on the new validation data and any additional validation work may be needed as judged by the statistician.

Tier I: Single Step Changes to the Assay

Tier I modifications are single changes in an assay such as a change to a reagent, instrumentation, or assay condition that is made either to improve the assay quality or increase the capacity without changing the assay quality. The changes can also be made for reasons unrelated to assay throughput or performance (e.g. change of a supplier for cost savings). Examples of such change are:

Changes in detection instruments with similar or comparable optics and electronics. E.g.: plate readers, counting equipment, spectrophotometers. A performance check for signal dynamic range, and signal stability is recommended prior to switching instruments.
Changes in liquid handling equipment with similar or comparable volume dispensing capabilities. Volume calibration of the new instrument is recommended prior to switching instruments. [Note that plate and pipette tip materials can cause significant changes in derived results (IC50, EC50). This may be due to changes in the adsorption and wetting properties of the plastic material employed by vendors. Under these conditions a full validation may be required].

The purpose of the validation study is to document the change does not reduce the assay quality.

Protocol

Conduct a 4 plate Plate Uniformity Study using the layouts in the “2 Plates per Day” tab of the Plate Uniformity Template (the layouts are the same as Plates 1 and 2 of Section C.2. Plates 1 and 2 should be done using the existing protocol, and Plates 3 and 4 done using the new protocol on the same day using the same reagents and materials (except for the intentional change). Use the 2 Day / 2 Plates per Day template to conduct the analysis.

Analysis

The main analysis is a visual inspection of the “all plates” plots to ensure that the signals have not changed in either in magnitude and/or variability. The mean and SD calculations for each plate can help, but visual inspection is usually sufficient.

Example

An assay was changed by replacing a manual pipetting step with a multidrop instrument. A 4-plate Plate Uniformity study was run as per the protocol, with the manual pipetting done in plates 1 and 2, and the multidrop in plates 3 and 4. The results show that the mean percent activity is the same, and the multidrop’s varability superior (i.e. lower) to the manual pipetting.

Tier I Validation study comparing manual pipetting (plates 1 and 2) versus Multidrop pipetting (plates 3 and 4) in GTPγS assay

Tier II: Minor Assay Changes

Tier II changes are more substantive than Tier I changes, and have greater potential to directly impact EC50/IC50 results. Examples of such changes are:

Changes in dilution protocols covering the same concentration range for the concentration–response curves. A bridging study is recommended when dilution protocol changes are required.
Lot changes of critical reagents such as a new lot of receptor membranes or a new lot of serum antibodies.
Assay moved to a new laboratory without major changes in instrumentation, using the same reagent lots, same operators and assay protocols.
Assay transfer to an associate or technician within the same laboratory having substantial experience in the assay platform, biology and pharmacology. No other changes are made to the assay.

Protocol and Analysis

Conduct the assay comparison portion of the Replicate Experiment Study discussed in Section D, i.e. compare one run of 20-30 compounds of the assay using the existing assay to one run of the assay under the proposed format and compare the results. If the compound set used in the original validation is available then one need to only run the set again in the new assay protocol, and compare back to Run 1 of the original Replicate-Experiment Study. The acceptance criterion is the same as for the assay comparison study: Both Limits of Agreement should be between 1/3 and 3.0.

Tier III: Substantive Changes

Substantive changes requiring full assay validation: When substantive changes are made in the assay procedures, measured signal responses, target pharmacology and control compound activity values may change significantly. Under these circumstances, the assay should be re-validated according to methods described in Sections IIC and IID. The following changes constitute substantive changes, particularly when multiple changes in factors listed below are involved:

Changes in assay platform: e.g.: Filter binding to Fluorescence polarization for kinase assays.
Changes in assay reagents (including lot changes and supplier) that produce significant changes in assay response, pharmacology and control activity values. For example, changes in enzyme substrates, isozymes, cell-lines, label types, control compounds, calibration standards, (radiolabel vs. fluorescent label), plates, tips and bead types, major changes in buffer composition and pH, co-factors, metal ions, etc.
Transfer of the assay to a different laboratory location, with distinctly different instrumentation, QB practices or training.
Changes in detection instruments with significant difference in the optics and electronics. For example, plate readers, counting equipment, spectrophotometers.
Changes in liquid handling equipment with significant differences in volume dispensing capabilities.
Changes in liquid handling protocol with significant differences in volume dispensing methods.
Changes in assay conditions such as shaking, incubation time, or temperature that produce significant change in assay response, pharmacology and control activity values.
Major changes in dilution protocols involving mixed solvents, number of dilution steps and changes in concentration range for the concentration-response curves.
Change in analyst/operator running the assay, particularly if new to the job and/or has no experience in running the assay in its current format/assay platform.
Making more than one of the above-mentioned changes to the assay protocol at any one time.

Substantive changes require full validation, i.e. a three day Plate Uniformity Study and Replicate Experiment Study. If the intent is to report the data together with the previous assay data then an assay comparison study should be conducted as part of the Replicate Experiment study.