This article discusses a novel approach by AstraZeneca to monitor primary DMPK assay performance & understand the inter and intra site assay variability.

An integral part of early drug discovery at AstraZeneca is the process known as the Design Make Test Analyse (DMTA) Cycle. DMTA relies heavily on chemical synthesis and primary screening being delivered within a quick turnaround time.

In 2012, AstraZeneca embarked on a strategic outsourcing strategy that resulted in the externalisation of 50% of discovery synthetic chemistry and the primary Drug Metabolism and Pharmacokinetics (DMPK) assays at the same partner.

The AstraZeneca DMPK primary screening (DMPK Wave 1) comprises five assays; Intrinsic Clearance in Human Liver Microsomes (HLM CLint), Intrinsic Clearance in Rat Hepatocytes (Rat Heps CLint), Human Plasma Protein Binding (Human PPB), Dried DMSO Solubility (Sol) and LogD at pH7.4 (LogD).

The data generated in these assays is key to much of the decision making undertaken in AstraZeneca Discovery Project teams during the DMTA cycle. In general, compounds are tested in DMPK Wave 1 assays at the location of synthesis, whether that is at an AstraZenca site or at the external partner. Doing so eliminates any lengthy shipping times during the time critical DMTA cycle.

Consequently, any given discovery drug project could be getting DMPK wave 1 data from one or more sites depending on the location of the chemistry synthesis. Therefore, it was critical to provide these discovery project teams with the confidence and ability to successfully compare internal and external data sets and thereby ensure that correct and timely decisions were made in the drug design process.

Two statistical tools were employed, firstly, the ‘Manhattan Tool’ which allows the scientist assay owners across each site a consistent way to easily visualise and monitor the performance of their control compounds and allows acceptance criteria to be set. Secondly, the ‘Minimum Discriminatory Difference/Ratio (MDD/MDR)’ which allows assay data to be compared from different locations and assists understanding and interpretation of the intra and inter site assay variability associated with each of these assays.

The tools described allow project teams to successfully compare internal and external data sets and thereby ensure that correct and timely decisions are made in the drug design process.

Manhattan Tool

The Manhattan Tool is a statistical tool developed by AstraZeneca which allows us to monitor the performance of the control compounds. It is a plot against the date an assay was run and identifies periods of stability using a method described by Woodward (1964) (2). The Manahattan tool gives the ability to set acceptance criteria for control compounds and to easily visualise when there is a statistical change in the performance of these compounds.

In the example shown in Figure 1, the log of the Verapamil CLint values (control compound used for the Rat Hepatocyte CLint assay) are plotted against time. The red line shows the Manhattan (sky) line, periods of constant CLint; there were three step changes observed, as indicated by the arrows A to C.

All three step changes correspond to changes to either assay conditions or assay data interpretation, eg a change in how CLint data is processed (A), switching the hepatocyte thawing method (B) and changing the batch of Rat Hepatocytes used (C). Identifying the cause of a step change can help improve the assay by understanding what contributes to its variability and also help to avoid these factors or minimise their impact for the future.

The initial upper and lower limits were set to between 2.04 and 2.23, the first step change A was small so it did not impact the acceptance criteria. However, the second step change B, was significant enough to merit a change in the acceptance criteria. In this example the acceptance criteria was changed to between 1.93 and 2.10. Similar to the initial step change, the last step change C was only minor and did not impact the acceptance criteria.

Measures for assay variability: MDD/MDR

The Minimum Discriminatory Difference or Ratio (MDD/MDR) (3) is a measure that allows an estimation as to whether the difference in the data generated for two compounds (each tested on one occasion only) is likely to be a real difference or not. The MDD/MDR defines the statistically significant difference/ratio threshold at the 95% confidence level.

The MDD reflects the minimum difference between two compounds of a parameter to be looked at on its natural raw scale, eg LogD. The MDR reflects the minimum ratio between two compounds of a parameter that is best looked at on the Log scale, eg HLM CLint, Rat Heps CLint, Sol and Human PPB.

It can be calculated based on the control compounds for between two sites and thus giving the inter site view of the assay, or for one site only giving the intra site view of the assay. We have taken the approach to calculate the MDD and MDR for each of the wave 1 assays, where control compounds data from a three-month interval are considered. In order to see the variation over time, a Rolling MDD/MDR plot and table is generated, doing the calculation every month on the previous three months of data, thus having a two-month overlap for each calculated value.

The intra and inter site variability has been produced for all five Wave 1 DMPK assays. The Rat Heps CLint Rolling MDR plots (Figure 2 and Figure 3) and corresponding summary table (Table 1) are shown below. The intra site plot (Figure 2) shows that before April 2013 the variability in the assay at AZ Site B was much higher than compared to the AZ Site A and External Partner (ExP) assays. However, since this time point the variation in all three assays have remained relatively constant. The inter site variation plot (Figure 3) shows that variation between sites was considerably reduced when the sites switched to a harmonised assay protocol (AZ Site A switched in January, while AZ Site B switched in April) as indicated by the arrows.

How representative are MDD/MDRs from control compounds for project compounds?

A question we have faced on several occasions is how representative are the control compounds in predicting the variability of the project compounds?

The control compounds are chosen because they behave well and predictably in the assays, and are also often not representative of current project chemical space; therefore can the MDD/MDRs based on the control compounds give AstraZeneca projects a true and accurate measure of the variation of project compounds in the assays?

To investigate this question, MDD/MDRs were generated for a small sample of representative project compounds and were compared with those generated from control compounds. On four different occasions, for Rat Heps CLint and HLM CLint, approximately 10 compounds from a previous run were repeated. Based on these four replicate results, the MDD/MDR for the project compounds were then calculated and compared with those calculated for the control compounds.

Table 2 shows the summary of these results for the AZ Site A and External Partner sites. The difference in MDD/MDR generated for control compounds and project compounds is small. Thus we could provide the AstraZeneca projects with confidence that the MDD/MDRs calculated from control compounds for the Rat Heps CLint and HLM CLint assays are an accurate measure of the variation of the project compounds.

How does an AstraZeneca project actually use the MDD/MDR values?

A dilemma an early drug discovery project team may find itself in is that they have three compounds that they would like to be able to differentiate between based on CLint values. AZ1 has been tested at External Partner producing a CLint of five, while AZ2 and AZ3 were tested at AZ Site A producing CLint of eight and 10 respectively. The compounds were tested in April 2014 so that MDD/MDR values from the last time period in Table 2 are applicable.

Since AZ1 was tested at our External Partner, it first needs to be converted to a theoretical AZ Site A CLint by multiplication with the conversion factor (5*1.01 = 5.05). Using the MDR for AZ Site A: External Partner (1.75) we can calculate the confidence interval around this theoretical AZ Site A CLint value for AZ1, when comparing with another compound at AZ Site A (2.89 to 8.84). AZ2, with a CLint value of eight is just within this confidence interval and therefore cannot be seen as differing from AZ1. However, AZ3 with a CLint value of 10 is outside this and therefore can be seen as having a higher CLint than AZ1 (Figure 4).

Conclusions

AstraZeneca, in conjunction with a strategic outsourcing partner, has successfully implemented a novel global approach that addresses the persistent problem of the comparison of data generated across multiple labs.

The monthly publication of MDD/MDR rolling plots provides a means to monitor both intra and inter site variability, enabling project teams to consider the assay variability in drug design work and when ranking compounds. Thus, project progression decisions can be made with higher confidence.

By aligning and making visible our quality control measures it has increased the confidence in getting data from an external organisation. In addition, we have shown that our control compounds provide a good representation of the variation observed for project compounds in the Rat Heps CLint and HLM CLint assays. DDW

—

This article originally featured in the DDW Fall 2014 Issue

—

Dr Claire Landqvist has worked with AstraZeneca since 2004 and is currently working as an Associate Principal Scientist in Externalisation for Drug Safety and Metabolism. Within this role she is serving as an alliance manager for two major relationships AstraZeneca has with Outsourcing Partners in the areas of DMPK and Secondary Pharmacology.

Brian Middleton MSc is an Associate Principal Statistician with 30 years’ experience of Pharmaceutical work, mostly in the non-clinical area.

Dr Barry Jones joined AstraZeneca in 2011. He currently holds the role of senior principal scientist in DMPK working on setting strategic direction, identifying new opportunities for research/development programmes and adding significant insight/expertise and value to drug discovery and development projects with a focus in the area of in vitro DMPK science.

Dr Charles O’Donnell is currently the Head of Externalisation for AstraZeneca ‘Drug Safety and Metabolism’. In this role he leads a group that is the central point of contact for AZ externalisation for preclinical ADME and toxicology in AstraZeneca.

References
1 Plowright, AT et al (2012). Hypothesis driven drug design: improving quality and effectiveness of the design-make-test-analyse cycle. Drug Discovery Today 2012 17:65-62.

2 Woodward, RH, Goldsmith, PL (1964). Cumulative Sum techniques. Oliver & Boyd.

3 Goedken, ER et al (2012). Minimum Significant Ratio of Selectivity Ratios (MSRSR) and Confidence in Ratio of Selectivity Ratios (CSRSR) : quantitative measures for selectivity ratios obtained by screening assays. J Biomol Screen 2012 17:857-867.