«

»

May
06

Why does this issue still give me nightmares?

A week or so ago I was seriously contemplating giving up blogging after just a so-so level of interest over 2 years and a couple of spikes that really only corresponded to issues around database quality which is not even the main focus of of my research. My blog is by far not the most important thing in my life but I was asking myself why I do it. Then one mention of a recent paper  on Derek Lowe’s “In the Pipeline” has ignited some interest in what I have written around the paper and gone to show that the blog can add something the paper cannot, a human factor, a sense of the frustration that goes into the scientific process. Ironically I have written about reviews on another paper in the past and had far less interest. I do not even think this paper is the most important thing I have ever done (e.g. I like this )– that is open for debate, but it has certainly been the cause for many sleepless nights (the night the paper appeared, last night etc. being pretty notable). Why you might ask?

Well lets begin with Antony Williams who gave a really nice back story as to our interest in data quality.

If it was not for Joe Olechno contacting us in early 2012 I would be none the wiser and the PLOS paper would not have appeared. While I had worked in HTS in the late 90s and early 2000’s I had pretty much left that screening world behind and not kept in touch with technological developments. The use of sound for moving liquids was as alien to me as the electric lightbulb must have been to those using gaslamps. I am sure this is also the case for many scientists still today.I was attracted to probing this further and seeing what pretty simple modeling could do. I did not expect what we found. And I certainly did not expect the responses from reviewers and journals that we had. I have had a much more balanced set of responses from audiences at an ACS local meeting in Raleigh last year, the SLAS meeting early this year and the recent ACS in New Orleans.

So lets think of all the places that dispense liquids to do analysis…Think about every hospital tox lab, think about those companies that analyze your biofluids for you, think about the contractors that look at the effect of compounds on the environment using animal or in vitro models (and the money that pours into this). It’s definitely not an issue just relevant to pharma, or med chemists (because lets face it there are far fewer of them). Just imagine do we think the labs in China, India etc. are dispensing with acoustic methods?

Do we think that all pharmas have shifted to acoustic? No what about those that use acoustic and tip-based dispensing together (why). An informal chat with an AZ employee mentioned their wholesale shift to acoustic dispensing 3-4 years ago. My guess is they kept it pretty quiet because it was acompetitive advantage, but then again it is likely all the top pharma’s did the same. Just goes to show the need for some precompetitive work in this area, or perhaps comparison of different dispensing techniques. And then there is my recent blog about the job opening at NIH regarding big data . Surely the kinds of analyses that we have done should be a requirement of the NIH. I am not aware that they have published on different dispensing techniques. I certainly have not seen meta data in PubChem or any other database that tells you what technique was used for dispensing.

So when PLOS tells us that our press release has statements regarding my nightmares that are “over-reaching in their representation of the study results.” and reiterate this via Twitter.. Well fine, they are my nightmares, they are very real and until someone clears up what is going on they are going to likely continue. Do we waste more time and money ignoring the issue of which technique to use for dispensing or does someone do the experiment (and fund it)?

Here are some of the responses to those comments on the  “in the pipeline blog”

8. Computationally Entertained on May 3, 2013 10:58 AM writes…

It reminds me of the complaints I had for a recent paper titled “do medicinal chemists learn from activity cliffs? …”

http://pubs.acs.org/doi/abs/10.1021/jm400147j

 

SE – This is interesting – could mode of liquid transfer suggest activity cliffs that are spurious?

 

In response to

9. Stephan on May 3, 2013 11:06 AM writes…

I think (hope) everybody in assay development is familiar with the sticky wall and volatile plate issues.
Our production folks certainly are. If I ask them to package 100µl of 1µg/ml (there aren’t any moles in their lab) detector antibody they ask if that is for immediate use (fine) use any time after after 2 weeks @ 4°C (fine, they will adjust the conc.), or to be used anywhere between 12h and 14 days after bottling (good luck, you are on your own).

 

SE – Clearly not a lot of people are aware of issues with compound dilution, stickiness, leaching or otherwise represented in the early references cited in the PLOS paper. Just using Google Scholar you can see the number of citations is very low and the much larger number of citations for the Science paper – likely due to higher exposure than the other journals? I think we were justified in trying to bring the issue into the open.

Ref 1

Pharmacological evaluation of different compound dilution and transfer paradigms on an enzyme assay in low volume 384-well format

T Spicer, Y Fitzgerald, N Burford, S Matson… – Drug Discovery …, 2005 – picoliterinc.com

Page 1. Pharmacological Evaluation of Different Compound Dilution and Transfer
Paradigms on an Enzyme Assay in Low Volume 384-well Format. Tim Spicer, Yvonne
Fitzgerald, Neil Burford*, Sandra Matson, Moneesh Chatterjee

Cited by 8

Ref 2

CITATION][C] Assay sciences: A model for improving efficiency through centralization

J Wingfield, D Jones, R Clark, P Simpson – American Drug Discovery, 2008

Cited by 4

Ref 3

Achieving accurate compound concentration in cellbased screening: validation of acoustic droplet ejection technology

RJ Grant, K Roberts, C Pointon… – Journal of …, 2009 – jbx.sagepub.com

Abstract Compound handling is a fundamental and critical step in compound screening
throughout the drug discovery process. Although most compound-handling processes within
compound management facilities use 100% DMSO solvent, conventional methods of

Cited by 14

Ref 4

Best practices in compound management for preserving compound integrity and accurately providing samples for assays

SL Matson, M Chatterjee, DA Stock… – Journal of …, 2009 – jbx.sagepub.com

Abstract Preserving the integrity of the compound collection and providing high-quality
materials for drug discovery in an efficient and cost-effective manner are 2 major challenges
faced by compound management (CM) at Bristol-Myers Squibb (BMS). The demands on

Cited by 13

Ref 5

Gradient, contactfree volume transfers minimize compound loss in dose-response experiments

D Harris, J Olechno, S Datwani… – Journal of biomolecular …, 2010 – jbx.sagepub.com

Abstract More accurate dose-response curves can be constructed by eliminating aqueous
serial dilution of compounds. Traditional serial dilutions that use aqueous diluents can result
in errors in dose-response values of up to 4 orders of magnitude for a significant

Cited by 6

Ref 6

Bioactive contaminants leach from disposable laboratory plasticware

GR McDonald, AL Hudson, SMJ Dunn, H You… – Science, 2008 – sciencemag.org

Abstract Disposable plasticware such as test tubes, pipette tips, and multiwell assay or
culture plates are used routinely in most biological research laboratories. Manufacturing of
plastics requires the inclusion of numerous chemicals to enhance stability, durability, and

Cited by 82

Ref 7

Nonylphenol ethoxylate plastic additives inhibit mitochondrial respiratory chain complex I

C Belaiche, A Holt, A Saada – Clinical chemistry, 2009 – Am Assoc Clin Chem

Am J Physiol 1998;274:1127-1139. ↵ McDonald GR, Hudson AL, Dunn SM, You
H, Baker GB, Whittal RM, et al. Bioactive contaminants leach from disposable
laboratory plasticware. Science (Wash DC) 2008;322:917. Abstract

Cited by 7

 

In response to

12. darwinsdog on May 3, 2013 12:04 PM writes…

In additional to adhesion to plastic surfaces, I would wonder about aggregation of cmpds (think Brian Shoichet) or other physical effects related to the properties of each cmpd that manifest by diff. dispensation methods.

SE – I am aware of this work and I think its worth testing as well – now if we could get our hands on the compunds..We tried to engage AZ at the outset but no joy.

 

21. darwinsdog on May 3, 2013 1:35 PM writes…

@16,19 First of all the paper in question is a meta analysis of data from other papers. So take it for what it is worth and realize that a lot more may be a variable here than just the dispensation method alone (usual list when changing assay protocols all the way to material lots over time) but it makes for a point to blog about and count as a PLOS publication (for whatever that gets you). But back to the detergent control – I do not know if the original papers included detergent controls in an attempt to address aggregation but I can tell you in general pharma companies (at least the ones I know about first hand) don’t usually dedicate wells in a plate to this type of control (I guess the cmpd libraries are deemed to sufficiently cover the ‘palmolive-space’ :-).

SE – Yes it is a meta analysis of data from same lab AZ in same patents..Actually just blogging about this was not considered because so few people read the blog, it had to go somewhere that was accessible. Hence the first attempts being science. We think part of the problem is the earlier papers on the tip vs acoustic differences were just not in widely read places.If you ask 99% of biologists and chemists (and probably physicists) they would have no idea of this issue..

As for PLOS, while it is not the same caliber as Science – it is very visible, it is free to the reader, the review process was fast and more reasonable than other journals tried (Science & Anal Chem).

24. cdsouthan on May 3, 2013 2:53 PM writes…

Time for a CASFEAR competition “Critcal Assesment For Enzyme Assay Reproducability”

Just send a few reference compounds (with LogP spread) out to 20 to 50 labs and ask for the IC50s and Ki’s back with some linearity curves thrown in. Then write the paper.

SE – -Chris – good idea – though just a comparison across dispensing techniques with a big set of compounds with structures that are accessible and open data would suffice to start with..

25. Hap on May 3, 2013 3:28 PM writes…

I can see why it wasn’t in Science, but I wouldn’t have agreed that it belonged in a medicinal chemistry journal.

How do journals get all the pretty biological data they show? In some cases (maybe a lot), there aren’t exogenous components in the assays or their standards and so aggregation/other issues during dispensing aren’t issues, but in the remaining cases components which may not behave correctly are added exogenously. Without further data, you can’t assume that all compounds behave like this (maybe the set of compounds used was funny in some way that led to the observed behavior), but I’d not be betting the house, its furniture, or even the toys that this doesn’t happen elsewhere. Doesn’t that possibility make an awful lot of biological data look less secure? If it does, that would be important to a lot of people.

SE – Perhaps you have a point, except the leaching work appeared in Science and this issue is on a par with that IMHO. The earlier work by AZ and BMS put out in posters and talks shows much bigger numbers of compounds as described in the paper and shown on Derek’s blog. They show the same issue so it is not just one lab, one target, or one set of hairy compounds.

 

30. Jose on May 3, 2013 10:40 PM writes…

I think it’s an important finding, but did they seriously think it was Science worthy enough to submit not once, but TWICE?

Thanks Jose. We addressed the editor comments and added text to broaden the relevance – because lets face it – it is not just med chemists or those in pharma that rely on dilution with tips (see comments above).. and tried again, and failed. Probably should have tried a third time?

1 comment

No ping yet

  1. sean says:

    I have just been alerted by the authors of another paper (which we were unaware of) that may be relevant – looks like this overlapped with our submission.

    “Comparison of Compound Administration Methods in Biochemical Assays Effects on Apparent Compound Potency Using Either Assay-Ready Compound Plates or Pin Tool–Delivered Compounds”, Journal of Biomolecular Screening, January 2013 vol. 18 no. 1, 14-25.

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>