My Theranos Story

I'm listening to the book "Bad Blood" by Carreyrou.  It is a fun read, and I think it is very valuable for me.  I'm a big cheerleader of Silicon Valley; a strong proponent of the risk-taking culture found here.  And I also follow a lot of medical research (obviously).  Therefore it is even more important for me to be reminded of the weaknesses both of Silicon Valley specifically and medical research in general.  This book is all about those weaknesses.

My Theranos Story

I'm on a mailing list of parents of kids who have type-1 diabetes.  It is a great resource, and I recommend everyone connected to type-1 diabetes join some sort of support and information sharing group.  In Dec 2013 there was a posting about a Silicon Valley blood testing start up called Theranos, that was going to make cheap, quick blood tests available which used only a few drops of blood.  Their first site was a Walgreens in Palo Alto.  They had a price list online (as I remember) and C-peptide tests were just a couple of bucks.  Less than the cost of a sandwich.  I had this idea to invite a bunch of parents and children from the email group to meet at the Walgreens one weekend morning.  We could all get tested, get the results, and maybe afterwards go out for ice cream later and talk about type-1 diabetes.  Since the parents would mostly have normal C-peptide numbers, and the kids (with type-1) would have nearly zero C-peptide numbers, it would immediately be obvious if the test worked or not.

However, based on many years of blogging about research, before I suggested this to the group, I figured I'd look up the research on their new testing machine.  So I did some of my standard queries on Pubmed (a US government database of publications), and scholar.google.com (Google's big research database), and clinicaltrials.gov (the FDA's clinical trials registry), and a few other places.  Nothing.  Not one paper.  I was really shocked.  Normally, you can't sell a treatment until it has gone through at least four human trials, so I was really surprised that I could find nothing for this new testing system.  (I did not know at the time that tests were regulated by a completely different US government agency, and in a completely different way, than treatments.)

Anyway, desperate for information, I looked online for the company's organization and basic balance sheet.  At that time, I had worked for Silicon Valley start ups for over 20 years, and I knew how to read a balance sheet, and how stock worked.  But there was none of that.  The only information that I found was Theranos's board of directors.  My years of working for start ups had given me experience with many different boards of directors, so I knew what to expect.  Or I thought I did.

The first name was George Schultz.  A guy famous for lying, and for being a Secretary of State (ie. foreign relations).  The next name I noticed was Henry Kissinger.  Another guy famous for lying, and foreign affairs.  [Note: I'm telling this story from my point of view.  I realize that other people may consider these two famous for other things.]  There was some long time senator (Sam Nunn, I think), and a big time general (Mattis?).  I was flabbergasted.  Why were these guys on the board of directors for a medical device company?  Where was the medical expertise?  Where were the expert venture capitalists (VCs) who funded medical devices?  Where were the people who had actually run a company before?  It was like a Saturday Night Live skit.

I thought about this for a while.  Should I judge a company based on its board of directors?  Maybe there was some reason all these guys were there.  Shouldn't I be deferential to all these famous, important people?  Maybe I could imagine some good reason for this.  But then I caught myself.  The only reason I was even looking at their board of directors was because there was no actual/published clinical trials showing the testing system worked.  There is no excuse for that.  If a treatment for type-1 diabetes had zero clinical trials, I would ignore it.  Well, I would wait until they started a clinical trial, and then evaluate it based on the results.  I should have the same standard for testing companies;  I would not evaluate their board of directors.

So I dropped the idea of getting a crew together to get tested by Theranos.  I decided to wait until they published results showing their tests worked.  They never did.  Years later Carreyrou published a series of newspaper articles describing what a fraudulent house of cards it was.

Learning From Theranos

After reading the book, and reflecting on my own experience with Theranos, a question popped into my head: why was it that in a couple of hours on a weekend morning, with no special insider data, and making a minor decision about how to spend a morning, I decided Theranos was not worth even a little time.  Compared to the venture capitalists (VCs) who spent weeks researching Theranos, could ask for insider information, had direct contact with the CEO/founder of the company, and were making million dollar decisions, and yet they decided Theranos was worth it.

Was I just lucky? Maybe.  But I do think I was helped by two things: (1) when I was making the decision, and (2) that I did not have direct contact with Theranos.

When Decisions Get Made
I had a big advantage over most of the investors, in that I was making my decision relatively late in the game, and therefore it was completely reasonable for me to expect public results.  When those were not available, I realized there was a problem.  Most of the investors made their decisions much earlier in the process, when less data was expected to be available.  Therefore, the lack of data did not make them suspicious.  How do we avoid that problem?   In two ways:

First, be willing to delay your judgement on new research.  The VCs wanted to invest early because they would make more money that way.  We need to invest early to nurture early stage research, but we don't need to decide that one particular line of research is going to be the cure.  That decision, the decision to emotionally commit to the research (to be a cheerleader), we can hold off on.  And we should.  If you think something might be the cure, you can always donate some money initially, but wait a few years, and check on their progress, to see if it really is the cure.  (Type 1 is forever, so waiting a few years is reasonable.)

Second, be willing to change your mind as the situation changes.  Most people have a hard time changing their minds, and this was on display during the whole Theranos saga.  Several VCs who had decided the company was going to revolutionize testing in 2006 still believed it in 2016, by which time they should have known better.  A good scientist changes their mind when new data is available, but that is the easier part.  The harder part is to change your mind when no new data is available, but forward progress should have resulted in new data, but hasn't.  The hard part is to realize when no new data is (in fact) bad data.

Answer The Right Question
The right question is: what is the data available that shows this is working (compared to the data that should be available).  The answer to that question was obvious when I did my web searches.  However, the questions that most of the investors were asking were different: Is what Elizabeth Holmes says is going to happen, really going to happen?  Is she going to revolutionize blood testing?  Is she telling the truth?  Those questions are much harder to answer, much more emotional, much more personal, than the question that I tried to answer.

Asking the right questions is harder than it sounds.  Humans have been making decisions about people and trustworthiness for over 100,000 years.  Our brains are hardwired to do that, because we lived and worked in small groups who knew each other well.  We worked with them every day and for our whole lives.  The scientific method has existed for only a few 100 years.  It is not the go-to method our brain naturally uses.  But the world has changed so that we are now making decisions about people who we fundamentally do not know, not in the way our brains evolved.

I do think that every one of us, when presented with new research, can ask the right question.  The moment we even start to think about "is this person right" or "are they trustworthy" or "are they going to change the world", to just immediately de-personalize the question: do they have the data that they should have at this point in the research?  Ignore excuses; ask about data.

Finally

These are a little off the beaten path, but well worth a read:
https://mondaynote.com/theranos-could-have-been-stopped-9670793e3431
https://www.stanforddaily.com/2019/01/15/theranos-whistleblowers-reflect-on-failure-of-the-silicon-valley-unicorn/
https://www.entrepreneur.com/article/311036
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3275363/
(On March 16 I asked for this paper to be retracted. We'll see how that goes.)

A reminder of how this fraud was presented in the news prior to the "Bad Blood" reporting:
https://www.wired.com/2014/02/elizabeth-holmes-theranos/
http://fortune.com/2014/06/12/theranos-blood-holmes/



Joshua Levy
https://ift.tt/29DuN3o 
publicjoshualevy at gmail dot com
All the views expressed here are those of Joshua Levy, and nothing here is official JDRF, JDCA, or Bigfoot Biomedical news, views, policies or opinions. In my day job, I work in software for Bigfoot Biomedical. My daughter has type-1 diabetes and participates in clinical trials, which might be discussed here. My blog contains a more complete non-conflict of interest statement. Thanks to everyone who helps with the blog.


via Cure Research