• 08 JUN 12
    • 9

    Why big data for healthcare is rubbish

    The Mckinsey Global Institute recently published a report entitled – Big data: The next frontier for innovation, competition, and productivity .

    We will show how the Mckinsey Global Institute report on big data is no more than a lengthy essay in fallacies, inflated hyperbole, faulty assumptions, lacking in evidence for its claims and ignoring the two most important stakeholders of healthcare – namely doctors and patients.

    IT System efficiency does not improve patient health

    According to the report:

    In health care, big data can boost efficiency by reducing systemwide costs linked to undertreatment and overtreatment and by reducing errors and duplication in treatment. These levers will also improve the quality of care and patient outcomes.

    To calculate the impact of big-data-enabled levers on productivity, we assumed that the majority of the quantifiable impact would be on reducing inputs.

    We held outputs constant—i.e., assuming the same level of health care quality. We know that this assumption will underestimate the impact as many of our big-data-enabled levers are likely to improve the quality of health by, for instance, ensuring that new drugs come to the market faster…

    They don’t know that.

    The MGI report does not offer any correlation between reduction in systemwide costs and improving the quality of care of the individual patient.

    The report deals with the macroeconomics of the pharmaceutical and healthcare organization industries.

    In order to illustrate why systemwide costs are not an important factor in the last mile of healthcare delivery, let’s consider the ratio of system overhead to primary care teams in Kaiser-Permanente – one of the largest US HMOs. At KP, (according to their 2010 annual report) – out of 167,000 employees, there were 16,000 doctors, and 47,000 nurses.

    Primary care teams account for only 20 percent of KP head-count. Arguably, big-data analytics might enable KP management to deploy services in more effective way but do virtually nothing for the 20 percent headcount that actually encounter patients on a day to day basis.

    Let’s not improve health, let’s make it cheaper to keep a lot of people sick

    Note the sentence – “assuming the same level of health care quality”. In other words, we don’t want to improve health, we want to reduce the costs of treating obese people who eat junk food and ride in cars instead of walking instead of fixing the root causes. Indeed MGI states later in the their report:

    Some actions that can help stem the rising costs of US health care while improving its quality don’t necessarily require big data. These include, for example, tackling major underlying issues such as the high incidence and costs of lifestyle and behavior-induced disease.

    Lets talk pie in the sky about big data and ignore costs and ROI

    …the use of large datasets has the potential to play a major role in more effective and cost-saving care initiatives, the emergence of better products and services, and the creation of new business models in health care and its associated industries.

    Being a consulting firm, MGI stays firmly seated on the fence and only commits itself to fluffy generalities about the potential to save costs with big data.

    The terms ROI or return on investment is not mentioned even once because it would ruin their argumentation. As a colleague in the IT division of the Hadassah Medical Organization in Jerusalem told me recently:

    “Hadassah management has no idea of how much storing all that vital signs from smart phones will cost. As a matter of fact, we can’t even afford the infrastructure to store big data”.

    It’s safe to wave a lot of high rhetoric around about $300BN value-creation (whatever that means), when you don’t have to justify a return on investment or ask grass-level stakeholders if the research is crap.

    MGI does not explain how that potential might be realized. It sidesteps a discussion of the costs of storing and analyzing big data, never asks if big data helps doctors make better decisions and it glosses over low-cost alternatives related to educating Americans on eating healthy food and walking instead of driving.

    The absurdity of automated analysis

    ..we included savings from reducing overtreatment (and undertreatment) in cases where analysis of clinical data contained in electronic medical records was able to determine optimal medical care.

    MGI makes an absurd assumption that automated analysis of clinical data contained in electronic medical records can determine optimal medical care.

    This reminds me of a desert island joke.

    A physicist and economist were washed up on a desert island. They have a nice supply of canned goods but no can-opener. To no avail, the physicist experiments with throwing the cans from a high place in the hope that they will break open (they don’t).

    The economist tells his friend “Why waste your time looking for a practical solution, let’s just assume that we have a can-opener!”.

    The MGI report just assumes that we have a big data can-opener and that big data can be analyzed to optimize medical care (by the way, they do not even attempt to offer any quantitative indicators for healthcare optimization – like reducing the number of women that come down with lymphedema after treatment for breast cancer – and lymphedema is a pandemic in Westerm countries, affecting about 140 million people worldwide.

    In Western countries, secondary lymphedema is most commonly due to cancer treatment.Between 38 and 89% of breast cancer patients suffer from lymphedema due to auxiliary lymph node dissection and/or radiation.

    More is not better

    We found very significant potential to create value in developed markets by applying big data levers in health care. CER (Comparative effectiveness research ) and CDS (Clinical decision support) were identified as key levers and can be valued based on different implementations and timelines

    Examples include joining different data pools as we might see at financial services companies that want to combine online financial transaction data, the behavior of customers in branches, data from partners such as insurance companies, and retail purchase history. Also, many levers require a tremendous scale of data (e.g., merging patient records across multiple providers), which can put unique demands upon technology infrastructures. To provide a framework under which to develop and manage the many interlocking technology components necessary to successfully execute big data levers, each organization will need to craft and execute a robust enterprise data strategy.

    The American Recovery and Reinvestment Act of 2009 provided some $20 billion to health providers and their support sectors to invest in electronic record systems and health information exchanges to create the scale of clinical data needed for many of the health care big data levers to work.

    Why McKinsey is dead wrong about the efficacy of analyzing big EHR data

    The notion that more data is better (the approach taken by Google Health and Microsoft and endorsed by the Obama administration and blindly adopted by MGI in their report is patently wrong for 2 reasons:

    1. EHR is based on textual/coded data of visits, and is not organized around patient clinical issue.
    2. Meaningful machine analysis of EHR is impossible

    Current EHR systems store large volumes of data about diseases and symptoms in unstructured text, codified using systems like SNOMED-CT. Codification is intended to enable machine-readability and analysis of records and serve as a standard for system interoperability.

    Even if the data was perfectly codified, it is impossible to achieve meaningful machine diagnosis of medical interview data that was uncertain to begin with and not collected and validated using evidence-based methods.

    More data is less valuable for a basic reason

    A fundamental observation about utility functions is that their shape is typically concave: Increments of magnitude yield successively smaller increments of subjective value.

    In prospect theory, concavity is attributed to the notion of diminishing sensitivity, according to which the more units of a stimulus one is exposed to, the less one is sensitive to additional units.

    Under conditions of uncertainty in a medical diagnosis process, as long as it is relevant, less information enables taking a better and faster decision, since less data processing is required by the human brain.

    Unstructured EHR data is not organized around patient issue

    When a doctor examines and treats a patient, he thinks in terms of “issues”, and the result of that thinking manifests itself in planning, tests, therapies, and follow-up.

    In current EHR systems, when a doctor records the encounter, he records planning, tests, therapies, and follow-up, but not under a main “issue” entity; since there is no place for it.

    The next doctor that sees the patient needs to read about the planning, tests, therapies, and follow-up and then mentally reverse-engineer the process to arrive at which issue is ongoing. Again, he manages the patient according to that issue, and records everything as unstructured text unrelated to issue itself.

    Other actors such as national registers, extraction of epidemiological data, and all the others, all go through the same process. They all have their own methods of churning through planning, tests, therapies, and follow-up, to reverse-engineer the data in order to arrive at what the issue is, only to discard it again.

    The “reverse-engineering” problem is the root cause for a series of additional problems:

    1. Lack of overview of the patient
    2. No connection to clinical guidelines, no indication of which guidelines to follow or which have been followed
    3. No connection between prescriptions and diseases, except circumstantial
    4. No ability to detect and warn for contraindications
    5. No archiving or demoting of less important and solved problems
    6. Lack of overview of status of the patient, only a series of historical observations
    7. In most systems, no search capabilities of any kind
    8. An excess of textual data that cannot possibly be read by every doctor at every encounter
    9. Confidentiality borders are very hard to define
    10. Very rigid and closed interfaces, making extension with custom functionality very hard

    Summary

    MGI states that their work is independent and has not been commissioned or sponsored in any way by any business, government, or other institution.

    True, but MGI does have consulting gigs with IBM and HP that have vested interests in selling technology and services for big data.

    The analogies used in the MGI report and their tacit assumptions probably work for retail in understanding sales trends of hemlines and high heels but they have very little to do with improving health, increasing patient trust and reducing doctor stress.

    The study does not cite a single interview with a primary care physician or even a CEO of a healthcare organization that might support or validate their theories about big data value for healthcare. This is shoddy research, no matter how well packaged.

    The MGI study makes cynical use of “framing” in order to influence the readers’ perception of the importance of their research. By citing a large number like $300BN readers assume that impact of big data is well, big. They don’t pay attention to the other stuff – like “well it’s only a potential savings” or “we never considered if primary care teams might benefit from big data (they don’t).

    At the end of the day, $300BN in value from big data healthcare is no more than a round number. What we need is less data and more meaningful relationships with our primary care teams.

    facebooktwittergoogle_plusredditpinterestlinkedinmailby featherLeave a reply →
  • Posted by Martin Wehlou on June 12, 2012, 7:42 pm

    Danny,

    I think the problem with big data, and even much of medium sized data, is much simpler. All this data is by its very nature retrospective data, which is of very low evidence value. Using retrospective data, or “experience”, is fraught with dangers, misleading us to draw the wrong conclusions about what is cause and effect. For this very reason, we replaced retrospective data with prospective, controlled, data when we dumped anecdotal medicine and replaced it with Evidence Based Medicine a few decades back. US Preventive Services Task Force regards retrospective evidence as Level III, i.e. the worst possible. UK National Health Service regards this evidence as Level C or D on a scale of A through D.

    In other words, big data is based on crappy data and will lead to crappy conclusions. We really don’t need that.

    Reply →
  • Posted by Robert Latino on June 18, 2012, 6:23 pm

    In 1996 The Joint Commission (TJC) issued the guidelines for hospitals to conduct Root Cause Analyses (RCA) and Failure Modes and Effects Analyses (FMEA) under specified conditions. They cite the requirment of the analyses to be ‘credible and thorough’ despite having sub-standards systems in place to assure these admirable goals were met.

    Despite having these TJC requirments in place over the past 16 years, there are no quantifiable studies that I am aware of that demonstrate a direct correlation between accredited RCA/FMEA efforts and improved patient safety.

    In my travels as an RCA/Reliability Expert over the past 27 years, I find that the average cost to conduct an RCA (even though they are really often performed as ‘shallow cause analyses’) is around $25,000 each when you take into account labor, consultants, lawyers, data collection, etcl).

    Now we have to ask ourselves are we getting our money’s worth for going through the motions of doing these analyses to meet minimal compliance requirements? Who is asking the right questions such as:

    1. Why isn’t our patient safety improving when we do so many RCAs/FMEAs?
    2. Why do we seem to do an RCA over and over again on the same issue (perhaps because the initial RCA’s were not ‘credible and thorough’ but obviously compliant)?
    3. What is our ROI for doing such analyses?
    4. Who is responsible and accountable for the ‘effectiveness’ of our RCA’s/FMEA’s?

    This underlying premise that regulation alone will improve patient safety because the government says so is flawed just like this report. Someone needs to do a comprehesive RCA on why people treat such hearsay as fact. Would such a report hold up in court under those rules of evidence?

    Bob Latino
    CEO
    Reliability Center, Inc.

    Reply →
    • Posted by Danny on June 18, 2012, 11:46 pm
      in reply to Robert Latino

      Bob

      This is correct.

      and thanks for the input – this is an important area that I’m not familiar at all with. Most of our work is in the patient privacy and data security space.

      I doubt that a report and why people treat hearsay as fact would not hold up under rules of evidence but I also doubt that a court would actually entertain a motion

      The best we can do is for people like you to continue to promote the message of patient safety for it’s own sake

      We’d love to host you on the pathcare.co – if you want to shoot over articles please go ahead – we’re with you on the crusade.

      Thanks again
      Danny

      Reply →
  • […] http://pathcare.co/why-big-data-for-healthcare-is-rubbish/?goback=%2Eanb_1839273_*2_*1_*1_*1_*1… takes direct aim at a recent report by the McKinsey Global Institute (Big Data: The Next Frontier for Innovation, Competition, and Productivity) http://www.mckinsey.com/insights/mgi/research/technology_and_innovation/big_data_the_next_frontier_for_innovation that projects substantial quantitative and qualitative benefits from implementing Big Data initiatives in health care.  Pathcare essentially states that McKinsey and Big Data ignore the two major stakeholders in healthcare – doctors and patients: “The study does not cite a single interview with a primary care physician or even a CEO of a healthcare organization that might support or validate their theories about big data value for healthcare. This is shoddy research, no matter how well packaged.” http://pathcare.co/why-big-data-for-healthcare-is-rubbish/?goback=%2Eanb_1839273_*2_*1_*1_*1_*1_*1 […]

    Reply →
  • Posted by BobbyG on September 11, 2012, 7:21 pm

    “a lengthy essay in fallacies, inflated hyperbole, faulty assumptions, lacking in evidence for its claims and ignoring the two most important stakeholders of healthcare – namely doctors and patients.”
    ___

    Ouch. I will have to take some time ASAP to review and understand your argument. I will likely cite you on my REC blog, and refer you to the hardcores at sciencebasedmedicine.org. I like combative people who don’t mince words, as long as their arguments are valid and sound.

    I liked this comment:

    “1. Why isn’t our patient safety improving when we do so many RCAs/FMEAs?
    2. Why do we seem to do an RCA over and over again on the same issue (perhaps because the initial RCA’s were not ‘credible and thorough’ but obviously compliant)?
    3. What is our ROI for doing such analyses?
    4. Who is responsible and accountable for the ‘effectiveness’ of our RCA’s/FMEA’s?

    This underlying premise that regulation alone will improve patient safety because the government says so is flawed just like this report. Someone needs to do a comprehesive [sic] RCA on why people treat such hearsay as fact. Would such a report hold up in court under those rules of evidence?”
    ___

    I would also commend to you the excellent work “Medicine in Denial” by Lawrence Weed MD and Lincoln Weed JD, as well as the ThedaCare works of John Toussaint, MD. et al.

    I’ve quickly surfed around your site trying to ascertain a business model here. It doesn’t just jump right out at me.

    Moreover, if you are trafficking in in ePHI between patients and clinicians, you are subject to the HIPAA Security Rule, 45 CFR 164.3xx and everything it entails — not to mention 164.5xx (Privacy Rule) 164.4xx (Breach Notification).

    “We secure and periodically audit Repnets services using HIPAA compliance security rule guidelines in technical, administrative and physical areas.”

    I assume you could pass a federal audit in that regard.

    Reply →
    • Posted by Danny on September 11, 2012, 8:27 pm
      in reply to BobbyG

      Bobby

      Thanks for the feedback and kind words. I will definitely look for Medicine in Denial and read your other recommended sources.

      I’m not surprised the business model for Pathcare doesn’t just jump right out. We sort of did that deliberately because we are pre-launch.

      Pathcare is a 1 to N private social network for a physician and her patients/caregivers. There is so much digital technology out there being thrown at EHR and self-management and medical question and answers and god knows what else – while the physician patient relationship is being pretty much ignored. Pathcare is a digital tool that helps the patient and doctor communicate easier and faster. Doctors provide guidance, and patients provide personal experience. In return for saving time (we’re much much more efficient than email) and privacy (we are totally private unlike FB) and easier decision making (better data at the point of visit) we will charge $15/month for a doctor for an unlimited number of patients.
      That’s the deal.

      We do not traffic in ePHI – we are a secure conduit for the doctor and patient. Unlike the current fads and fashion for social media, neither side can post status updates on twitter or FB, patients join the doctor network by invitation only. And yes – we comply with HIPAA security and privacy rules. FWIW – this is our day job – helping medical device vendors meet and implement the security and privacy requirements for HIPAA.
      The privacy is by design not something we bolted on as an afterthought after a lawyer or consultant said something.

      Love to hear what you think.
      Danny

      Reply →
  • […] talk about how big data is going to revolutionize healthcare. I have talked at length in my post “Why big data for healthcare is rubbish” about why I think we need to take big data for healthcare with a big grain of […]

    Reply →

Leave a reply

Cancel reply
facebooktwittergoogle_plusredditpinterestlinkedinmailby feather