How Big Data is destroying the US healthcare system

One thing I find ironic in the current controversy over problems with the healthcare.gov insurance sign-up web site is that the people complaining don’t really mean what they are saying. Not only do they have have little to no context for their arguments, they don’t even want the improvements they are demanding. This is not to say nothing is wrong with the site, but few big web projects have perfectly smooth launches. From all the bitching and moaning in the press you’d think this experience is a rarity. But as those who regularly read this column know, more than half of big IT projects don’t work at all. So I’m not surprised that there’s another month of work to be done to meet a deadline 5.5 months in the future.

Yes, the Obama Administration was overly optimistic and didn’t provide enough oversight. Yes, they demanded fundamental changes long after the system design should have been frozen. But a year from now these issues will have been forgotten.

The most important lesson here for government, I’d say, is to be more humble. At the heart of the federal problems you’ll find arrogance. California and Kentucky, after all, are  offering identical services that are reportedly going well: why didn’t the Feds learn from them? I’d guess it’s because they felt they had nothing to learn from the provinces.

"Not all smart people work at Sun Microsystems," Bill Joy used to say. And not all smart people work for the US Government, either.

I’d especially keep in mind that the people who are most upset about site performance issues are those who oppose its very existence. They are glad the problems are happening -- happy to complain. It’s not like they want the site to actually be more usable. I’d venture to say that these critics, while demanding specific improvements, would really be happier if those improvements didn’t happen. They are hypocrites.

Meanwhile there’s another vastly bigger and more serious point being missed here, I think, and that’s the role of Big Data in this whole US healthcare fiasco. This was explained to me recently by Jaron Lanier, by the way, so I don’t claim having thought of it.

Jaron is my hero.

His point is that Big Data has changed the US health insurance system and not for the better. There was a time when actuaries at insurance companies studied morbidity and mortality statistics in order to set insurance rates. This involved metadata -- data about data -- because for the most part the actuaries weren’t able to drill down far enough to reach past broad groups of policyholders to individuals. In that system, insurance company profitability increased linearly with scale so health insurance companies wanted as many policyholders as possible, making a profit on most of them.

Then in the 1990s something happened: the cost of computing came down to the point where it was cost-effective to calculate likely health outcomes on an individual basis. This moved the health insurance business from being based on setting rates to denying coverage. In the US the health insurance business model switched from covering as many people as possible to covering as few people as possible -- selling insurance only to healthy people who didn’t much need the healthcare system.

The goal went from making a profit on most enrollees to making a profit on all enrollees. Since in the end we are all dead, this really doesn’t work as a societal policy, which most of the rest of the world figured out long ago.

US health insurance company profits soared but we also have millions of uninsured families as a result.

Given that the broad goal of society is to keep people healthy, this business of selling insurance only to the healthy probably can’t last. It’s just a new kind of economic bubble waiting to burst.

Some might argue that the free market will eventually solve this particular Big Data problem. How? On the basis of pure economic forces I don’t see it happening. If I’m wrong, please explain.

Tell us all in detail how this will work.

Image credit: martan/Shutterstock

© 1998-2014 BetaNews, Inc. All Rights Reserved. Privacy Policy.