In part one we learned about data and how it can be used to find knowledge or meaning. Part two explained the term Big Data and showed how it became an industry mainly in response to economic forces. This is part three, where it all has to fit together and make sense -- rueful, sometimes ironic, and occasionally frightening sense. You see our technological, business, and even social futures are being redefined right now by Big Data in ways we are only now coming to understand and may no longer be able to control.
Whether the analysis is done by a supercomputer or using a hand-written table compiled in 1665 from the Bills of Mortality, some aspects of Big Data have been with us far longer than we realize.
In Part one of this series of columns we learned about data and how computers can be used for finding meaning in large data sets. We even saw a hint of what we might call Big Data at Amazon.com in the mid-1990s, as that company stretched technology to observe and record in real time everything its tens of thousands of simultaneous users were doing. Pretty impressive, but not really Big Data, more like Bigish Data.
The real Big Data of that era was already being gathered by outfits like the U.S. National Security Agency (NSA) and the UK Government Communications Headquarters (GCHQ) -- spy operations that were recording digital communications even though they had no easy way to decode and find meaning in it. Government tape libraries were being filled to overflowing with meaningless gibberish.
Big Data is Big News, a Big Deal, and Big Business, but what is it, really? What does Big Data even mean? To those in the thick of it, Big Data is obvious and I’m stupid for even asking the question. But those in the thick of Big Data find most people stupid, don’t you? So just for a moment I’ll speak to those readers who are, like me, not in the thick of Big Data.
What does it mean? That’s what I am going to explore this week in what I am guessing will be three long columns.
Not very long ago I started answering questions on Quora, the question-and-answer site. My answers are mainly about aviation because that’s my great hobby and one of the few things besides high tech that I really know a lot about. But there was a question last week about Internet news coverage that I felt deserved better answers than it was getting.
So I contributed an answer that has been read, so far, only 388 times. I don’t like making a real effort that is so sparsely read. So here, with a little mild editing, is my answer to "What are the flaws in online journalism and media today?" And "How can they be addressed?"
Several readers have asked for my take on Microsoft’s purchase last week of LinkedIn for $26.2 billion -- a figure some think is too high and others think is a steal. I think there is generally more here than meets the eye.
Microsoft definitely needed more presence in social media if it wants to be seen as a legit competitor to Google and Facebook. Yammer wasn’t big enough. LinkedIn fits Redmond’s business orientation and was big enough to show that Satya Nadella isn’t afraid to open up the BIG CHECKBOOK.
Rumors are flying within IBM this week that the z Systems (mainframe) division is up for sale with the most likely buyer being Hitachi. It’s all a big secret, of course, because IBM management doesn’t tell IBM workers anything, but the idea is certainly consistent with Big Blue’s determination to cut costs and raise cash for more share buybacks. And the murmurs are simply too loud to be meaningless. Think of this news in terms of a statement made last week by an IBM senior executive: "In a world of Cloud Computing, it does not matter what equipment or whose hardware the cloud runs on. We are a Cloud company…"
This move by IBM would not surprise me in a bit. It is my guess IBM wants someone else to make and support the hardware. They’ll be happy to sell time sharing services, AKA cloud services. They’ll be happy to let someone else sell and maintain systems.
Bill Gates is a blogger, did you know that? His blog is called Gates Notes and generally covers areas of interest not only to Bill but also to the Bill and Melinda Gates Foundation, which means there’s more coverage of malaria than Microsoft. His latest post that a reader pointed out to me today is about raising chickens, which Bill says he’d do if he was a poor woman in Africa.
I’ll wait while you follow the link to read the post, just don’t forget to come back. And while you are there be sure to watch the video…
There is a difference between knowledge and understanding. Knowledge typically comes down to knowing facts while understanding is the application of knowledge to the mastery of systems. You can know a lot while understanding very little. Just as an example, IBM’s Watson artificial intelligence system that defeated the TV Jeopardy champs a few years ago knew all there was to know about Jeopardy questions but didn’t really understand anything. Ask Watson to apply to removing your appendix its knowledge of hundreds of medical questions and you’d be disappointed and probably dead. That’s the problem with most analytics, which is why it can be a hard sell.
The answer to this problem, we’re told, is not just machine learning but Deep Machine Learning, the difference between the two being that plain old machine learning is a statistical process that could be (and used to be) replicated by hand, while the deeper variety looks several generations deep in a longitudinal analysis that quickly grows too big for mere mortals to comprehend. Deep machine learning will, theoretically, find all the interconnections and dependencies that until now we’ve had to rely on domain experts to provide, yet even then it can only happen if you happen to be gathering the right data.
Apple this week invested $1 billion in Xiaoju Kuaizhi Inc. -- known as Didi -- by far the dominant car-hailing service in China with 300 million customers. While Apple has long admitted being interested in car technology and has deals to put Apple technology into many car lines, this particular investment seems to have been a surprise to most everyone. Analysts and pundits are seeing the investment as a way for Apple to get automotive metadata or even to please the Chinese government. I think it’s more than that. I think it is a potential answer to Apple’s huge problem of foreign cash and a grab for leadership in what may well be the second automotive age.
Apple has about $200 billion in offshore investments. That number is continuing to grow yet making very little return compared to Apple’s phone and computer businesses. As I’ve written before Apple has been very good at leveraging its cash to get better terms from suppliers but that game isn’t going to be getting any better (or worse) and the cash continues to pile up.
One of the frustrations of nanotechnology is that we generally can’t make nano materials in large quantities or at low cost, much less both. For the last five years a friend of mine has been telling me this story, explaining that there’s a secret manufacturing method and that he’s seen it. I’m beginning to think the guy is right. We may finally be on the threshold of the real nanotech revolution.
Say you want to build a space elevator, which is probably the easiest way to hoist payloads into orbit. Easy yet also impossible, because no material can be manufactured that is strong enough to make an elevator cable to space. The weight of the cable alone would cause too much tensile stress: it couldn’t carry itself, much less a commercially-viable payload, too. Some exotic new material is required, one with a strength-to-weight ratio beyond any present material, even spider silk. So we talk about space elevators, we have conferences about space elevators, we draw picture after picture of space elevators, yet we can’t make one. Or couldn’t… until now.
This is the kind of thing you find on the bedroom floor of a 14 year-old boy. It’s a gift from last Christmas, still sitting in its box, not yet flown for a reason that often comes down to some variation of "but the batteries need to be charged". I’d forgotten about it totally, which means the little drone missed the FAA’s January 20th registration deadline. Technically, I could be subject to a fine of up to $27,500. If the unregistered drone is used to commit a crime the fine could rise to $250,000 plus three years in prison.
Do you have an unregistered drone sitting in a closet somewhere?
This is the promised second part of my attempt to decide if IBM’s recent large U.S. layoff involves age discrimination in violation of federal laws. More than a week into this process I still can’t say for sure whether Big Blue is guilty or not, primarily due to the company’s secrecy. But that very secrecy should give us all pause because IBM certainly appears to be flaunting or in outright violation of several federal reporting requirements.
I will now explain this in numbing detail.
Is IBM guilty of age discrimination in its recent huge layoff of US workers? Frankly I don’t know. But I know how to find out, and this is part one of that process. Part two will follow on Friday.
Here’s what I need you to do. If you are a US IBMer age 40 or older who is part of the current Resource Action you have the right under Section 201, Subsection H of the Older Worker Benefit Protection Act of 1990 (OWBPA) to request information from IBM on which employees were involved in the RA and their ages and which employees were not selected and their ages.
Back in the spring of 2012 Congress passed the Jumpstart Our Business Startups Act (the JOBS Act) to make it easier for small companies to raise capital. The act recognized that nearly all job creation in the US economy comes from new businesses and attempted to accelerate startups by creating whole new ways to fund them.
The act required the United States Securities and Exchange Commission (SEC) by the end of 2012 to come up with regulations to enable the centerpiece of the act, equity crowd funding, which would allow any legal US resident to become a venture capitalist. But the regulations weren’t finished by the end of 2012. They weren’t finished by the end of 2013, either, or 2014. The regulations were finally finished on October 30, 2015 -- 1033 days late.
I promised a follow-up to my post from last week about IBM’s massive layoffs and here it is. My goal is first to give a few more details of the layoff primarily gleaned from many copies of their separation documents sent to me by laid-off IBMers, but mainly I’m here to explain the literal impossibility of Big Blue’s self-described "transformation" that’s currently in process. My point is not that transformations can’t happen, but that IBM didn’t transform the parts it should and now it’s probably too late.
First let’s take a look at the separation docs. Whether you give a damn about IBM or not, if you work for a big company this is worth reading because it may well become an archetype for getting rid of employees. What follows is my summary based on having the actual docs reviewed by several lawyers.