Articles about Big Data

Where are our cities headed in the future? What can we expect from the long sought after "smart cities"? Can we really expect anything different?

If big data has anything to do with it, we certainly can. We've already seen tremendous strides over the last 20 years in creating more energy efficiency, cleaner air, better transportation and improved security, but it's only a small fraction of what we can expect to see in the future with big data.

Continue reading →

Given the amount of hype surrounding it you could be forgiven for thinking that big data is the answer to most business and scientific problems.

But a new survey by database specialists Paradigm4 reveals that almost three-quarters of data scientists believe that big data has made their research harder. The reason for this is not, as you might expect, the volume of data but the variety.

Continue reading →

Database specialist DataStax is using the 2014 Spark Summit to announce the release of its latest product. DataStax Enterprise 4.5 includes some major updates aimed at improving ease of use and security.

Built from the ground up, upgrades in DSE 4.5 include lightning fast in-memory analytics with Spark -- thanks to a recently announced partnership with Databricks. In addition automated diagnostic and performance tuning drives operational simplicity and enhanced visual management. For the first time companies can now easily merge Cassandra data with Hadoop and avoid the complexities of data management thanks to partnerships with Hadoop vendors Cloudera and Hortonworks.

Continue reading →

Search analytics specialist Elasticsearch has announced the 2.0 release of its Hadoop connector, bringing advanced search techniques to the popular big data platform.

It's also now certified for Cloudera Enterprise 5. This means Elasticsearch is now compatible across all Apache-based Hadoop distributions, helping businesses extract immediate insights regardless of where their data is stored.

Continue reading →

The SAP HANA in-memory analytic tool can be deployed in house or in the cloud, but to get information from it users have previously had to choose between aging legacy business intelligence (BI) tools that take months to implement and require large teams to support, or limited data discovery tools that are restricted to analysts.

Now though cloud BI specialist Birst is launching a new product to make data analysis more accessible.

Continue reading →

Big data is all the rage at the moment but when it comes to security it shouldn't be treated in isolation from the rest of the organization.

According to a new report from research specialists Gartner, policies need to take account of all forms of data if security problems are to be avoided.

Continue reading →

Big data isn't just about finding more effective ways to market and advertise -- it's also about making the world a better place.

One of the industries in which big data is having its greatest impact is health care. So many improvements are being made with better results in hospitals and healthcare facilities across the world. One area, however, that continues to prove extremely difficult for the healthcare industry, and other industries, is suicide prevention.

Continue reading →

Big data places enormous demands on storage and in many cases conventional technologies are struggling to keep up.

In an effort to deliver improved economics and at the same time enable organizations to access and process any type of data, on any type of storage device, anywhere in the world, IBM has unveiled a portfolio of software defined storage products.

Continue reading →

Hitachi Data Systems has just revealed the results of its "Information Innovation Index," which is a study that aimed to measure how businesses are able to tap into and make good use of big data. And the short answer is that many feel they're not making best use of the swathes of data they're sat on.

The study was conducted by Vanson Bourne across 200 Chief Information Officers and IT decision makers in the UK, and almost half (46 percent) of respondents felt that their current IT setup was hindering their ability to effectively crunch big data for valuable insights.

Continue reading →

Big data jobs are some of the most well-paid jobs across all sectors, and in honor of Big Data Week 2014, we thought we'd put together a guide on how you can get in on the action.

There's been a predicted shortfall of 243 percent of skilled individuals needed to fill big data analytics jobs in the UK alone, and companies and organizations are currently scrambling over each other to recruit emerging talent. What's more, a data scientist can emerge from just about any field. Most analysts will tell you that it's misleading to describe big data simply in terms of a set of tools or programming languages. The common thread is the need for data-savvy professionals who have the ability to draw meaningful insights from the flood of data pouring into the modern organization.

Continue reading →

Research specialist IDC has released the latest results of its Worldwide Semiannual Software Tracker showing that the software market grew by 5.5 percent in 2013, reaching a total size of $369 billion.

IDC divides the market into three segments: Applications; Application Development and Deployment (AD&D); and Systems Infrastructure Software. These segments all grew between 5.4 and 5.6 percent, meaning that none of them had a significantly better performance in 2013.

Continue reading →

Almost every organization today has employees that regularly violate standard security policies and protocols. However, oftentimes these violations aren’t the work of a malicious insider -- they’re usually the actions of an employee trying to do his/her job or taking a shortcut to get the job done. Corporate information security teams have the challenge of determining the motive behind these violations. While network monitoring security tools and InfoSec point-solutions are designed to help catch these infractions, they can, and do, unintentionally create an immense volume of work by flagging every policy violation as a threat -- creating thousands or even hundreds of thousands of security events to sift through daily. To make matters worse, these events are often labeled as high-urgency alerts.

The challenge is empowering your IT security teams to identify and respond to the most urgent threats while maintaining compliance with industry regulations.

Continue reading →

IBM's S/360 mainframe celebrated its 50th birthday on Monday. It was the machine that turned IBM into a corporate colossus, but because it was smaller and faster than other offerings of the early 60s it also put big-league computing power in the hands of businesses, not just governments and universities.

Since the personal computer revolution of the 1980s people have been predicting the death of the mainframe at the hands of distributed low-cost servers, but the big beasts continue to stalk the land of serious business computing.

Continue reading →

It's sometimes said that big data is like teenage sex, everyone talks about it but few are actually doing it. To which you could add that those who are doing it aren't really sure if they’re getting it right.

In an effort to find out how big data is being used in the real world, we spoke to the heads of three startup companies which are employing it in unique ways to pursue specific business opportunities.

Continue reading →

The data contained in logs can be useful for a variety of purposes, but analysing it in order to extract what you need can be difficult and time consuming.

Search analytics specialist Elasticsearch is aiming to make log data more accessible with the release of Logstash 1.4.

Continue reading →