Articles about Big Data

You can't go too far in the big data world without encountering Apache Hadoop. The open source framework was created in 2005 to handle large scale processing.

A new infographic from data management company Solix looks at what Hadoop is, the four modules that make it up, and how it’s used in the enterprise, along with a glimpse at its future.

Continue reading →

Mention web or mobile surveillance, and you're sure to raise a few hackles. But the current Ebola outbreak is showing that the data collected from handsets can be extremely useful. The idea of tackling a disease with 'big data' gathered from mobile phones might seem a little odd, but it's actually an incredibly valuable source of information. Telecom firms such as Orange have been working with data scientists, using anonymized data gathered from phones to track population movement in regions affected by Ebola.

The BBC points out that even in relatively poor countries in Africa, mobile phone ownership is still high. Experts have been able to use this data to determine the best places to set up treatment centers, and it's an idea that has been pounced upon by the CDC.

Continue reading →

The main problem that organizations face when combating cyber attacks is that they don't know what to look for and find it difficult to interpret all the data they get from their networks.

Big data analytics company Exabeam has a new product that can cut through the forest of data to make it easier to detect attacks and insider threats in real time using existing security information and event management (SIEM) details.

Continue reading →

The Apache Spark framework is a popular add on to Hadoop for handling big data, particularly for building machine learning algorithms.

Until now though it's been hard to effectively monitor Spark performance. That's about to change as Brooklyn-based performance management company Sematext is launching its SPM for Spark.

Continue reading →

Identifying complex anomalies that show up the presence of a persistent threat or pinpoint the cause of network and IT performance problems is the sort of thing big data analysis was made for.

The problem though is that big data is often just too big. This means it's usually necessary to transfer the information before doing any work on it which makes real-time analysis impossible and means valuable insights aren't available when they're most needed.

Continue reading →

For organizations that deal with large quantities of data, the NoSQL database is a popular choice. However, it can present problems for traditional database programmers.

For this reason developers are always on the look out for ways to build applications more easily and unlock the potential of unstructured data. With the release of its latest Riak Enterprise 2.0, NoSQL specialist Basho is addressing these needs.

Continue reading →

IBM's Watson supercomputer is set to tackle scientific research head-on after being re-programmed to analyze big data in the cloud.

Currently, the testing of scientific hypotheses and theories often takes days or months of arduous work, but with Watson's Discovery Advisor program, this can now be carried out at a significantly faster rate.

Continue reading →

Big data analysis can open up valuable insights that are locked up in databases, but releasing that information without access to a team of data scientists isn't easy.

Analytics company Prelert is aiming to make big data accessible and valuable for all businesses with its anomaly detection engine, built using unsupervised machine learning technology. No human intervention is required to set parameters or tell it what to look for, once it's pointed in the right direction it will go to work on massive volumes of streaming data.

Continue reading →

Big data is no longer just the domain of big companies. As the perception of big data moves from futuristic hype to real-world opportunity, the promise of improved decision making, increased operational efficiency and new revenue streams has more organizations actively engaging in data analysis projects than ever before. That no longer only means more enterprise organizations, either. Midmarket companies are jumping on the big data bandwagon in a major way.

In fact, a recent survey by Competitive Edge Research Reports indicates that an astounding 96 percent of midmarket organizations are either already in flight with a big data initiative, or plan to start one in the next year. That's a whole lot of companies whose big data projects are either going to sink or swim in the very near future.

Continue reading →

Though it's one of the most popular big data tools, Hadoop is still evolving and as a result doesn't always have the sophistication of more traditional databases, which can lead to higher support and operation costs.

A new product from big data security specialist Zettaset aims to streamline Hadoop deployment for the enterprise with software automation that eliminates many of the manual configuration processes.

Continue reading →

Big data is already making a big impact all over the world. Large corporations, world organizations, and governments have hopped aboard the big data bandwagon, hoping to utilize new sources of data to improve operations and increase productivity among many other reasons.

It's easy to see from the latest technology news how such massive organizations can benefit from big data considering their available resources and history of using the technology. Finding ways big data is impacting life outside of large, urban areas may seem a little more difficult.

Continue reading →

Time consuming data retrieval requests are putting a big strain on IT teams, at least according to a new piece of research.

Iron Mountain (a storage and data management outfit) conducted the study, and spoke to a number of senior IT bods across Europe (including the UK) to get their opinions on exactly what weight big data was putting on their shoulders these days.

Continue reading →

Companies are increasingly keen to take advantage of the insights that big data can offer. But this does lead to problems with controlling access and keeping the data secure in a way that works with existing systems.

Big data security specialist Zettaset is now making its Orchestrator product available as a standalone offering. This will allow customers to take the whole offering or individual applications as required to customize and secure their environment.

Continue reading →

Data makes the modern world go round and by being able to use it effectively companies can gain a competitive advantage, improve customer experience and enhance operational efficiency.

Database specialist Oracle has combined a number of its offerings including the newly acquired BlueKai Audience Data Marketplace to form Oracle Data Cloud, a platform that provides customers access to world-class data-as-a-service (DaaS) offerings to allow companies to exploit marketing and social media data.

Continue reading →

The increased demand for solutions based on big data has led to something of a shortage of data scientists, which means in many cases companies are struggling to unlock the information they already have.

A new tool from data specialist GraphLab provides enterprise-grade machine learning to simplify and automate the handling of big data. By bringing together ease of use and computing scale the software makes it possible for one data scientist to do the job of many.

Continue reading →