Articles about Hadoop

Back in 2006 Apache Hadoop emerged and soon began to revolutionize the nascent world of big Data. It’s one of the key factors that helped shape a new industry and -- with the cloud -- helped drive a raft of new consumer industries and business services.

But the data lakes of Hadoop became a challenge to manage, and many Big Data and analytical projects became more of a quagmire than a sparkling lake of truth. What’s more the number compute engines overpopulated. They were operationally complex and demanded specialized skills to maintain. Those data lakes became very disparate compute engines, sharing the same storage whilst they operated disparate workloads. It became a mess. Managing them with the tools available was no picnic.

Continue reading →

With technologies such as artificial intelligence, machine learning, IoT and advanced analytics hitting a critical mass, it’s no surprise that the big data market continues to grow rapidly. According to a forecast by Statista, the big data market reached $42 billion dollars in 2018 and is expected to reach $64 billion by 2021.

Big data presents major opportunities for organizations to gain new insights, deliver better products and improve operations, but the traditional storage approach to big data is fraught with many challenges. It’s time for another way.

Continue reading →

Almost 60 percent of enterprises adopting big data cite both increased productivity across the organization and improved efficiency leading to reduced costs as their biggest gains.

This is among the findings of a survey by enterprise software company Syncsort, which finds that the most dramatic increase in reported benefits was higher revenue and accelerated growth, which 55 percent named as a benefit this year compared with only 37 percent last year.

Continue reading →

Companies of all sizes have started to recognize the value of big data collections and the need to take advantage of them. Development of software systems play a big role in big data analytics.

As companies proceed on their big data journey, they usually start by batch processing their big data assets. This can mean gathering and aggregating web log data, telemetry from IoT devices, user clicks from an app and more.

Continue reading →

Hadoop, for all its strengths, is not without its difficulties. Business needs specialized skills, data integration, and budget all need to factor into planning and implementation. Even when this happens, a large percentage of Hadoop implementations fail.

To help others avoid common mistakes with Hadoop, I asked our consulting services and enterprise support teams to share their experiences working with organizations to develop, design and implement complex big data, business analytics or embedded analytics initiatives. These are their top seven mistakes, and some advice on how to avoid them.

Continue reading →

Unless you’ve been hiding away from the world of computing for the last few years, you’ll have come across Hadoop.

Apache Hadoop, to give it its full name, is an open source framework designed to handle the storage and processing of large amounts of data using low-cost commodity hardware. Since its initial release in 2011, it has become one of the most popular platforms for handling big data.

Continue reading →

Hadoop is one of the most popular frameworks for creating distributed processing environments using commodity hardware. This makes it the first choice for many when implementing big data.

However, the nature of this type of system means there are inherent performance limitations. Distributed systems performance specialist Pepperdata is launching Hadoop Health Check, a complimentary, expert assessment that evaluates and diagnoses Hadoop clusters of 100 nodes or more, and provides full visibility into current cluster conditions.

Continue reading →

Big Data promises businesses a number of advantages, but in order to harness these effectively they must first choose the right software to engage with and analyze vast quantities of information.

For many organizations, including high profile firms like Facebook and Yahoo, Hadoop is the software of choice when it comes to managing Big Data. In fact, the global Hadoop market is reported to be growing at around 55 percent a year and is expected to be worth $20.9 billion by 2018.

Continue reading →

The popularity of big data has boosted demand for Hadoop systems. Yet data handling and analytic capabilities are often seen as the key features while governance and security get relegated to a support role.

In corporate environments though the governance side of things is important which is why big data specialist Datameer is launching new tools for maintaining data validity and manageability in the Hadoop ecosystem.

Continue reading →

You can't go too far in the big data world without encountering Apache Hadoop. The open source framework was created in 2005 to handle large scale processing.

A new infographic from data management company Solix looks at what Hadoop is, the four modules that make it up, and how it’s used in the enterprise, along with a glimpse at its future.

Continue reading →

Though it's one of the most popular big data tools, Hadoop is still evolving and as a result doesn't always have the sophistication of more traditional databases, which can lead to higher support and operation costs.

A new product from big data security specialist Zettaset aims to streamline Hadoop deployment for the enterprise with software automation that eliminates many of the manual configuration processes.

Continue reading →

Search analytics specialist Elasticsearch has announced the 2.0 release of its Hadoop connector, bringing advanced search techniques to the popular big data platform.

It's also now certified for Cloudera Enterprise 5. This means Elasticsearch is now compatible across all Apache-based Hadoop distributions, helping businesses extract immediate insights regardless of where their data is stored.

Continue reading →

Microsoft's ongoing process to improve the company's cloud platform, Windows Azure, has reached a new phase. The software giant has, yet again, introduced a number of new features for Windows Azure, including the HDInsight service for Hadoop clusters, support for Dropbox deployment and Mercurial repositories, as well as enhancements to Mobile Services.

Windows Azure Mobile Services can now be used as a backend by "pure" HTML5/JavaScript clients, Apache Cordova/PhoneGap apps and Windows Phone 7.5 clients. The feature complements the previously-introduced Android Client SDK (Software Development Kit) and support for iOS, Windows 8 and Windows Phone 8.

Continue reading →

Yahoo has turned to the Hadoop open source cluster framework for crunching the data from "500 million users per month and billions of interesting events per day," said Yahoo's Ajay Anand, speaking at the Cloud Computing Expo in New York City this week.

Researchers from Yahoo Mail have already used Hadoop as a platform for finding botnets that are spewing out spam. Other Yahoo researchers have collaborated on Webmap, "a gigantic table of information [showing] every Web site, page, and link that Yahoo knows about," he told a packed hotel conference room on Tuesday.

Continue reading →

Well backed Silicon Valley start-up Cloudera has now released a free, private cloud-oriented distribution of a Linux software environment first built by major Web enterprises for "big data."

"Hadoop offers capabilities for capturing, storing and analyzing data that are unmatched. But it's something that enterprises have shied away from until now," said Michael Olson, a former VP at Oracle and now CEO and co-founder of Burlingame, CA-based Cloudera, in a briefing with Betanews.

Continue reading →