Enterprises turn to Kubernetes to handle big data

Kubernetes is rapidly becoming the standard for cloud and on-premises data workloads according to a new study from big data performance management company Pepperadata.

The study of 800 participants shows the motivations for adopting the container platform include improving resource utilization for reduced cloud costs (30 percent). While 23 percent want to enable their migration to the cloud; 18 percent to shorten deployment cycles; 15 percent to make their platforms and applications cloud-agnostic; and 14 percent to containerize monolithic apps.

"Kubernetes is increasingly being adopted by our customers for big data applications. As a result, we see customers experiencing performance challenges," says Ash Munshi, CEO at Pepperdata. "This survey clearly indicates that these problems are universal and there is a need to better optimize these big data workloads."

Porting hundreds or thousands of apps over to Kubernetes can be a challenge and the biggest hurdles for survey respondents are cited as initial deployment, followed by migration, monitoring and alerting, complexity and increased cost, and reliability.

Interestingly despite the buzz surrounding public cloud, this survey finds that 47 percent of respondents are using Kubernetes in private cloud environments. On-premises use makes up 35 percent and just 18 percent of respondents say they're using Kubernetes containers in public cloud environments.

The study shows 45 percent of Kubernetes workloads are in development and testing environments, as users move production workloads into a new resource management framework. 30 percent are doing proof-of-concept work. 66 percent of respondents say 75 to 100 percent of their big data workloads will be on Kubernetes by the end of 2021.

IT operations is the clear leader in application deployment with 80 percent deploying Spark and other big data apps built on Kubernetes. Engineering follows with 11 percent, with business unit developers at just nine percent.

You can get the full report from the Pepperdata site.

Image Credit: Maksim Kabakou/Shutterstock