The role of private clouds in enterprise data strategy [Q&A]

As AI reshapes business operations, companies are facing new challenges around cost uncertainty, security, and data integrity. The rise of hybrid cloud strategies -- combining private clouds with public infrastructure -- is becoming a key approach to addressing these concerns.

While enterprises focus on cost control, safeguarding sensitive corporate data, and preventing AI-driven data leakage, they are also trying to leverage LLMs to exploit data in the public cloud while retaining sensitive data in private clouds that they control.

We spoke to Neil Carson, CEO of SQL data platform Yellowbrick, to find out why private cloud solutions are gaining traction and how businesses can best exploit them.

BN: What are the key benefits of a private cloud?

NC: Private clouds mean different things to different businesses; for some, it means running applications and keeping data inside your cloud provider account vs. using a shared SaaS service; for others, it means using an 'on-premises' environment running in a co-location or even a privately owned data center.

The topic is becoming top of mind due to a renewed focus on security, data residency and cloud repatriation, as well as cost savings. According to Harvard Business Review, 80 percent of data breaches in 2023 involved data stored in the cloud. Individuals are worried about this; I personally witnessed this firsthand during the Snowflake leaks, when I arrived home to letters from my telco, healthcare provider and various retailers stating my data had been stolen. In tandem, international businesses are continually having to meet more complex data residency requirements and governments are rightly expending increasingly more effort to safeguard their data.

Misconfiguration in cloud deployments is so common. It's trivial to grant public access to services. Just a couple of clicks and all the data stored in an S3 bucket becomes public for everyone to access. The famous verifications.io data breach happened because of a MongoDB database with 763,000,000 email addresses sitting on the internet with no password. Netflix, TD Bank and Ford had data stolen due to a public bucket at Attunity. The architectural choices of the last few years to compose applications of multiple SaaS services means that each one is a potential attack vector, as seen with Snowflake and others.

Running your software and all associated services in a closed-off cloud account affords many services far more protection. Putting them in an on-premises environment behind a firewall where nothing can ingress or egress offers even more protection, since nothing can be made public due to a simple misconfiguration. In the past only Michael Dell talked about cloud repatriation, despite the obvious cost savings of running on-premises. However, the tide is turning, with more wise businesses realising that hybrid cloud and on-premises deployments are not just more cost effective, but also more secure.

BN: How is this of value as AI adoption accelerates?

NC: Social media platforms such as Facebook, Instagram, LinkedIn and X train their AI models using their customers’ data, including photos, videos and public posts, to enhance user engagement and boost revenue generation.

Enterprise CIOs realize that a model which is trained on their private data, confidential business processes and operational procedures would be of massive value to an attacker or competitor. Furthermore, if the training of that model results in said data or practices being fed back for others to consume or enable the AI vendor to become more capable in their particular industry or vertical, competitive advantages are diminished. For these reasons many enterprises are wanting to keep both their core enterprise data and AI models private.

BN: Is it necessary to have a private cloud entirely in house or can you use a virtual approach using public infrastructure?

NC: Most private cloud architectures involve some hybrid components. Much enterprise data is still generated at the edge (for example, in a factory) or on-premises (for mainframes or core OLTP systems), customer loyalty and behavioral data is often sourced in the cloud, and most modern applications exist to some degree in the cloud or in both worlds.

BN: What are the challenges of integrating public and private in a 'hybrid' cloud?

NC: Many of the challenges resolve around vendor lock-in, application architecture, management complexity, platform differences, heterogeneous infrastructure, as well as traditional concerns like compliance and cost management. The key enabling technology here is Kubernetes: Thanks to K8s, it's now possible to use a uniform approach to building, deploying, testing and managing applications and their associated security models and infrastructure across all public cloud and private/on-premises deployments. This is what’s lead to a new generation of enterprise apps that can run anywhere, and more efficient use of on-premises infrastructure to make it cost-advantageous vs the major cloud providers. In the not-too-distant future, we'll see software sold as software again, running on top of Kubernetes, rather than being provided by SaaS providers who must mark up and resell infrastructure.

BN: Can you give us some use cases where private cloud is of particular benefit?

NC: Our customers turned to private clouds to address data residency and sovereignty requirements and to cut costs. For example, Melco Resorts & Entertainment has player data that cannot leave certain regional casinos. Telcel, handles vast amounts of call data records, found public cloud-only options both risky and too expensive. On the supply chain front, the US Navy conducts supply chain analytics for the Department of Defense using both on-premises solutions and AWS GovCloud. Similarly, Insightsoftware runs complex SAP financial management applications with sensitive data kept secure in private environments.

Image credit: BiancoBlue/depositphotos.com