Should you use Amazon S3?

In any discussion about cloud computing, it is becoming harder and harder to ignore Amazon Simple Storage Service (S3), the massively scalable, cost-effective cloud storage solution developed specifically to house the huge influx of data created by organizations worldwide.

Amazon S3 commands twice the market share of all its closest competitors combined and is likely to be the storage platform of choice for on-premise hybrid or private cloud deployments for some time to come.

To a certain degree, S3 has already become the standard for cloud storage. Almost every application connects to S3 and most storage vendors have already announced that they connect to S3 or are working to do so.

In addition to Amazon, there are a number of competing storage implementations, including Google Cloud Storage, Openstack Swift, Rackspace’s Cloud Files and Ceph, that are S3 compliant. These services use the standard programming interface, but have different underlying technologies and business models.

The rise of S3, which Amazon describes as "cost-effective object storage", has also helped drive the adoption of object storage. In addition to Amazon, some of the biggest names of recent times such as Facebook, Netflix, Dropbox and Twitter use object storage.

It’s also deployed by enterprises for applications that require massive amounts of unstructured data, including content media storage, backup and archiving, data analytics, private cloud, file distribution and sharing.

With S3 and object storage so heavily interlinked, there are compelling reasons why companies seeking to implement a cloud strategy, whether it be across public, private or hybrid clouds, should take a closer look at the credentials of their storage supplier.

Compatibility Is Key

For organizations considering deploying an open-hybrid cloud and/or moving data between S3 and the private cloud, it is of the utmost importance that they understand the level of compatibility a storage platform claims versus its compatibility in reality. With S3 quickly becoming the object storage standard, choosing the right storage platform for the hybrid or private cloud can save organiZations money and shave months off the time to deploy. In essence, compatibility matters.

Companies and developers implementing applications with the Amazon S3 API depend significantly on its compatibility to ensure their applications remain compatible and functional. But because no one enforces standards over S3 compatibility, choosing the right storage platform can be much more complex than providers would like customers to believe.

There are 51 total operations available through the S3 API (nine simple, 18 moderately complex and 24 advanced), a good reference to S3 API set is here, but compatibility is based on a storage platform’s ability to perform some, many, or all of them.

This means that, in practice, storage platforms may claim S3 compatibility even though they only support the nine simple operations that allow only very basic manipulation of data through the API. And while other platforms might be able to provide a moderate level of compatibility, most of them are still unable to perform all of the 18 additional moderately complex operations.

Needless to say, most storage platforms find it even harder, if not impossible, to provide the 51 total operations of the S3 API. This should be a concern for organizations and developers that want the assurance their applications are S3 compatible and will work seamlessly with their hybrid or on-premise cloud, because they need to choose a storage platform that provides advanced compatibility with the S3 API. Only then can they have certainty that whatever strategy they adopt and whatever application they use will survive into the future.

If, however, organizations opt for a platform with only limited S3 compatibility, they could find that their storage does not support the commands required by future applications further down the line.

SDS in Private and Hybrid Clouds

The storage platforms best suited to providing private or hybrid cloud are highly likely to be software defined storage (SDS) solutions. Because SDS abstracts logical storage services and capabilities from the underlying physical storage systems, it is massively scalable, resilient, highly available and reduces complexity by allowing a single application to execute data services across multiple storage hardware platforms. Allowing organizations to use commodity hardware instead of expensive proprietary equipment, saves costs by eliminating vendor lock-in and the expense of forklift upgrades.

In effect, SDS allows organizations to replicate the Amazon S3 experience in their data centers and on their private and hybrid clouds. This makes it even more beneficial for enterprises to look for SDS-based object storage platforms that are truly S3 compatible.

Which means that the answer to the question, to S3 or not S3, is both.

Neil Stobart, EMEA Technical Director at Cloudian.

Published under license from ITProPortal.com, a Net Communities Ltd Publication. All rights reserved.

Photo Credit: rvlsoft/Shutterstock