Dealing with data: What to do and where to store it

Today’s digitally-enabled organizations generate huge volumes of data that needs to be stored, maintained, and accessed. Indeed, it’s been estimated that globally, around 2.5 quintillion bytes of data is created every day. A figure that is escalating at a rapid rate as enterprises pursue big data analytics and IoT projects.

Added to which, the rising use of video, rich imaging, and AI applications means that much of this data is 'unstructured'. So much so that according to IDC as much as 80 percent of the world’s data will be unstructured by 2025. All of which adds further complexities into the equation when it comes to storing and preserving data so that it is accessible and available for analysis.

Scaling out storage infrastructures to cope with this data deluge is just one of the challenges confronting organizations. In today’s data-driven world, these infrastructures will need to be capable of handling data resources in a manner that supports the demands of increasingly dynamic workloads.

So it’s no surprise that finding the most cost effective and efficient way to store and manage data has become a mission-critical priority for organizations operating across every industry sector.

Defining a strategy for success

Determining the best method of storage for data assets begins with gaining a deep understanding of what data assets the organization generates and how this data is or will be used.

So undertaking a comprehensive data audit represents the first essential step for ascertaining the various types of data that are generated or collected, and how much of this needs to be archived or retained for the future and why.

For example, some data needs to be readily available for analytics that will directly affect business decisions that are made today or in the immediate future. Meanwhile, other data sets such as video footage may need to be retained for a mandated time period for compliance purposes.

Understanding all this will support much more informed decisions relating to the best method of storage for each type of data, together with a deeper understanding of how much capacity will be required.

Understanding which combination of storage options work best

Each organization will have very individual data storage priorities and goals and should tailor their data storage strategy accordingly. Armed with insights gathered during the data discovery process, the next step is to align the best storage options against these identified needs.

Let’s take a quick review of which storage solutions are best suited to a variety of different storage drivers.

On-premises storage solutions

Organizations will need to determine whether tape, disk or a combination of both will be needed to satisfy their varied requirements. Key considerations to take account of will include data access needs, whether an offline solution is necessary, and of course any budgetary constraints that may apply. But other factors may also sway the decisions that are made.

For example, while disk-based storage is faster than tape, tape storage offers organizations an 'air gapped' backup facility that ensures an offline copy of data will be securely maintained in the event of a successful cyber breach event. However, requiring less energy for power cooling and emitting less CO₂, tape-based storage systems represent a more sustainable option that has a lower overall impact on the environment.

By combining both disk and tape storage, organizations will be able to harness the benefits of both technologies. Enabling fast disk storage for high-performance applications while taking advantage of the low-cost, secure, and long-term archiving attributes that tape storage offers.

Cloud storage solutions

Ideal for non-permanent workloads that require very fast uploads and instant sharing, public cloud storage provides the scalable, high performance, and on-demand capacity that’s required for today’s demanding applications. However, as a long-term storage option for hosting objects and files it can prove prohibitively expensive.

For this reason, many organizations choose instead to opt for a hybrid strategy that blends cloud and on-premises storage solutions to support all the varied needs of their different workloads. Creating in effect a 'best of both worlds' solution that enables them to cost-effectively manage the entire data lifecycle -- creation, storage, usage, archiving -- in a durable, secure, and scalable way.

Looking to a data-filled future

To derive value from their data assets, organizations will need to ensure it is appropriately retained and secured for the short and the long term in the most resilient and cost-efficient manner possible so that data can be leveraged easily and more value can be extracted.

As organizations strive to manage their growing volumes of data and ensure this is provisioned effectively, they will need to make truly smart decisions about the data storage strategies they pursue. By evaluating what data they have and how it is used, alongside a range of other key drivers, they will be able to identify which solutions to combine to optimise their data security, total cost of ownership, and compliance postures.

Image Credit: Wayne Williams

Eric Bassier is Senior Director Products, Quantum.