Undertaking a data diet this Black Friday

Like every year, Black Friday marks the start of the end-of-year shopping season, and with it a significant increase in online sales. Although it's very much an American event, Black Friday in the UK has grown in popularity in recent years. In 2022, UK consumers spent a record £9.42 billion over the Black Friday weekend. Of this, 61 percent was spent online.

For e-retailers in the UK, the cyber weekend represents a significant increase in activity and traffic, leading to an accumulation of data generated and collected by e-commerce sites, requiring large storage capacities with a high environmental impact.

Data growth in figures

Hosting data involves considerable energy consumption, which is only going to increase as the amount of data collected by companies increases, so much so that it is difficult to estimate the volume of data in the years to come.

In Europe alone, the OECD (Organization for Economic Cooperation and Development) estimates that the amount of data will increase from 86 exabytes in 2023 to 225 exabytes in 2030. On a global scale, this figure is doubled by the amount of data produced alone, which is expected to reach 463 exabytes per day by 2025.

Can storage-related energy consumption be rationalized?

We believe that companies need to change their behavior towards data and adopt habits that will reduce the volume of data stored and, ultimately, its environmental impact. They can apply the following practices:

• Index data using data management software to enable them to  identify obsolete, redundant, orphaned, and out-of-date data so that it can be safely deleted.
• DeDuplicate and compress data to eliminate redundant copies and/or lighten certain structures, automatically replacing original data with a lighter version. The amount of data can be reduced by up to 97 percent depending on its type.
• Classify data to reduce "Dark Data" mountains, which represents up to 75 percent of a company's data, the content of which they are unaware of. This can represent an enormous amount of space on their storage platforms, which can be freed up to make room for relevant data without the need to buy more technology to accommodate growth, especially for short super-peak periods such as Black Friday/Cyber Monday.

But most importantly, companies have an interest in exploiting the capabilities of AI, which they are already using for customer experience, to optimise the storage of their data. Although energy-intensive, AI is proving to be a considerable help in clarifying the content and value of data, and so being able to automatically identify obsolete, orphaned and redundant data that can be deleted immediately.

To meet modern energy challenges while continuing to amass data, companies need to focus on indexing, classification, and lifecycle-centric data management.

Most companies have their infrastructure crammed with data, where on average they don't even know 70 percent of the content. In this unstructured dark data, cat videos can be found as well as the menu from the last Christmas party, aged copies of databases, research results, all mixed with data that must be retained for regulatory and commercial purposes.

This data needs to be cleaned, and not only to reduce its risk of litigation. Anyone who cleans up and disposes of data waste will be able to feed their AI with high quality content and free up space for new data. To do this, the data must be indexed and classified according to its content and value for the company. AI also plays a key role here in classifying the content very accurately at pace.

Companies should consolidate their data on a common platform instead of continuing to operate dozens or even hundreds of separate silos. There, this data can be further reduced using standard techniques such as deduplication and compression. Reduction rates of 96 percent are possible in everyday life.

What each individual can do

Every user can help reduce overall power consumption and slow down data growth. Because everyone can search through their data in the cloud and delete what is useless. This can be X-fold versions of the same photo, with a slightly different perspective. Or videos that you once found funny and haven't watched since. That cat video perhaps.

Every bit we can save through reducing our stored data, will reduce energy consumption. So let’s start cleaning up.Technological innovations such as AI should also be approached as a tool to optimise on-premise and cloud storage, through a better understanding of the data they host. When integrated directly into a data management solution, AI can reduce the amount of data stored, and therefore the energy resources consumed.

Image credit: AndrewLozovyi/depositphotos.com

Mark Molyneux is CTO for EMEA at Cohesity.