Data Sprawl and the Pandora’s Box It Opens: Dealing With Data Overload in the Digital Age

Data is the fulcrum of this digital era.

Nothing gets done without data. It is both a map and compass, guiding businesses on what to do and providing different pathways to that end.

It fuels innovation. It empowers technological advancement. It moves the world.

It also opens up a Pandora’s Box of challenges that when left unaddressed may render data useless—or at least diminish its usefulness.

One of these challenges, perhaps ironically, is directly related to the data boom where an estimated 2.5 quintillion bytes of data are being created every day.  That amount is forecast to grow even more by 2025 when the world will create 463 exabytes each day.

That’s a whole lot of data . . . and it might just be too much for an organisation to handle.

Too Much Is Never a Good Thing: The Problem That Is Data Sprawl

This preponderance of data, according to Sophos Field Chief Technology Officer for Asia Pacific and Japan, Aaron Bugal, is bound to create one of the major challenges businesses will be facing in this data-driven landscape. And it can leave them exposed to another massive challenge that can have far-reaching implications for the business in more ways than one.

“One of the key challenges businesses will face during the peak of the information age is data sprawl,” Aaron told Data & Storage ASEAN via email. “Businesses are so accustomed to collecting data—disk is cheap—that most have not put a lot of thought into what data is being collected and why it’s needed.”

Data sprawl is the unrestrained growth of data across an organisation’s IT environment—from the network to the different systems and apps and everywhere else. In other words, data sprawl is essentially data overload, and it can happen innocuously enough.

Companies generally use a variety of enterprise applications, including project management tools, customer relationship management software, and video conference platforms, and these acquire loads of both structured and unstructured data that are stored in oftentimes disparate endpoints. That’s on top of the data company employees store on their own devices, the data obtained from the use of new technologies, and the data stored on cloud provisions.

Before the company knows it, data is everywhere—stored all across the company’s IT infrastructure. Some are useful but the others, not so much.

Getting Value from Data Is the End-Goal

Figuring out which data are useful and which ones aren’t is a challenge organisations will have to navigate, or else render all that data useless. Unfortunately, it can be a Herculean task—one that will require a clear, strategic, purposive approach and some help from key technologies.

“Gathering data is a straightforward task, yet transforming them into actionable insights demands a coherent strategy,” Ben Lim, Senior Country Manager Malaysia at Epicor, told DSA in an exclusive interview. “Organisations should aim to unify the data infrastructure across platforms while minimising vendor reliance. This brings about unified governance that permeates throughout businesses. With these in place, productivity thrives from accurate, centralised data—the single source of truth.”

It sounds simple enough. But where do organisations even start? How do they begin this process of bringing about unified governance?

Ben has a recommendation: Leverage the cloud.

“To kickstart the process of getting the right infrastructure for scaling up data leverage, cloud migration is key. Business stakeholders can integrate workflows and define goals, policies, and challenges within a coherent data framework,” Ben explained. “Enhanced employee access is crucial for today’s hybrid workforce for them to be able to effectively use organised data, while cloud environments facilitate scalable security measures, vital in combating prevalent malicious threats like ransomware.”

Utilising the cloud is consistent with Ben’s belief that “technology plays an important role in building up individuals to maximise return on investment”—and, indeed, innovation will play a central role in managing all this data.

At the forefront is Artificial Intelligence (AI), which enables integration and automation that “are essential for achieving digital cohesion,” according to David Irecki, Director of Solutions Consulting, APJ, at Boomi.

The focus is on intelligent integration and automation, guiding users on how to be smarter and faster, and the motivation behind Boomi AI, which delivers a suite of radically intuitive capabilities powered by generative AI for users to connect applications, data, processes, people, and devices,” David explained. “For organisations to thrive in the AI era and to be AI-ready, a robust data ecosystem is definitely needed, as the effectiveness of AI for analytics relies entirely on the quality of its available data… [but] only with a unified platform, powered by AI technology can organisations unlock greater productivity and accelerate business outcomes while mitigating unwanted complexity.

Don’t Worry, It’s Normal

While data sprawl is a major issue, it is not unique to these times. For Sandie Overtveld, Senior Vice President, Asia-Pacific, Japan and Middle East and Africa, at Freshworks, it is actually quite normal.

“Many organisations are working with a myriad of legacy technology, like on-premises services and new cloud and SaaS systems. It is a norm for operational and security challenges to surface due to disparate tech stack and migration to new systems,” Sandie told DSA via email.

And, as with any complication, simplifying things is the next logical step, according to Sandie.

“The key to combating data sprawl is decluttering and simplifying. Practising app rationalisation entails a continuous and proactive audit of a company’s tech stack to ensure a strong focus on organisational security,” said Sandie. “This reduces the burden and enhances the organisation’s understanding of data and software, usage, and compatibility of tools. Business leaders have to evaluate if the tool integrates seamlessly with the existing suite and whether it aligns with the organisation’s needs. If it is not, it has to be replaced with a new option or check if an existing tool can be used instead.”

Sandie’s advice echoes something similar David pointed out to DSA about “tech stacks becoming money pits,” resulting in systems operating in isolation and licences becoming underutilised—all while returns on investments are left unrealised or hard to see. Translation: Having too many tools isn’t the answer to data sprawl. In fact, this condition can, according to David, reduce the possibility of scaling rapidly, adapting to seize new market opportunities, or creating great customer experiences.

In other words, find the right tools (tech)—the ones that work best—and stick with them. Just as important, decide which data can be useful to the organisation and which ones can be discarded.

“Being ‘data-informed’ encompasses questioning and exploring available data’s usefulness for specific business goals. Having vast amounts of data serves no purpose if they do not contribute to a positive bottom line,” Ben pointed out. “Next, being ‘data-driven’ by leveraging existing data to derive insights and enhance decision-making empowers employees, with models built around trends, alleviating stress points and replacing outdated processes. A secure and reassured workforce leads to better customer service, increased innovation and higher retention rates.”

Making a Challenging Proposition Even Harder

Simplifying a tech stack, though, should not come at the expense of cutting corners on hi-tech features—especially those related to security.

With so much data, organisations are now prime targets of cybercriminals. And, as it is, safeguarding data from these cyber adversaries is hard enough given how prolific they have become—40 million records were compromised in 2023 alone, up from 33 million in 2022—and how sophisticated they are becoming. Lately, in fact, Southeast Asia has been rocked by some notable cyber incidents, the latest being the database breach of Singapore’s Poh Heng Jewellery just this March, which comes on the heels of the R00TK1T group allegedly infiltrating Malaysia’s Maxis Berhad.

Put simply, data is under threat. It has become the target of choice for cybercriminals.

“Amongst the largest cybersecurity breaches of recent times, data theft is a common objective that criminals aim for and use as leverage over their victims to almost guarantee they’ll pay a ransom demand,” Aaron noted. “These exfiltrated data sets are typically available on the dark web for anyone to download, and analysis of this data has shown in some cases there’s a lack of governance around data collection. Typically, too much extraneous data is collected by businesses, which subsequently finds its way into a database with no time frames on expiry and forgotten about—thus becoming a prime target of cybercrime.”

Data sprawl only exacerbates the situation by making it harder to store data and safeguard it—two responsibilities of organisations, whom Aaron describes as “data custodians” in this digital age.

“The data sprawl issue will remain for those who have been collecting large pools of data, storing it, and perhaps not managing that data in the best interest of privacy,” he said. “As a business that collects information, you’re now acting as a data custodian. You are responsible and accountable over the correct collection, storage, and use of others’ data and you must discover where this information resides in your business and safeguard it to your best ability.”

Unfortunately, according to Sandie, “security often becomes an afterthought” as organisations rely on too many tools and apps. This is why it is imperative for organisations to know their data and its storage and implement strategies to safeguard and govern these data because without this understanding, “internal collaboration for effective data protection becomes challenging.”

Modern Storage for Modern Problems

At the very least, though, organisations appear to be recognising the urgency to safeguard their data—useful or otherwise—and are trying their best to comply with data regulations.

“The availability, integrity, and confidentiality of data that a business chooses to store about its employees, customers, supply chains, and artefacts as a result of conducting business over the internet will generate a tectonic shift in attitudes towards better information security,” Aaron claimed. This will challenge data collection norms and guide businesses to collect only what they need and to store it responsibly—or else risk financial penalties and potential sanctions for failing to do so.

These shifting attitudes appear to be skewing towards enhanced, proactive data security. Modern storage solutions are at least giving organisations willing to invest a fighting chance by leveraging innovations like AI, encryption, and even cryptography.

“Storage systems will increasingly integrate AI-based defences as an integral part of their security strategy,” Aron Brand, Chief Technology Officer of CTERA told Forbes in a digital storage look-ahead feature for 2024. “These systems will use machine learning to detect and neutralise cyber threats in real-time, protecting data from ransomware and ensuring the integrity of stored information. This integration will strengthen cybersecurity and streamline incident response.”

Solid-state controller company Phison believes the next generation of storage systems will integrate even more advanced technologies so they can better keep data safe.

“SSD, GPU, DRAM, and other essential data centre components will increasingly include device-level cryptographic identification, attestation, and data encryption to help better guard data against attack as AI deployments expose new digital threats,” said Phison in a press release last year. “Ultra-rapid advancements in AI and LLMs will challenge AI infrastructure reliance on GPU and DRAM, resulting in new approaches to architecture that take greater advantage of high-capacity NAND flash…”

A Challenge That Needs Solving

Data sprawl—along with other data-related challenges—is a side effect of digital transformation, said David. And it results in rigidity at a time when responsiveness is a requirement. This is why solving the data sprawl issue is paramount, and it starts with utilising the right technologies to put data in one overriding repository that will serve as the organisation’s single source of truth. From there, available technologies like AI and automation can be used to great effect.

“A single source of truth is necessary across the entire ecosystem to analyse the data, without the hassle of navigating between multiple cloud providers or dashboards. Automation is an essential part of monitoring operations, collecting and analysing data, and providing insights to administrators or business teams in real-time increasing productivity,” Sandie concluded.

The end result is an enterprise well-prepared for the digital future—data-driven, AI-ready, and responsive to changing times.

That all begins with data—the fulcrum of this digital era.