AI and the Global ‘Datasphere’: How Much Information Will Humanity Have By 2025? 

By: Tom Sullivan, LinkedIn

The sheer volume of data being created around the world is only expected to continue exploding in the short term. Consider the scope: “The Global Datasphere will grow from 33 zettabytes in 2018 to 175 zettabytes by 2025,” according to research from IDC, while Statista projects 180 Zettabytes in the same timeframe and the data protection company Arcserve predicted that the world will have 200 Zettabytes.

Given that every zettabyte is one billion terabytes those are stark differences — but, taken together, they point to the reality that executives and data professionals need to move quickly to prepare their organizations or risk falling behind competitors.

But which industries are growing the most quickly? What are the common opportunities and obstacles? And which technologies have the potential to help companies leverage their information as a strategic asset?

Much of the data growth today is being driven by top industries including manufacturing, financial services, media and entertainment, and healthcare, which is now the fastest growing sector with projections to create more than a third of all data.  

Healthcare has not traditionally been the data volume leader but it is now growing 6% faster than manufacturing, 10% beyond financial services and 11% more than entertainment and media. But while healthcare is outpacing the others, those are still seeing considerable increases as well.

It’s also important to note that “all sectors” constitute more than a quarter of expected data growth as this phenomenon is not unique to a limited number of industries.

While the idea of becoming data-driven holds considerable promise for companies relative to streamlining operational efficiencies, managing risk, identifying fraud, driving deeper and more rapid innovation, perhaps the Holy Grail is gleaning insights about customers to deliver personalized products and services that engage users on a daily basis, if not multiple times every day.

Research, in fact, supports the notion that people want personalization. Take healthcare, for example: 71 percent of participants IDC surveyed expect personalized interactions, 77 percent are frustrated by a lack of personalized services, and 83 percent indicated a willingness to share sensitive information in return for more personalization.

Gartner, meanwhile, found that the top drivers of brand loyalty across industries are the user experience (66 percent), the brand itself (20 percent) and, of course, price (13%).

That said, executives and data professionals are currently facing many challenges in managing the growth of data, notably a lack of expertise (53 percent) and a lack of necessary technologies (43 percent), among others covered in a MarTech Alliance report.  On the workforce front, 35 percent of professionals said the growth of data creates unnecessary work, 32 percent said it causes burnout and 26 percent find it stalling career progress, as we previously reported.

Those factors may be changing, however, as the United States Bureau of Labor Statistics projected a 36 percent increase in the employment of data scientists between 2021 and 2031, which the agency noted is “much faster than the average for all occupations.” 

Not surprisingly, cloud-based storage, compute, analytics and data management technologies will play a critical role in addressing the aforementioned obstacles. Specifically, data lake analytics are among the solutions being developed to enable organizations to securely store and leverage data. The data lake market is poised to grow in lockstep with increasing amounts of data.

“Through the period of 2022 – 2027, the global data lake market is set to increase at a staggering compound annual growth rate of 23.9 percent, increasing to $29.9 billion from $10.2 billion,” according to BCC Research, which explained that deploying data lakes can be a competitive advantage. “Companies implementing data lake services outperform similar companies by 9 percent in organic revenue growth.” 

The power of AI came into focus in late 2022 thanks to the release of ChatGPT3 and has since sharpened with so much attention on the subsequent ChatGPT4.

Generative AI technologies offer a glimpse into how companies can use artificial intelligence and machine learning to drastically improve productivity, with prominent CEOs from Alpha, IBM, Meta and Microsoft suggesting the tech could eliminate thousands of jobs at their companies – and the World Economic Forum estimating that nearly 25 percent of all jobs will be disrupted over the next five years. 

The future of AI more broadly than GPTs also holds considerable potential to usher in positive developments: PwC predicts that 45 percent of economic gains will come from product-oriented innovations that drive consumer demand. “This is because AI will drive greater product variety with increased personalization, attractiveness and affordability over time,” the firm wrote in its Global Artificial Intelligence Study: Exploiting the AI Revolution. 

Whether the global datasphere reaches 175, 180, even 200 zettabytes by the year after next or not, the need for companies to strategically leverage data will be increasingly important for the foreseeable future. The ongoing data explosion’s opportunities and obstacles, along with some of the critical technologies and related challenges of managing and strategically leveraging that information are among the reasons we are hosting Data Universe 2024.

Running April 10-11 at the North Javits Center in New York City, the comprehensive event will convene a global data and AI community shaping the future and sharing their expertise across industries while growing their perspectives, skills and networks. From leading multinational teams to startup innovators, connect with data professionals who are inspired by the same challenges and opportunities as you.

Stay tuned for updates on the data lake technologies to be featured at Data Universe – coming soon.

Learn more about why to attend Data Universe 2024.

Sources:

1 PwC: https://www.pwc.com/gx/en/issues/data-and-analytics/publications/artificial-intelligence-study.html

2 World Economic Forum: WEF_Future_of_Jobs_2023.pdf (weforum.org)

3 PwC: https://www.pwc.com/gx/en/issues/data-and-analytics/publications/artificial-intelligence-study.html