Your business doesn’t stay still— and neither does the data landscape. While the next 12 months will no doubt contain many surprises, twists, and turns, one thing is certain. Data will continue passing through the veins of business industries and economies.
With even more channels, at greater velocities, and containing even more insights, organizations will have no choice other than to evolve toward data-driven business models. The question for business leaders is: Will it be proactive and dynamic— or more reactive and involve playing catch-up?
Previous years may have meant a data deluge made it harder to crunch and extract insights, since it was during a time when big data challenges were more around storage and security. Things are now changing dramatically. We’re seeing more and more organizations starting to realize their data-driven potential. Successful use cases are broad and cross-industry. Customer experience teams are providing data-driven interactions. HR leaders are optimizing engagement and retention processes based on behavioral insights. Delivery departments are accessing real-time performance to innovate better and faster.
“By 2025, 30% of Gartner clients will protect their data using “A need to share” approach rather than the traditional “Need to know” approach.” (Gartner)
Of course, there are many challenges ahead. Although for organizations wanting to gain a data-driven competitive advantage, now’s the time to act. Here are five trends to help you choose where to start.
1. Cloud Data Governance
From remote working to AI, cloud continues to underpin the reshaping of modern business. More than 70% of organizations have migrated at least some workloads into the public cloud.
However, the race to become cloud-native isn’t without risks, ranging from budget overspend through to migration delays.
“Inefficiencies are costing the average company 14 percent more in migration spend than planned each year, and 38 percent of companies have seen their migrations delayed by more than one quarter.” (McKinsey)
Migration and ecosystem challenges will be heightened by the ongoing shortage of DevOps talent, particularly among highly regulated industries, where legacy and on-premises infrastructure figure highly, with disparate workloads less suited to a “lift and shift” approach.
Organizations will need to look at other ways to stay competitive, such as automation and self-service data analytics.
These cloud-based management systems offer a way to transform raw data and deliver it to the right users at the right time. Without the need for IT or data analysts to first prepare the reports.
Instead, vast volumes of data can be stored and accessed on demand. Beyond the traditional and static method of using a data warehouse, and instead bringing customizable dashboards for each individual user and their associated use cases.
Crucially, cloud-based services are now increasingly supported with AI and ML offerings. These unlock the potential for businesses to apply AI to optimize existing processes, such as by automating workflows.
The learning element can also be applied based on historical requests, ensuring a cycle of continuous improvement for modern data governance.
2. Adaptive AI
In today’s changing world, the concept of “Business As Usual” calls for increased flexibility, dynamism and readiness to adapt to survive.
Expect 2023 to demonstrate this through the rise of adaptive AI, where systems continually learn, adjust and retrain models, based on new data. It’s unlike the traditional and more static AI, where a human developer is required to update models and prevent them becoming outdated or obsolete.
With ongoing learning effectively “built-in”, AI will require fewer manual interventions. What’s more, the ability to adaptively learn from data will generate new insights to support executive decision-making, allowing businesses to introduce Applied Observability.
This is where AI-based decisions can be analyzed for further recommendations. A feedback loop can then be created to track previous outcomes. The resulting evidence-based insights can then be used to improve accuracy of predictions and inform future strategies.
“By 2026, enterprises that have adopted AI engineering practices to build and manage adaptive AI systems will outperform their peers in the number and time it takes to operationalize artificial intelligence models by at least 25%.” (Gartner)
Adaptive AI has the potential to solve some of the historical challenges arising from machine learning models where outliers often impacted training data, skewing results exponentially through every iteration instead of being disregarded.
Of course, the impact of a genuine novelty observation or real-world change may be easy to detect in a small dataset. Whereas such outliers are far harder to pinpoint amid the volumes required for AI.
Adaptive AI can therefore reduce such a risk of algorithmic bias. By dynamically adjusting processes, adaptive AI can also help businesses ensure more effective governance, by applying more intelligent automation.
3. Real-time Data
Data keeps businesses running, but real-time data provides the competitive edge.
From financial institutions trading by the millisecond to e-commerce stores approving payments and processing PII, further demand for real-time data will come from rising customer expectations, fueled by on-demand self-service experiences.
Creating real-time data pipelines can also reduce processing costs compared to batch data pipelines. Whereas batch data has to be repeatedly queried from the source, real-time only has to react to new data or events.
Some use cases only require batch-based pipelines for working with historical data. However, as datasets— and associated governance requirements— grow larger, many organizations will have to make some big infrastructure calls.
The scale of this evolution, coupled with the processing and power required, is why data analytics automation will play such an important role through 2023, from simple scripts that automate orders, through to complex algorithms that automatically detect anomalous or risky activities.
Organizations that can successfully harness automation will be able to boost productivity, uncover insights faster, and better manage complex variables. It just requires the right choice of platform where the data life cycle can be automated, but still provide a unified source of truth with the required level of visibility.
4. Data Access Governance
Data privacy, protection and governance are high on the to-do lists of governments around the world.
The EU’s GDPR, Canada’s PIPEDA, and China’s PIPL— these and others have shown that it’s possible to align legislation at scale. Such momentum puts data governance and data access control at the heart of business strategies for 2023.
“As of 2020, ten percent of the population worldwide had personal data covered under modern privacy regulations. In 2023, it is projected that a total of 65 percent of the global population will have personal data covered under privacy regulations.” (Statista)
When multiple business functions coordinate and align, there will be many opportunities arising from these trends throughout 2023 and beyond.
From an external perspective, demonstrating compliance can act as a brand differentiator to build trust among consumers. From an internal viewpoint, automated data governance and policy management boosts productivity across the business.
Employees can be freed to access the data they need— without having to manually check if they’re compliant. The data can arrive dynamically, for aggregation, sharing and integration with other BI tools.
Naturally, along with flexibility and robustness, it starts with the foundational requirement to comply with necessary regulations that are updated or call for more control over PII or transparency around bias-free algorithms.
Once a data protection framework is in place, data governance can become a competitive advantage where the focus is less on simply controlling the data, and more focused on the people who require the data.
5. Data Democratization
Demand for democratizing data will continue to rise during 2023, requiring businesses to move away from the traditional top-down approach to data governance.
Instead, the focus will be on getting data into as many (approved) hands as necessary. Rather than expecting human expertise to have to seek out the data (often through manual and lengthy processes and bottlenecks), compliant data will become more accessible and available on-demand.
This will mean business intelligence becomes more oriented toward self-service— rather than being the preserve of IT. Corporate culture will also change, with employees increasingly incorporating data into decisions and collaborations.
“Organizations have an ever-increasing appetite to leverage their data for business advantage, either through internal collaboration, data sharing across ecosystems, direct commercialization, or as the basis for AI-driven business decision-making” (Forrester)
From generating enriched data visualizations, to building applications, the rise in low code has shown what can be achieved by non-technical users.
Democratizing data— structured and unstructured— is a natural evolution of the process where usability is prioritized, alongside reducing the complexity and rigidity of traditional data governance processes.
Turning Data Trends into Stepping Stones for Success
Keeping up with trends has always been a central part of staying competitive. The five trends above form part of something more, though. They signal a structural, permanent shift. To a world where the concept of data— and data legislation— will constantly evolve. Leaving behind the days of gut instinct and HIPPO decision-making.
Building a dynamic form of governance, able to adapt to changing circumstances, is the bare minimum. Organizations require robust orchestration tools, along with automation to manage, cleanse and ensure integrity of the data.
As an aggregated and consolidated foundation, forming an all-encompassing “broad-spectrum” approach to data governance where data discovery and policy definition are combined (such as with encryption and tokenization as standard access control), data security platforms (DSPs) will therefore be crucial for business success.
Organizations also benefit from broad capabilities when adopting advanced analytics, cloud-based data lakes, and automation of policy enforcement. Granular and dynamic access controls can be implemented for data masking and governance. DSP-protected sensitive data can also be used directly, without masking.
