Among all the rapid changes brought about by the pandemic, perhaps the most significant has been the emergence of data as a critical public good. However, data’s unique characteristics mean that its value erodes when it is seen as untrustworthy.
We’ve all seen the most successful companies and large enterprises fined large sums and suffer massive reputational damage as a result of failing to uphold the highest standards for data management and security. It has therefore become a strategic imperative for companies to ensure their data governance remains at the highest standards, yet flexible.
At a recent Economist Impact webinar with National Australia Bank, FWD Insurance, AEON and Databricks, we affirmed how robust data governance can only be derived from a deep understanding of privacy, security, and operational risks, and also driven in sync with ongoing changes across the compliance landscape. We also recognise the myriad of challenges standing in the way of good governance, from a misalignment between data management activities and operating models; a disconnect between legislation and security policies; and a lack of understanding as to how democratized data access generates business transformation.
The participants also discussed how data governance has become more complex for organizations operating across borders. The increasing complexities of cross-border data regulation is a pain point that requires more granular management, where data stores and business applications now have different types of data models, complexities, and security apparatus of their own in each market, along with the mandatory compliance and regulatory requirements.
Moreover, the need for the right engineering, analytics and data skills was spotlighted as essential to successful data transformation and governance. It was agreed amongst the speakers that not only do businesses need to hire the relevant talent, but existing employees should also be exposed to, upskilled, and educated on new, deeper knowledge, including how the flexibility of open-source tools can be used to unify governance across various disparate IT systems and to truly understand how data moves throughout their organizations.
Best practices to maximize data’s economic value
Weighing out the considerations from all these factors will be no easy task. Organizations must recognise that strong data governance can only be built when they approach it as a journey rather than a destination.
That’s where using the latest technologies to extract the most value from data can be extremely beneficial. Artificial Intelligence (AI) and Machine Learning (ML) are solutions that can incorporate privacy and security into data management models, process large amounts of data, and handle data on a large scale. New, emerging architectures, such as the Data Lakehouse, which combines cloud-native flexibility with AI and ML, have the potential to assess various and new types of risks as the macroeconomy progresses and matures, analyze how these market risks impact compliance and regulations, and deliver the necessary adjustments to enterprise data.
Moreover, companies should aim to groom and develop as many data stewards as possible, and this can be achieved through two-way collaboration in the form of public-private partnerships. Such a goal will allow good governance frameworks and practices to be distributed and decentralized in every department, yet still tightly integrated across the organization. These efforts also need to be supplemented by embedding privacy by design across all platforms, operations and processes of the business, as increasing volumes of data get moved, stored, and managed by internal and external stakeholders.
We recognise that companies are under pressure today to continuously rethink their infrastructure but at the same time also need to manage emerging risks that come with rolling out transformation strategies. As a result, key initiatives like moving to the cloud and decommissioning legacy platforms are often approached with a sense of caution and dread. However, data management and governance must be perceived as key components of business growth, and open-source platforms exist as essential tools that can enable the right balance of governance versus accessibility and use.
Watch 60 minutes full content on “Making every byte count: maximizing value by modernizing data governance“
