Data Governance – Nuisance or Opportunity
With the increasing volumes of data that can be cost effectively stored in the cloud, comes increasing responsibility. The current trend in the industry shows increasing regulatory requirements for; privacy, storing data, managing the lineage of data and securing the data sufficiently.
Whilst Data Governance is not a new concept it is increasingly more relevant in modern enterprises. Especially, since the more stringent requirements of the European, General Data Protection Regulation (GDPR) have taken effect. Companies have found themselves needing to review and update their processes and security frameworks.
In many cases, this has caused organisations to struggle with compliance and many moan about the effort and cost that it takes to implement the new processes and safeguards. Many regard the new regulations as too stringent and see the implementation as a nuisance that should be kept to the bare minimum.
However, others have turned a problem into an opportunity. As we are entering the third wave of business insights, data storage and analytics are moving into more de-centralised silos that are closer to production and transactional systems. This data is only loosely coupled and is left mostly in its original format allowing a reduction of complex and expensive ETL processes.
This has multiple advantages. Business insights can be surfaced much faster and directly integrated into the business process. The other main advantage is that instead of moving data into a centralised Data Warehouse it can be processed at the point of acquisition. This directly provides data driven benefits to customers and frontline business units, reducing the data cycle to seconds and minutes rather than hours and days.
The result is that businesses can create better services and improved sales opportunities. Many organisations have used this to disrupt established markets to the point that companies following traditional business models struggle to keep up.
Decentralisation of data responsibility into business domains requires a Governance Model that allows decentralised data management and centralised oversight.
Cornerstone of this model is a well-designed architecture that creates the foundation of how data is stored, what tools are available and how overall security is implemented and managed.
When we talk about decentralised data responsibility we do not mean that the data is necessarily stored in different locations. Cloud computing makes data increasingly location agnostic. What we mean is that instead of having a centralised BI team that owns all the data and business insights, individual business domains take responsibility of the data. This is achieved with Data Stewards assigned to the business domains.
Data Stewards are responsible for their data domain, maintaining the inventory of data types and elements. Updating a centralised data catalogue that not only contains all meta data for individual data domains, but that also pro-actively suggests connections between domains is a key to a successful governance framework.
Data Triage is becoming an important part of managing enterprise wide data connections. This is the responsibility of a Data Council, consisting of the Data Stewards from all domains, to determine and govern enterprise wide policies, standards and processes especially for data that goes across boundaries.
How does this help the organisation? It addresses a number of issues that companies face with the massive increase of data volumes in recent years. Firstly, they can gradually move away from expensive monolithic data warehouse solutions and achieve a best of breed architecture that keeps them current in the fast changing technology landscape of modern analytics platforms.
Secondly, they resolve many issues with domain oriented analytics solutions built by business units that currently sit in spreadsheets and MS Access tables. Whilst this might not remove these it helps to better integrate these into a wider framework or better yet, remove the need as a faster time to market can be achieved with Domain ownership of data.
Finally, it provides a more comprehensive data management governance structure, as the ownership of data is decentralised with a centralised Governance model that allows the building of well designed controls into the organisation without putting up too much red tape or creating delivery bottlenecks in a centralised team.
Let Fusion Professionals work with you to develop the most appropriate Data Governance framework and strategy for your organisation.
Achim Drescher is the Managing Consultant of the Big Data and Analytics Practice at Fusion Professionals.
With 30 years in the IT industry, he is an Expert in Enterprise Software and Data Architecture, Data Governance frameworks and modern analytics platforms for Big Data and Data Lakes.
Many organisations don’t realise it, but in our current environment Data has become the main differentiator in the market. Most…MORE INFORMATION
Professional services, one of the fastest growing sectors of the Australian economy, covers a broad group of companies and organizations…MORE INFORMATION
We experience an increasing polarisation in our political landscape with tribalism becoming a real issue. This is partially to be…MORE INFORMATION
Oracle’s introduction of the self-driving, self-securing, and self-repairing Autonomous Database draws upon its decades of expertise in automating databases and…MORE INFORMATION
In a recent blog post from Dataiku, the leading data science, machine learning, and AI platform, Lynn Heidmann explored ways…MORE INFORMATION
“With Great Power Comes Great Responsibility” One of the biggest ongoing responsibilities that comes after commissioning an Exadata appliance is…MORE INFORMATION
According to Constellation Research, a little more than half of traditional Fortune 500 companies have disappeared due to the lack…MORE INFORMATION
Fusion Professionals has signed a partnership agreement with Dataiku, one of the world’s leading machine learning platforms that moves companies…MORE INFORMATION
Statistical language models apply probability distributions to a sequence of words. These models are finding increasing use as natural language…MORE INFORMATION
Challenges The Company, one of Australia’s largest and fastest growing Telco companies had 2 primary SharePoint environments that had different…MORE INFORMATION
Containerization allows applications to run on any machine- anytime, anywhere so long as they are compatible. By virtualizing your OS,…MORE INFORMATION
So you’ve finally decided that the cloud is safer than corporate data centers and digital assets and you’ve chosen to…MORE INFORMATION
Building a system that houses your organisation’s data can be daunting, especially now that data acquisition is growing rapidly. The…MORE INFORMATION
Human-to-machine communication has not yet been perfected, but enterprises are already beginning to integrate this groundbreaking technology into their operations,…MORE INFORMATION
Fusion Professionals has signed a partnership agreement with MapR Technologies, provider of the industry’s leading data platform for AI and…MORE INFORMATION
“Big data is at the foundation of all of the megatrends that are happening today, from social to mobile to…MORE INFORMATION
In recent years data volumes have been increasing dramatically. This has created major challenges for traditional analytics platforms in terms…MORE INFORMATION
With the increasing volumes of data that can be cost effectively stored in the cloud, comes increasing responsibility. The current…MORE INFORMATION
With the advancement of technology and abundance of data your business receives on a daily basis, companies are now in…MORE INFORMATION