Lies, damn lies and statistics

Entries for tag "data"

Data Autonomy - Case Study

A couple of contracts back I was consulting as a solution architect at a national retail organisation I ran an experiment as a proof of the cadence which is possible using Data Autonomy.

Shortly after the project went live, I decided to build it again myself using Data Autonomy as realistically as possible. The result was a far better solution in a third of the time & cost.

Data Autonomy - BI & Analytics

BI & analytics loves Data Autonomy and event driven architecture. In the operational side of Semantic Hub / Data Mesh, data is already clean and in business form. Data engineering can subscribe to all significant business object changes and metrics can be automatically calculated live. Dimensional modelling also becomes a much simpler process as canonical data is available near real time.

The data engineering and integration teams can become an active part of data governance and data stewardship. Working closely with the business domain SMEs, everyone is on the same page and reporting is simplified.

Data Autonomy - Holistic Data Mesh

Standard event driven integration practice is to take data from a system, transform into a middle model and then transform to each destination. Data Autonomy simply says that while we have the data in the middle form, lets save it.

Each business domain must be free to evolve and mature independently as the view of significant business objects evolves. The data governance group with data stewards and SMEs should be responsible for producing domain aligned data product definitions which are then realized into data products. The features of these data products should cover interfaces and persistence for both operational and analytical data.

Data Autonomy Overview

Data Autonomy is a holistic data strategy which is based on the rationale discussed in Vestigial Technology. This is an overview of the concepts.

The purpose of Data Autonomy is to counteract increasing costs and loss of flexibility in IT as the business grows and changes. Data Autonomy limits IT complexity by providing loosely coupled, consistent, accessible, and secure information anytime, anywhere, on any device.

Data Autonomy works best in conjunction with modern IT practices like automated regression, DevOps and infrastructure as code. We must embrace change and be good at implementing small changes regularly.