When trying to understand your data environment, you should focus on the following objects:
- Reports - anything that presents data for consumption. This can be reports, dashboards, data visualizations, excel files, and more.
- Data Sources - anything that stores or creates data. This can be tables, views, cubes, extracts, and more.
- Data Movement (Jobs) - anything that moves, transforms, manipulates data. This can be ETL jobs, Kafka producers, stored procedures, and more.
- Business Terms - all definitions of key metrics or attributes to drive consistent understanding.
|Report||Data Source||Data Movement|
As mentioned in the series "Enabling Data Driven Organizations", having an inventory of all of these object types across all platforms within the organization enables a lot decisions. Before we can get to those decisions, this exposure is a critical step. A public exposure of this content can be seen as airing the "dirty laundry" of reports, data sources, data movement, and business terms. This in itself is very powerful as there are likely way too many reports, way too many tables, and more. The mere public exposure of this inventory will motivate your company to manage its inventory. This is an effective way to motivate teams to clean up and self manage. This I find is much more effective than a top down governance mandate.
The exposure of this inventory can help drive the following:
- Counts of Inventory - this allows you to know how many reports, data sources, and more are within the company.
- Metadata Coverage - this allows you to know how much metadata (descriptions, owners, etc.) is available within your inventory.
- Activity Progress - this allows you to show how much content is getting created (or deleted) over time which is a measure of your data team's activity.
Bottom line, like the levels of data needs, exposing the inventory can motivate teams, build a base understanding of content, and measure activity/impact of teams. This is the first step in "Enabling Data Driven Organizations".