Sunday, February 28, 2016

Data Exposure Drives Motivation

As part of the first post in the series "Enabling Data Driven Organizations", the focus of this post is about exposure of the data inventory.  To be a data driven organization, you not only need to know what data you need to run your business, but you also need to know what data is currently available, how to get it, and where it is being presented.  This is where the "data inventory" comes into play.

When trying to understand your data environment, you should focus on the following objects:

  • Reports - anything that presents data for consumption.  This can be reports, dashboards, data visualizations, excel files, and more.
  • Data Sources - anything that stores or creates data.  This can be tables, views, cubes, extracts, and more.
  • Data Movement (Jobs) - anything that moves, transforms, manipulates data.  This can be ETL jobs, Kafka producers, stored procedures, and more.
  • Business Terms - all definitions of key metrics or attributes to drive consistent understanding.

ReportData SourceData Movement

As mentioned in the series "Enabling Data Driven Organizations", having an inventory of all of these object types across all platforms within the organization enables a lot decisions. Before we can get to those decisions, this exposure is a critical step.  A public exposure of this content can be seen as airing the "dirty laundry" of reports, data sources, data movement, and business terms.  This in itself is very powerful as there are likely way too many reports, way too many tables, and more.  The mere public exposure of this inventory will motivate your company to manage its inventory.  This is an effective way to motivate teams to clean up and self manage.  This I find is much more effective than a top down governance mandate.

The exposure of this inventory can help drive the following:
  • Counts of Inventory - this allows you to know how many reports, data sources, and more are within the company.
  • Metadata Coverage - this allows you to know how much metadata (descriptions, owners, etc.) is available within your inventory.
  • Activity Progress - this allows you to show how much content is getting created (or deleted) over time which is a measure of your data team's activity.

Bottom line, like the levels of data needs, exposing the inventory can motivate teams, build a base understanding of content, and measure activity/impact of teams.  This is the first step in  "Enabling Data Driven Organizations".

Sunday, February 21, 2016

Enabling Data Driven Organizations

With the previous series that was focused specifically Data Needs and Making it Useful, this new series is taking a bigger look at "Enabling Data Driven Organizations."  With organizations often having several teams and technologies that deliver on data needs, as time goes on this builds a massive inventory of data deliverables.

With this massive inventory of deliverables, organizations have a treasure trove of information that is often right under their noses.  When focusing on data, organizations often focus on Sales, Traffic, Daily Visits, and other key performance metrics, the treasure trove of data that is often just as valuable is what is available within the organization that has already been built.  What reports, tables, data flows, and terms exists within the organization.

Why is this information so difficult to find?  

Like many organizations, this inventory is often separated across the different silos of teams as well as different technologies.  How many companies have a single database or a single reporting tool?  While many technologies claim to be "Enterprise BI" tools, the truth of the matter is that there is often not a single technology across the entire organization.  Understanding that this is impacted by both process and technology, this can be a tricky problem to solve.

In the following posts, I plan to focus on:
  • Exposure of Inventory - as we all know, getting access to the data is the first step of data needs. 
  • Understanding Value - after getting access to the inventory, understanding the value of this data is critical.
  • Influencing Change - after understanding the value of this data, focusing on changing how the organization approaches problems and utilizes data "on data" to make decisions.
  • Building for the Future - after the organization understands and utilizes this information, how can you make this engine solid for the future?

I am excited to share my experiences and where I feel this can "Enable Data Driven Organizations."