Knowing Your Data

Is it possible to be an effective data professional without knowing anything about the domain of data you support?

I remember having a conversation a few years ago with a friend of mine, a fellow data practitioner. In his role as a database administrator, he asserted, he could do his job without knowing anything about the underlying data. His argument was that most tasks required of a DBA – including backups, corruption checks, and performance monitoring – could be done generically without requiring any knowledge of how the data is used or any of the business processes around the data.

Knowing Your Data

I can agree that some of these things are fairly generic. After all, every database should be on a backup rotation, and there aren’t that many different variables that go into performing a backup. However, the more generic the process, the easier it is to outsource or automate. With the advancements in automation and machine learning of late, those tasks that can be streamlined through systems or software will no longer require human hands, thus doing away with jobs that require little analysis or data domain knowledge.

I will unashamedly admit bias here, based on my work on business intelligence projects for most of the last ten years. Having built hundreds of data warehouse and ETL solutions, I can say with certainty that almost all of those required having an understanding of the underlying data and the processes that generated and consumed that data. Simply knowing what the metadata looked like wasn’t enough; most of these types of projects required a lot of business process knowledge and an understanding of how the data would be used. For each of these data warehouse and ETL projects, I could have delivered a solution without knowing something about the domain of data or how it was being used, but it wouldn’t have been a great solution.

Be a Business Professional First

Most all of us have tasks we perform that could be done by most anyone with a similar skillset and without domain-specific knowledge of the data, but I’d wager that this represents a small part of what most data professionals actually do. None of should be just a DBA, or just a developer, or just an analyst. We should be business professionals first and foremost, with a big-picture understanding of what’s happening with the data we support.

Yes, it is possible to survive as a data professional if you don’t know about the data you support or how it is used, but the future is limited for that type of job. Tomorrow’s data practitioner will need to understand business processes and usage patterns as well as technical details.