So you think you have a data quality problem, or you have been told that you have a data quality problem. Let me tell you, you don’t. Even with the fanciest data quality tool, etl engine, or business intelligence application; your data quality problems are not going to go away.
In Reengineering the Corporation, Hammer states:
Seemingly small data quality issues are, in reality, important indications of broken business processes
I completely agree.
In the Kimball Group Reader, Kimball responds by saying:
Not only does this insight correctly focus our attention on the source of data quality problems, but it also shows us the way to the solution
So when you hear people say they have a data quality problem, you know that they have a business process problem. The challenge is what to do about fundamentally broken business processes, since you are the business intelligence / data warehouse person. This opens up a whole new world for BI professionals. You may be asked to spend significant time involved in business process improvement so that the output of the business process is quality data that can lead to improved decision-making. At minimum you need to educate executives in your organization that business intelligence can not be successful without quality data. Specifically, that poorly designed business processes result in poor data quality. Hopefully the executives at your organization will understand and focus efforts on improving business processes. Only then can a successful business intelligence project begin.
You have to be careful to do due diligence on every candidate business intelligence project so that you have confidence that you have quality data to work with. If you find that you don’t, you need to educate the sponsors of the project on the importance of quality data, and why this candidate BI project should be put on hold until the business process issues are resolved. You just have to make sure you don’t say “no” to every candidate BI project. If you do, you may be out of a job.
If you find that poor data quality is a constant problem at your organization you may want to acquire some skills on business process improvement so that you can help solve the problem. A great book on redesigning business processes is Workflow Modeling: Tools for Process Improvement and Application Development.
John Levandowski 