Who is responsible for Data Quality

The world is creating 2.5 quintillion bytes [yeah that’s a lot!] of data every day; so much that 90% of the data in the world today has been created in the last two years alone [www.sciencedaily.com]. All Information stored in the form of data is critical for any organization in terms of decision making, resource sharing and predictive analysis. Huge volumes of enterprise data stores in the form of data warehouses and now big data make data validation and integration testing a must for any organization.

The main challenge of Data Quality Management is that no single business unit seems to take the responsibility of ensuring data quality across the enterprise. Furthermore, once the data is fed to the web or high end servers, business units often hold IT responsible for anything that has to do with this data. What the business does not understand is that IT cannot create business rules to govern this data. This direction has to come from the top executives and other related management associates.

From what I have been reading [experimenting, researching, googling] about Data Quality so far the best I have come across is the Data Stewardship approach as proposed by Jonathan G. Geiger from a US based firm Intelligent Solutions, Inc. [I hope he is reading this and smiling]

So he tells us that effective data quality management requires an organization adopt a Data Stewardship approach. A steward is “A person who is expected to exercise responsible care over an asset that he or she does not own. The data is actually owned by the enterprise. The steward is responsible for caring for that asset”.

So we have a solution now; but the difficult part is establishing the data stewardship program. The key challenge is to identify the group or person who would assume responsibility for data quality management. The HR department would only want to be responsible for data concerning employees such as recruitment, appraisals and challenge management. Employees would only want to look after their immediate project data. The tricky part is dividing shared data sections such as payroll [financials about employees managed by HR].

A second challenge is to get key stakeholders to accept this as a functional responsibility. Marketing managers and sales personnel will have a hard time understanding IT jargon. Same would be the case for any other employee working in a non – IT department.

This brings us to the best practices that can be established in order to ensure success of data stewardship program:

• Establish a clear partnership between business and technology groups and then carefully select Data Stewards.
• Bring IT on board to establish a Data Steward Education Program which can help these people understand the key concepts involved in data modeling, data warehousing and database management.
• Ensure all Data Stewards have great communication and technical writing skills
• Create responsibility matrices and re-evaluate often
• Finally don’t forget to reward great Data Stewards!

Data Stewardship programs are tedious to implement and it’s surely hard to convince business departments to take responsibility for anything that is stored on machines! However if an organization manages to carry out a successful stewardship program the rewards are massive. A disciplined approach to data quality management in today’s competitive world can position any company to stay ahead of its competition. To ensure the success of such a program key stakeholders must be educated, a stewardship function implemented, and appropriate technology/software must be acquired!