Successful Data Quality Monitoring and Data Governance

Murali Doddapuneni, IAIDQ Member
Data Quality Analyst
Aera Energy LLC

Wednesday 2:00pm - 3:00pm

Level: Intermediate

Successful data quality monitoring and data governance are helping our company achieve near-zero defects for our most critical data. Several teams consisting primarily of data creators (business staff) assisted by the appropriate IT support personnel have been set-up to assure the quality of the data they are accountable for, throughout its lifecycle. Each team focuses on a specific data family and oversees the relevant data capture system and the specific warehouse database tables that contain that data. To support the teams, we have implemented a robust DQ monitoring system to measure and report data quality levels. At the core of the system, queries are built to check data against more than 700 pre-defined data quality rules. Error reports are generated daily or weekly and automatically distributed to the right people in each team. Standard data quality metrics are updated automatically and reviewed weekly by each team and monthly by business unit managers. This presentation describes:

1. The architecture and components of the DQ Monitoring system (record the DQ rules, fire the QC queries, capture, transfer, aggregate and report the data quality data)
2. The governance structure adopted to establish the proper accountabilities across business units and through the hierarchy
3. The processes implemented to ensure errors are eliminated and prevented
4. Success factor, lessons learned, and future direction

Murali Doddapuneni currently works as data quality analyst at Aera Energy LLC, where his responsibilities include data quality assessment, data conversion, BI and application support. Prior to joining at Aera, Murali was data warehouse consultant for 5 years. He is a member of IAIDQ.