|
Leveraging Meta Data to Automate the ETL Process |
![]() Tom Harrocks
Senior Principal
Knightsbridge Solutions |
|
Monday, April 24, 2023
5:00 pm - 6:00 pm
Level: Intermediate
A common goal of enterprise data warehouse (EDW) projects is to use meta data to facilitate the ETL. By defining business rules within a rigorous technical framework, and using coding templates for data manipulation, significant proportions of the ETL code in a data warehousing project can be automated. Automation reduces maintenance costs, improves code quality and provides higher precision testing. A major health insurer was concerned with providing a basis for automated code generation while retaining business value. The insurer developed a framework for storing enriched technical meta data to support business rules, semantic translations and data lineage. The accuracy and completeness of the technical meta data was ensured by providing a migration utility to import source and target database schemas. An interface was developed that associated business rules with a specific source to target transformations. This innovated approach created dramatic efficiencies in creating and maintaining accurate meta data. The organization reduced development time by 50% and ongoing maintenance by 25%. In this case study-driven session, learn how to:
Mr. Harrocks is a senior principal with Knightsbridge Solutions. He has over 20 years experience delivering enterprise information integration solutions. His specialty is the development of highly configurable solutions that focus on configuration efforts to develop reusable patterns. Mr. Harrocks has 15 years experience providing integration solutions in the development of analytical systems in the pharmaceutical and healthcare industries. He has also authored published articles in scientific academic journals.
|