Using Metadata to Drive ETL Processing
Richard Bartsch
Data Architect
Orbitz
Tuesday, April 25, 2006
2:00 pm - 3:00 pm
Level: Intermediate

This discussion describes the use of application metadata as the central element in the design of a system that transforms and loads raw medical claims files, replacing ad hoc, custom development for each new client. The system features metadata to implement the following:

- Shared file layouts
- Automated value translations
- Automated QA/QC statistics collection
- System based job scheduling

Reusable modules have eliminated all coding of QA/QC reporting and most ETL coding. Existing mapping documents were modified slightly to provide the source of metadata, making the business owners integral participants in the process and relieving IT of the task of translating non-standard documents to code.

The use of application metadata and indirect coding techniques have resulted in an application that allows quick implementation of new clients, accelerated processing of production data feeds, and the ability to scale as necessary to handle a large client load.

Mr. Bartsch has been involved in IT for 26 years, 11 of those as a consultant and 6 of those as an independent consultant. His experience spans application development through project management, with an emphasis on data.