Method and system for developing extract transform load systems for data warehouses

DETAILED DESCRIPTION Embodiments of the present invention include a method of developing an extract/transform/load (ETL) system in which one or more application programs perform a sequence of operations to transform data prior to loading the data into a data warehouse.
In an embodiment, the invention receives a data flow diagram having one or more processes and one or more input and output data store formats associated with each of the processes.
For each process, the invention parses the input and output data store formats associated with the process to create a step metafile for the process.
The invention also generates a description of the data flow for all the processes and stores the generated description in a control metafile.
According to an embodiment, the invention supports the development of data warehouses with minimal or reduced development cost and design time.
In particular, the invention includes a set of tools, a process, and a notation for specifying an ETL system that enables the design of the ETL system in a relatively short amount of time (e.
g.
, two weeks).
In accordance with one aspect of the invention, a method develops an ETL system.
The ETL system includes a sequence of operations performed by at least one application program.
The method includes generating a metafile for use by the application program in the ETL system for controlling the operation of the application program.
The generating occurs in accordance with a data flow diagram describing the ETL system.
The method also includes generating an installation script for installing the generated metafile and the application program on a computer such that the installed application program executes per the generated metafile to implement the ETL system.
In accordance with another aspect of the invention, one or more computer-readable media have computer-executable components for developing an ETL system.
The ETL system includes a sequence of operations performed by one or more application programs.
The components include an interface module, a process module, and a flow module