Before a database is to be of any use, it must first be loaded with data. Shadowbase Online Loader (SOLV) and Shadowbase ETL are utility products designed to perform this task, providing online and offline loading (or unloading) of a source database into a target database. Both of these products are included with each Shadowbase license. Shadowbase SOLV loading and Shadowbase ETL may also be licensed separately if the other data replication features of Shadowbase are not required.
The Shadowbase Online Loading (SOLV) utility provides online loading of a source database into a target database, meaning that the source and/or target databases can be open for reading and updating while the load (or copy) occurs. In contrast to other loading products, there is no need to take either the source or target databases offline, making them inaccessible to applications, while the load (or unload) occurs.
The Shadowbase SOLV utility can load audited and non-audited HPE NonStop Enscribe source files and HPE NonStop SQL tables into any target environment and database combination supported by the Shadowbase line of data replication products (e.g., HPE NonStop Enscribe or HPE NonStop SQL targets, or other server targets such as Oracle, SQL Server, DB2, and MySQL). The Shadowbase SOLV utility can perform the load while regular replication is also taking place, consuming the source database changes and merging them with the data being loaded to keep the target fully synchronized with the source while the load takes place.
The Shadowbase ETL toolkit uses and extends the SOLV loading capabilities to allow for reading and injecting events from flat files into the Shadowbase replication engine for processing, as well as producing flat files of database data or database change events that can then be subsequently processed by an ETL tool. The Shadowbase ETL Toolkit is used to extract database changes or initial load data into flat files for subsequent ETL loading into a data warehouse using comma-separated value (CSV), fixed-position, and tab-delimited formats. It is particularly useful for feeding high-volume quantities of loading data into a data warehouse, and when feeding row-based data into columnar databases.
More specifically, for certain applications, you may want to use an existing vendor’s ETL loading utility to load either initial data, or change data, into a data warehouse. The Shadowbase ETL toolkit allows a Shadowbase user to perform these tasks. For example, when coupled with the Shadowbase SOLV loader, the Shadowbase ETL toolkit can be used to extract select data from a source database, transform/cleanse it into the proper format, and save it into flat files that your ETL loading utility can understand and load into a target database environment not directly supported by Shadowbase replication. Using the Shadowbase ETL Toolkit, you can also periodically extract source database change data from the source database’s audit trail (change log), and process it into a flat file format that can be incrementally loaded into the target environment using a micro-batch incremental update approach to keep the target environment synchronized with the source.