- About
- Solutions
- Essentials
- Utilities
- Publications
- Product Delivery
- Support
Either method can be used to replicate the source data to the target database. For example, SOLV can copy the source data into the target database to create an initial copy of the source data in the target database. Subsequently, Shadowbase Real-time Replication can read the audit trail for the source data changes and replicate them into the target database to keep the target database synchronized.
Shadowbase SOLV supports snap-shot replication for loading non-audited Enscribe file and SQL table data into a target database.
Shadowbase software cannot replicate individual DML changes made to a non-audited file or table (with one notable exception, see below). In order to replicate the DML changes for non-audited files or tables, the HPE NonStop AutoTMF product is required (and must be properly configured for the file or table).
Exception: For certain application-maintained non-audited “log” files or tables, where the DML changes are always INSERTs (appends) at the current end-of-file (EOF), Shadowbase SOLV can periodically “chase” the file or table’s EOF and replicate the records/rows that are appended into the source file (or table) to the target database. In this specific situation, HPE NonStop AutoTMF is not needed for Shadowbase software to replicate the inserted/appended data.
For more information, see HPE Shadowbase SOLV Manager (below).
HPE Shadowbase SOLV Snapshot Loading, also known as HPE Shadowbase point-in-time loading, is shown in Figure 1.
When the source file/table is non-audited, HPE Shadowbase SOLV Snapshot Loading can be similarly used to copy the source file/table data into the target database. However, since the source file/table is non-audited, Shadowbase software will not see the subsequent changes made to the source file or table (as they do not appear in the TMF audit trail). To pick up subsequent changes made to the non-audited source file or table, another SOLV Snapshot Load would need to be run (see below for alternative methods to replicate the subsequent changes).
Discover SOLVFigure 1 illustrates SOLV. This technique reads and replicates TMF-audited or non-audited source Enscribe files and SQL tables into a target database. When the source file/table is audited, Shadowbase software uses real-time replication to replicate subsequent changes made to the source file/table to the target database to keep it synchronized.
Figure 1 — HPE Shadowbase SOLV Snapshot Loading
HPE Shadowbase SOLV Manager (SOLVMGR, previously called “File Chaser”) can be used to replicate non-audited file/table changes. This approach works for application environments that create and maintain a log file containing the change events to be replicated (similar to an audit trail of the change events).
SOLVMGR and SOLV can be configured to “chase” the EOF of a file, and replicate the newly appended data into the target database in near real-time. SOLVMGR acts as the manager for SOLV and restarts SOLV after it stops for any reason. For example, SOLVMGR completes a load by hitting EOF or when the SOLV process prematurely stops due to system issues. When run in this mode, SOLV replicates all the data in the file up to EOF and stops; SOLVMGR sets a delay timer, delays, and then re-runs SOLV to pick up and replicate any newly-appended data in the file using an “EOF chaser” cycle. (The delay timer is configurable and can be very short.)
SOLVMGR and SOLV can replicate the file sequence. SOLVMGR uses special logic that looks at the file creation information for the sequence of files to know when to roll from one file to the next in sequence. For example, if the application has a fileset called FILEnnnn, where the application fills FILE0000 first, then it fills FILE0001, followed by filling FILE0002, etc., SOLVMGR will watch for the application rolling to the next file in sequence, and then instruct SOLV to chase the newly appended data in that file.
Figure 2 illustrates SOLVMGR.
For applications that create and maintain their own log files/tables, SOLVMGR can be configured to read these application-generated log files/tables, optionally transform the events, and inject the events into the replication stream to the target environment, as the application appends data to the log files/tables. This technique avoids the need for the source database to be audited, while helping to ensure that such changes are also replicated.
During configuration, SOLVMGR can be given a range of sub-volumes and application-maintained files/tables to monitor for changes. This is useful, for example, when the application creates a sequenced list of log files and periodically rolls through them as it fills them. As the application appends change-data into these log files/tables, SOLVMGR will see those changes, and periodically read/inject them into the Shadowbase data replication engine for processing.
Figure 2 illustrates SOLVMGR. An application makes database changes to non-audited Enscribe files and SQL tables, and generates log files of these changes (business transaction data). SOLVMGR periodically launches SOLV loaders, which read the log files in sequence from a saved “restart point” as each successive SOLV loader runs. This step allows for optional and periodical transformation and maintenance of the application-generated log files/tables. The SOLV loaders inject the log data into the replication stream, which is made available to any Shadowbase target environment/data replication system for processing.
Figure 2 — HPE Shadowbase SOLVMGR Architecture
SOLVUTIL is available for loading, copying, validating, and file chasing Open System Services (OSS) Regular Files (or “Flat Files”) that have data inserted at the EOF in append mode. (Note: the OSS Regular Files are never TMF-audited.) SOLVUTIL does not replicate subsequent updates (to the previously inserted data).
SOLVUTIL is illustrated in figure 3. SOLVUTIL currently requires Expand when replicating between systems. (Contact Gravic if you are interested in TCP/IP support.) SOLVUTIL can track updates to multiple source files and maintain their sequence when replicating into similar target files. SOLVUTIL has numerous parameters, is configured with TACL environment variables, and can handle massive data volumes.
SOLVUTIL can also compare two OSS Regular Files. It confirms that the files match or lists the offset of the first byte of the mismatch. SOLVUTIL can also be used to correct mismatched data in the target file by overlaying the mismatched target data with the correct data from the source file.
*Note: SOLVUTIL Verification requires an HPE Shadowbase Compare license.
Please reference: SOLVUTIL Manual
Figure 3 shows an application inserting data to an OSS Regular File in “append mode” (data is periodically added to the EOF) . When SOLVUTIL sees that the source file has changed, it then chases the EOF of the source file, and in near-real time replicates the newly inserted data to the target OSS Regular File. SOLVUTIL inserts the data into the same relative position as it was on the source file. SOLVUTIL replicates all the data in the file up to the EOF, sets a delay timer, delays, and then checks for and replicates all newly appended data (since the old EOF). The delay timer is configurable.
Figure 3 — HPE Shadowbase SOLVUTIL Snapshot Loading
HPE NonStop AutoTMF is a product available from HPE which utilizes non-intrusive intercept technology to automatically invoke TMF transaction protection for applications that do not use TMF when updating data files. This technology enables non-audited applications running on a NonStop system to access and create audited Enscribe files and to perform I/O against those files without program modification or recompilation.
AutoTMF software dynamically determines effective and efficient TMF transaction boundaries and automatically consolidates multiple database updates, ensuring low overhead. There is no need to make any source code changes to programs, or even to recompile them, since AutoTMF works with existing application object files. AutoTMF–enabled applications use audited operations, which adds performance optimization benefits, such as advanced file buffering and elimination of the block-splitting overhead associated with insertions to a file. Another benefit is improved data consistency, which keeps the base file and its alternate key files consistent with each other regardless of the outcome of the transaction or if there is a file system failure.
HPE NonStop AutoSYNC is a product available from HPE which replicates non-database files such as program object code files, command scripts, OBEY files, etc., to a target environment, usually for NonStop to NonStop business continuity purposes. It can be configured to periodically check for files that need to be replicated so that the target will always have an up-to-date copy in case of failover. AutoSYNC is a powerful addition to a business continuity solution when non-database files need to be replicated.
Copy link to clipboard
Email link
Print