HPE Shadowbase Streams for Data Integration

HPE Shadowbase Streams is a powerful and flexible facility that enables diverse applications to interoperate with each other at the data level (called data integration). This capability is provided even for legacy applications that were never intended nor designed to work together. HPE Shadowbase Streams focuses on distributing information in real-time throughout the enterprise, enabling the provision of a Real-Time Enterprise (RTE), a concept in business systems design focused on ensuring organizational responsiveness. It rapidly delivers information where and when it is needed without customer application modification. The application’s responsibility is simply to make functional use of the data delivered by Shadowbase Streams.

With data integration, HPE Shadowbase Streams creates a real-time copy of selected data from a source system’s database in a target system’s database. Target system applications may use this copy of data updates from the source application for expanded functionality. Therefore, integration using Shadowbase Streams typically involves sending a source application’s database changes from one environment to a target application’s database in another environment.

The Shadowbase Streams data distribution process is shown in Figure 1. Application and database components are connected by communication channels. Shadowbase Streams seamlessly moves selected data in real-time across this communication channel from a source database to a target database, where it can be used by a target application. As changes are made to the source database, they are immediately collected via Shadowbase change data capture technology, and replicated to the target database to keep it synchronized with the source database. (For this reason, data integration is often called data synchronization.) Data replication is transparent to both the source application and to the target application. Upon delivery, the target application can immediately make use of this real-time data.

Diagram of Shadowbase Streams data distribution process. Please see the paragraph that starts with "The Shadowbase Streams" for a full image description.

Figure 1 — HPE Shadowbase Streams Data Distribution Process

Shadowbase Streams uses the transaction log of the source system to obtain the database updates that are being replicated, as shown in Figure 2. In a transaction processing system with data synchronization configured, at least one application and database exist on both the source and target systems. A source application makes database changes to a source database, and a transaction manager writes every database change to a transaction log. (Transaction managers use the transaction log to reconstruct the database if it becomes lost or corrupted.) Examples of transaction logs are the TMF Audit Trail in HPE NonStop systems and the redo log in Oracle databases. HPE Shadowbase Streams reads the TX log and sends the database changes across a communication channel to the target database. Source database changes are immediately made available to the target application, which may read them from the target database and execute any appropriate business logic as a result. If a transaction log is unavailable, Shadowbase Streams can also be driven by database triggers or by an application change log.

Diagram of Shadowbase streams for data integration. (Please see the paragraph that starts with "Shadowbase Streams uses" for a full image description.)

Figure 2 — HPE Shadowbase Streams for Data Integration

Therefore, Shadowbase Streams has real-time access to database updates as they occur. As soon as Shadowbase Streams detects a database update, it replicates that update to the target database. In the process, it makes any necessary format changes to the update in order to meet the needs of the target system. In addition, it can filter updates and can eliminate updates that are of no interest to the target system.

Using HPE Shadowbase Streams for data integration introduces many benefits to the enterprise by integrating diverse databases, replicating changes to multiple databases, consolidating data from several sources, and more.

Shadowbase Streams supports many models for data distribution, including:

  • maintaining a remote database of critical data
  • sending critical data to client applications or directly to servers via queues, interprocess messages, TCP/IP sessions, or client APIs
  • publishing data to applications that have subscribed to the data
  • responding to poll queries from client applications

Shadowbase Streams is extensible. It allows the user to embed custom-processing logic into the data-processing path. It readily filters, transforms, and adapts data from one application or database environment into the protocol or format required by another application or database environment, all without requiring any changes to the existing source application or database.


Related Solution:
Related White Papers:
Related Case Studies:
Related Solution Brief: