Disaster Recovery for Brokerage Firm’s Sizzling-Hot-Takeover

Problem:

A leading brokerage firm located in the Midwestern U.S. needed to implement data replication to provide a redundant active/backup HPE NonStop Server system for a sizzling-hot-takeover in the event of a primary system failure.




Shadowbase Solution: Active/Almost Active Backup Disaster Recovery

async-sizzling-hot-takeover-szt

  • In this configuration, the active and backup nodes are identically (or similarly) configured with active bi-directional replication used to maintain synchronism between the two databases.
  • Though the applications are active on both nodes, client interactions (broker requests) are only directed to the primary node.
  • If the primary node fails or is taken offline, routers switch the clients to the other node within a sub-second time frame.
  • Takeover is virtually instantaneous and imposes very little impact on user processing, since the application is already active (up and running and waiting for requests) on the backup node.
  • It is impossible to tell which node is the primary node and which is the backup node. They are both similarly configured, with all processes active at all times (databases opened read/write, external connections enabled, etc).
  • To test the system’s failover capability, management often forces a router switch to reverse the nodes.
  • It is extremely important in a failover architecture to actually test the disaster recovery plan to ensure that it works, that the disaster recovery plan stays up-to-date, and that the operations staff is well-versed in the process.  Periodically switching the polarity of the nodes accomplishes this test.

Additional Shadowbase Benefits:

  • Offloading Queries/Reporting from Production and Avoiding Planned Outages
    • The brokerage firm also takes advantage of off-loading queries and reporting from the active node, which reduces the load on the active node and utilizes the capacity of the stand-by node for productive work.
    • When the brokerage firm needs to upgrade its application, database, or perform other forms of changes that would normally require production downtime, it can avoid the planned outage by leveraging Shadowbase bi-directional replication capabilities to keep the databases synchronized, even if the data formats/schemas are changed.  This is called a Zero Downtime Migration (ZDM).
    • When such an upgrade is needed, the router feed is switched to the standby node, the original node is downed, replication to the downed node is stopped, the downed node is upgraded and restarted, and Shadowbase replication is then used to resynchronize the downed node’s database after it is brought back online in preparation for a switchover.
    • The changes can then be rolled through to update the standby node.  Please visit our page on HPE Shadowbase Zero Downtime Migration for more information.

Contact us for more information on this Shadowbase solution.


The above was adapted from the book: Breaking the Availability Barrier, Volume III: Active/Active Systems in Practice by Paul J. Holenstein, Dr. Bruce Holenstein, and Dr. Bill Highleyman.