Data Validation — a Must-have for Mission-Critical Databases

In Business Continuity architectures, consistency between the production and backup systems is paramount. After all, if the backup system doesn’t match, isn’t it essentially worthless?

Unfortunately, there are several common industry misconceptions and pitfalls regarding this topic. We’ve heard things like:

“I have a replication engine, therefore both systems and their data match.”
“Our production and backup systems are never out of sync.”
“We can make application and database changes without affecting our replication engine.”

Examining These Misconceptions in Detail

“I have a replication engine, therefore both systems and their data match.”

Actually, mismatches are more common than you might think. Here are some common examples:

Database Administrator (DBA) actions can affect replication:
- The application is accidentally configured to make changes the target database instead of the source database
- The Transaction Monitoring Facility (TMF) is disabled – even momentarily
- TMF (or a related journal log) Audit Trails are deleted
- A DBA changes the target database structure instead of the source
Lack of coordination between the application and infrastructure teams:
- Adding new files or tables to the database but not to the
  replication configuration
- Forgetting to enable the TMF Audit flag after adding new files or tables to the database and replication configuration
Forgetting to link non-audited intercepts into the application programs after rebuilding a non-audited application

2. “Our production and backup systems are never out of sync.”

In practice, if you are running an asynchronous replication engine (which accounts for most of today’s HPE NonStop users), your production and backup databases will be out of sync almost 100% of the time!

Why? Because of replication latency, which is the time from when the input/output happens on the source to when it is picked up and replicated to the target by the replication engine. It means that if the replication stops (or is delayed for any reason, including a slow network), the two databases are not in sync.

Should the delay or outage be long-lasting, it is helpful to know exactly what is out of sync. Knowing this enables fixing differences faster. Differences can be rapidly fixed via a compare/repair solution, or through the replication engine by applying a “NO-OP” update that reapplies existing data to the production system, and allows the backup system to be corrected by the replication engine.

Avoiding this process of database validation and identifying differences creates unnecessary business risk.

3. “We can make application and database changes without affecting our replication engine.”

There is some truth to this. Certainly, you can create new subvolumes and data directories without affecting replication, but what about changing the structure of production files or tables that are being replicated? Yes, this happens in the real world, and it typically causes problems.

The Bottom line: HPE NonStop Business Continuity architectures require Data Validation to ensure the Source and Target databases match.

Here are some major benefits:

Data validation proves that your backup database is consistent, providing peace of mind and satisfying stakeholders, management, and auditors.
It ensures the backup system is ready for an immediate takeover, and that no data will be lost due to an outage.
Data Validation creates a timeline of data points recording when the databases are matched, which expedites root cause analysis when mismatches occur since less Audit Trail or related data needs to be analyzed.

Using powerful and fast Data Validation software adds other benefits, which we’ll discuss later. For now, let’s look at a real-world use case.

Use Case: Data Validation at Major Financial Services Firm

Situation

One of the world’s largest financial services firms provides investment services, transaction processing, and asset management on HPE NonStop systems.

The firm is required to satisfy various regulatory requirements, including validating that its business continuity solution’s target data is correct so that data will be correct, complete, and consistent and will not be lost after a switchover occurs. Unfortunately, the firm trusted but never verified that its source data was accurately and completely applied to the target database by its replication solution.

Problems

Auditors indicated that any data inconsistency would put the firm at risk and be in violation of regulatory requirements.
Continuously validating the data posed a complex and time-consuming challenge; the firm could not take an outage to perform the validation.

Solution: HPE Shadowbase Compare for Data Validation

After carefully considering its options and undergoing an extensive proof of concept, the firm chose HPE Shadowbase Compare to validate, repair, and enable regulatory compliance of its NonStop database environment.

HPE NonStop Shadowbase Compare Software (BE445AC/QSA53V6)

Compare Only Requires 3 Simple Commands for Data Validation

For illustration (skip to 17:00 to see Compare in action):

Watch the Demo!

The figures below illustrate HPE Shadowbase Compare in action.

Step 1: Compare Data.

Compare iterates over files and tables, generating a “Table of Differences” which can be output to a summary or detailed report.

2. Repair Data (optional).

Compare contains a Repair function. This particular example illustrates Repair overwriting data in the target database to correct the differences.

The firm’s DBA can either use the HPE Shadowbase Repair function or fix the mismatch a number of other ways such as performing a “NO-OP Update” by re-running transactions on the source environment and allowing replication to overlay and correct the target database’s differences.

Outcomes

HPE Shadowbase Compare:

Reassures the firm’s key stakeholders that a failover to the target would have a consistent and complete database image
Identifies and resolves data differences without taking an application outage or interfering with replication activity, including ignoring false-positives due to replication latency.
Satisfies auditing best practices and guarantees regulatory compliance by verifying replication activity and automatically certifying the production and backup databases match.
Helps monitor that both the production and backup systems are properly working, notifying the customer when mismatches occur and alerting them that there may be a problem between the two systems.
Strategically positions the firm by enabling them to use Compare for validating their data during migrations, platform refreshes, upgrades to new systems, etc.
Provides a layer of replication insurance: Compare works regardless of the replication engine in use, and does not interact or interfere with the replication engine (including HPE Shadowbase).

Wrap Up

In Business Continuity architectures, consistency between the production and backup systems is paramount. Just because there are misconceptions in the industry (and what some would call “insane” Business Continuity practices), this doesn’t mean that you have to partake.