Validate and Investigate a live #DRBD replicated Oracle Instance using LVM clone

By | July 1, 2017

This is another post related to the DRBD replication setup I describe in post #DRBD based disk replication of a production cluster to a remote site cluster on RHEL 6.

Sometimes we need to be able to mount the offline replicated Oracle database instance for investigations. This can be done by using logical volume snapshots.

Follow the following procedure and execute it on the site that acts as disaster recovery site.

On the cluster node where the DRBD_Slave_Service is running freeze the cluster service by executing:

The above will ensure that all the applications associated with DRBD_Slave_Service are running but the cluster will no longer monitor this service.

Make a back-up of /etc/lvm/lvm.conf

Edit the lvm.conf and add under the volume_list directive the “vg_data” volume.

Scan the volumes

Create a snapshot of the lv_data volume where the database files are stored.

The snapdb together with lv_data will ensure that we will have a snapshot of the logical volume lv_data at the moment the above command was taken. Under snapdb logical volume only the changes from lv_data will be stored.

At this point we can mount snapdb snapshot as the database directory.

Then we start the Oracle listener:

Also as oracle user copy the control file to the flash_recovery area, we need to do this because the flash_recovery_area is not on the replicated drive.

Then start the Oracle database:

When the startup procedure is finished the Oracle instance is opened for investigation using sqldeveloper or we can even start a client application.

After investigation is done we must do a clean-up. Note that any changes done in the database are not affecting our “real” database from lv_data, only the snapdb copy

Stop Oracle by executing:

Then also as oracle user stop the Oracle Listener:

Then as root user unmount the data directory:

Remove the temporary snapdb logical volume:

Copy back the lvm.conf back-up:

Back-up the initrd image of the current kernel in case the new version has an unexpected problem.

Now rebuild the initramfs for the current kernel version:

Unfreeze the DRBD_Slave_Service cluster service

Note that during the above operations the replication between the live database and the disaster database is still going on without any interruption.