Database backup – The Failover way – Part 1

Monitoring solutions involves data storage and enterprise monitoring solutions involves huge volume of data ! NetFlow data exported from monitored devices can be used to generate detailed reports for real time traffic behavior, historic reports, application performance, link utilization and so on. The data stored by the NetFlow based tools is very valuable for comparison of links over time periods, trend analysis and making capacity planning decisions which all finally helps in cost cutting. You can find many more advantages of using NetFlow data and NetFlow Analyzer from here.

With solutions that provide critical information from huge volume of data, backup and up time of the solution is as primary as any reporting feature. This brings in the need for a proper data backup mechanism which can save the day in case of a product crash, database corruption or virus attacks. Data backup can be done either online, ie. without shutting down the product or offline, which is to backup the data after shut down of the data collecting engine.

Online data backup is most commonly preferred because it does not involve product shut down and thus no down time. But, when it comes to products that involves continuous collection, storage and reporting from huge volumes of data, every day or every week scans for incremental backup can strain the product performance, affect the stability and might also cause a data corruption. Backing up data to save data from loss during future crashes causing a database corruption. A vicious circle it is !
Here, a workaround solution would be to do a offline backup by through a maintenance shut down. When the backup involved huge volumes of data, the shut down and restore is not minutes or hours, but could be days. Because down time of a business critical monitoring system is unacceptable, the last method is not even considered as a solution by many administrators.

This is where a failover mechanism can help. A failover feature has many advantages and solves multiple problems. With failover:

1. You get to to continue the network monitoring and report generation even if the primary server goes down.

2. You can have the standby server, and thus the standby database, at a remote location or even your DR center ensuring a surefire way of data backup and recovery.

3. You get an incremental data backup mechanism without ever having to shutdown the product

For an argument, one may say failover means additional servers and resources which means increased cost. But, with increasing number of hard disk failures and the high cost of data retrieval with no guarantee, adding one more server is just a percentage of cost in the long run.

And then again, there is the argument that failover or data replication is not the same as a data backup. The intention of data backup is to have a source for data to be used when the product crashes or if part of the data is corrupted due to errors. But this is exactly what the failover through database replication is achieved in the Enterprise edition of NetFlow Analyzer. The data replicated to a secondary server acts as a source to be used if the primary server is down and also can be used if the data base in the primary server is corrupted.

The Enterprise edition of ManageEngine NetFlow Analyzer, about which we have discussed earlier, with its highly scalable architecture and features for distributed setup like timezone view and hierarchical device groups, is useful for large organizations with branched offices or for enterprises with a large number of devices to be monitored. The Enterprise edition provides failover feature which can be setup and configured within a matter of minutes. All you need is a server identical to your primary server with the same or higher volume of hard disk space to which data replication takes place.

The failover feature in Enterprise edition works in such a way that the secondary server is automatically enabled when the primary one goes down. When the primary server goes down, the collectors which are sending flows to the central server will automatically redirect to the secondary server which then becomes the primary server. With a hassle free switch over and data replication, failover is the perfect data backup mechanism one needs in an enterprise network. For all those techies, Part 2 and Part 3 of this blog will explain the architecture and setting up of failover in NetFlow Analyzer Enterprise edition.