Being a niche player in the SAAS market, Zoho brings an amazing level of engineering expertise to ManageEngine in building highly secure and scalable distributed applications. And hopefully you know, Adventnet has recently changed its name to Zoho Corp and formed three divisions namely ManageEngine, Zoho, and WebNMS.

 ManageEngine NetFlow Analyzer Enterprise Edition is a truly distributed NetFlow collection and reporting application, purpose-built for large organizations managing hundreds and thousands of networking devices and links across their geographically distributed business locations. When we started building NetFlow Analyzer Enterprise Edition, one of the biggest challenges we faced was improving the flow handling capacity and building a unified view of geographically separated networks. After experiments, the engineering team concluded that offloading flow collection from the reporting center drastically improved the flow handling capacity.

 Below is the architecture of our distributed edition. You can see the collectors are deployed at every major business locations and data centers for flow collection. These collectors compresses the exported flow data and sends it via HTTPS connection to the central server for reporting purposes. Here, most of the flow processing functionalities were offloaded to collectors which helps the central server to generate reports within seconds for any particular device.

NetFlow Analyzer EE Architecture
Many of the NetFlow Analyzers available in the market are not truely distributed in nature. They parse and store the flow records in the same collector and cannot give you the unified view of all the collection points. And there is no automatic crash recovery of data is possible. Unlike in ManageEngine, it involves individual backup and upgrade procedures which requires lot of maintenance activities. All these procedures are automated in ManageEngine NetFlow Analyzer Enterprise Engine via failover and smart upgrade manager technologies. And this is why we call ManageEngine NetFlow Analyzer is a Enterprise class distributed NetFlow collection and reporting engine suitable for any large organisations. And when we say distributed we mean it.

Before you start evaluating a distributed and scalable netflow monitoring solution, please ensure that you have the following Enterprise class features are available.

1. Distributed flow collection capability and optimized bandwidth usage between collectors and central reporting server.
2. Scales upto 20000 interface with 15000 flows per second. Any number of collectors can be added without any additional license.
3. Support for NetFlow V5,V7,V9 /sFlow, JFlow, NetStream, IPFIX.
4. Support for Cisco NBAR and correlate NBAR data with NetFlow data.
5. Support for CB-QoS (Class Based - Quality of Service) monitoring. Identify Pre and Post policy metrics and fine tune your QoS configurations.
6. Failover support - automatic crash recovery and data replication. Please visit this link for more information.
7. Ability to use your existing SAN (Storage Area Network).
8. Compatible with VM ware.
9. No data loss even after a link failure between Collectors and Central Server.
10. Ensure separate 64 bit binaries are available for increased flow handling and reporting performance.
11. Secure data transfer - https mode between collector and central server
12. Smart upgrade manager. Upgrade patchs are pushed automatically from the central console to collectors. 
12. User defined dashboards and views.
13. Group devices based on their location and build tree view for easy access and troubleshooting.
14. Ability to work in multiple time zones
15. Network Forensics using raw data


NetFlow Analyzer EE View


 And remember thousands of users like Cisco,Adobe, Ferrari and many fortune companies cannot be wrong.

 Please download and try our 30 day full featured trial edition in the following link


Full Feature List is available in the following link


  Kindly write your questions to netflowanalyzer-eesupport@manageengine.com. We are happy to assist you at any moment. 

Thanks
Raj

Having outlined the need for failover which also substitutes as a data backup mechanism in NetFlow Analyzer Enterprise edition through our first blog, we will tell you in detail about the architecture of the failover feature in our product and how you can enable and make use of the feature.

The Enterprise edition of NetFlow Analyzer is based on a collector - central server architecture. In, the distributed architecture data collection to be done by collectors which sends the collected data to the central server from where all the reporting takes place. This provides higher scalability and performance to the Enterprise edition by making it capable to handle up to 20,000 interfaces and each collector to handle 10,000 flows per second.

To use failover feature, all you need is a second server identical to the primary central server in configurations which acts as the hot standby server.The machine can be located in the same network or even in a geographically separated DR center. You only need to ensure that both the primary and hot standby server (failover machine) can communicate with each other and that the collectors can reach the failover machine if the primary server is down.





In this setup, the primary server is the central machine which receives all the data from the collectors located at various locations.The collectors are configured with the primary server IP Address so that the collected data is sent across in real time. To enable failover, you can navigate to 'Settings' under 'Admin Operations' and click on the the 'Failover Settings' tab. From here, you can enable failover by simply clicking on the 'Enable' radio button.


Failover Setup


After enabling failover, shut down the primary server and copy the data folder from <NetFlow_Central>/mysql/ directory and tmp folder from <NetFlow_Central> directory to a safe backup location. Start the central after the this step is complete and you will be prompted with the message "Replication is enabled. How would you like to start your server?". Click "primary server". This will start your existing central server as the primary server in the failover setup.

Now download and install the Enterprise edition central server on the machine you intend to use as the hotstandby server. You need to ensure the following:

1. The Operating System and hardware on both the primary server and hotstandby server should be identical.

2. The time and timezone on the primary server and hotstandby server should be same.

After the installation is complete, do not run the central server installation. Before running the installation, copy the data and tmp folder which were backed up earlier to the <NetFlow_Central>/mysql/ directory and<NetFlow_Central> directory respectively. You will be prompted to overwrite the existing file and you can proceed with this step. This is needed so that the data and configurations which have been stored in the primary server is available in the hotstandby server. Once the copying of database is complete, start the hotstandby server from 'All Programs > ManageEngine NetFlow Analyzer EE - Central server > Replication > Start hotstandby server'. You will be prompted to enter the host name/ IP Address of the primary server and HTTPS port (by default it is 443).

You can change the port used for communication whichever port you prefer. At this step, the hotstandby server communicates with the primary server and updates its details to the primary server database. The primary server will update all the collectors with the information about the hotstandby server thus relieving the user from the task of updating all the collectors with this information.

Guess you can take a break now ! Part 3 of this blog will outline how the hotstandby server becomes the primary server. For those who would like to have a go at it now, do try the NetFlow Analyzer Enterprise edition.

Download (30 day trial) | Interactive Demo | Product overview video

Regards,
Don Thomas Jacob



Monitoring solutions involves data storage and enterprise monitoring solutions involves huge volume of data ! NetFlow data exported from monitored devices can be used to generate detailed reports for real time traffic behavior, historic reports, application performance, link utilization and so on. The data stored by the NetFlow based tools is very valuable for comparison of links over time periods, trend analysis and making capacity planning decisions which all finally helps in cost cutting. You can find many more advantages of using NetFlow data and NetFlow Analyzer from here.

With solutions that provide critical information from huge volume of data, backup and up time of the solution is as primary as any reporting feature. This brings in the need for a proper data backup mechanism which can save the day in case of a product crash, database corruption or virus attacks. Data backup can be done either online, ie. without shutting down the product or offline, which is to backup the data after shut down of the data collecting engine.

Online data backup is most commonly preferred because it does not involve product shut down and thus no down time. But, when it comes to products that involves continuous collection, storage and reporting from huge volumes of data, every day or every week scans for incremental backup can strain the product performance, affect the stability and might also cause a data corruption. Backing up data to save data from loss during future crashes causing a database corruption. A vicious circle it is !
Here, a workaround solution would be to do a offline backup by through a maintenance shut down. When the backup involved huge volumes of data, the shut down and restore is not minutes or hours, but could be days. Because down time of a business critical monitoring system is unacceptable, the last method is not even considered as a solution by many administrators.

This is where a failover mechanism can help. A failover feature has many advantages and solves multiple problems. With failover:

1. You get to to continue the network monitoring and report generation even if the primary server goes down.

2. You can have the standby server, and thus the standby database, at a remote location or even your DR center ensuring a surefire way of data backup and recovery.

3. You get an incremental data backup mechanism without ever having to shutdown the product

For an argument, one may say failover means additional servers and resources which means increased cost. But, with increasing number of hard disk failures and the high cost of data retrieval with no guarantee, adding one more server is just a percentage of cost in the long run.

And then again, there is the argument that failover or data replication is not the same as a data backup. The intention of data backup is to have a source for data to be used when the product crashes or if part of the data is corrupted due to errors. But this is exactly what the failover through database replication is achieved in the Enterprise edition of NetFlow Analyzer. The data replicated to a secondary server acts as a source to be used if the primary server is down and also can be used if the data base in the primary server is corrupted.

The Enterprise edition of ManageEngine NetFlow Analyzer, about which we have discussed earlier, with its highly scalable architecture and features for distributed setup like timezone view and hierarchical device groups, is useful for large organizations with branched offices or for enterprises with a large number of devices to be monitored. The Enterprise edition provides failover feature which can be setup and configured within a matter of minutes. All you need is a server identical to your primary server with the same or higher volume of hard disk space to which data replication takes place.

Architetcture of failover setup

The failover feature in Enterprise edition works in such a way that the secondary server is automatically enabled when the primary one goes down. When the primary server goes down, the collectors which are sending flows to the central server will automatically redirect to the secondary server which then becomes the primary server. With a hassle free switch over and data replication, failover is the perfect data backup mechanism one needs in an enterprise network. For all those techies, Part 2 and Part 3 of this blog will explain the architecture and setting up of failover in NetFlow Analyzer Enterprise edition.

Download (30 day trial) | Interactive Demo | Product overview video