Network administrators implement QoS policies to ensure that their business-critical applications receive the highest priority on the network. CBQoS can make network performance more predictable and bandwidth utilization more effective. NetFlow Analyzer CBQoS reporting provides you in-depth visibility into the policies applied on your interfaces and the traffic patterns in your various class of traffic.
NetFlow Analyzer is capable of monitoring the QoS policies applied on the interfaces of the router and generate reports on Pre-Policy, Post-Policy, Drop and Queue metrics for each class. You can check the CBQoS reporting in NetFlow Analyzer from this link.
Though NetFlow and CBQoS reporting are used for traffic monitoring, they are based on diverse technologies. Some time back,ie. till version 7, NetFlow Analyzer had options to monitor only NetFlow supported interfaces for CBQoS statistics too. Customers who loved our CBQoS reporting feature were not able to monitor the devices that did not have NetFlow capabilities for CBQoS stats. Not fair ? Correct. That is why we brought the capabilities to add non NetFlow interfaces to the product for CBQoS monitoring.
NetFlow Analyzer detects the interfaces of a routing device based on the NetFlow packets exported from it and adds it to the database to show the traffic reports. If there are QoS policies available on the interface, polling for CBQoS data can be enabled and CBQoS reports will be available in the product.
But there are cases we mentioned about before. Customers do have devices which are either non NetFlow capable or NetFlow reports are not needed for. These devices may be an edge router used for branch connectivity which has no NetFlow capability but has quite a number of QoS policies for bandwidth shaping or a data center device where you are not interested in NetFlow reports but need to monitor QoS policies. Lets go on to see how such devices can be monitored for CBQoS stats.
Add Device.
If you have just installed NetFlow Analyzer and started the product for the first time with no device exporting NetFlow packets, you will see a message which states "No device exporting NetFlow packets to UDP port 9996, Click here to add device which has QoS policy" as soon as you login. In this case, you can click on the 'Add Device' option to add the devices you need to monitor for QoS. Here, provide the SNMP credentials for the device and NetFlow Analyzer will poll the device for CBQoS stats and generate reports.
If you are already using the product and wish to add a new device for CBQoS monitoring alone, navigate to NBAR/CBQoS configuration page, select the 'QoS Configuration' tab and from here click on the "Add device" link. You will be given options to enter the router IP Address and SNMP parameters for the device. Once this is done and you click on 'Scan', NetFlow Analyzer will detect the device and show the interfaces having CBQoS policies on them. You can then enable polling for the specific interfaces you need report for from the 'Polling for CBQoS data' category.
And again, what happens when you have CBQoS policies on a main physical interface which has no IP Address and it is the sub interface with IP Address that you are monitoring for NetFlow data? Most of the NetFlow tools detect only L3 NetFlow exporting interfaces, the sub interfaces in this case. But the traffic through the sub interfaces is shaped by the policies applied on the main physical interface and so it is necessary that the main interface is monitored for QoS analysis. NetFlow Analyzer will automatically detect such main interfaces though they are not L3 NetFlow exporting interfaces and show them in the list of interfaces available with QoS policies. You can add these main interfaces to CBQoS monitoring to get an idea on the CBQoS performance.
With NetFlow Analyzer not limiting you to monitor only NetFlow interfaces for CBQoS stats, why wait? Go ahead and add your routers to NetFlow Analyzer to see CBQoS reports. Try our 30 day trial with no feature limitations to know more.
Demo | Download 30-day Trial | Twitter | Customers
Regards
Praveen Kumar
In previous discussions, we have mentioned that NetFlow Analyzer offers various kind of reports for bandwidth analysis. Just thought we should highlight the various types of reports available in NetFlow Analyzer and how they help in better bandwidth monitoring and traffic analysis.
To be simple, NetFlow Analyzer depends on the NetFlow packets exported from the routers and switches and generates various reports which can be helpful for bandwidth analysis, bandwidth measuring, troubleshooting and trend analysis etc.
NetFlow Analyzer shows information on the interfaces and their traffic from the product UI itself with PDF and CSV export options available. In addition to these, the product has more reports to help in detailed bandwidth analysis. Following are some of the reports available in NetFlow Analyzer :
1. Troubleshoot report
2. Search Report
3. Consolidated Report
4. Compare Reports
Troubleshooting Report:
I believe you have an idea about the storage pattern in NetFlow Analyzer with help of Data Storage Pattern Blog . Troubleshooting report is generated from the raw data, (about which we have discussed in the Data Storage Pattern Blog) and is used for detailed traffic analysis, helps identify cause of network spikes with complete port level information.
Troubleshooting Report can be generated by clicking on the troubleshooting icon present in the Interface View for each interface or we can drill down to specific interface then click on More Reports present at the right corner of the user interface. We can generate troubleshooting report by specifying criteria as per our report generation needs. Troubleshoot report can be generated for the time period raw data is stored in NetFlow Analyzer. So, any time you need a detailed analysis of traffic, dont forget the troubleshoot report.
Search Report:
Search report is similar to troubleshooting report but this report generated from aggregated data which is based on top 100 (Again the Data Storage Pattern Blog should give you an idea). You can can generate search report by clicking on More Reports available in Interface View right corner. You can select the interfaces for which you want to generate report by clicking on "Select Device" and like troubleshooting report, you can specify different criteria as per report generation needs. This report is most helpful when you need to analyze specific information going back in time. The report, since it is generated from aggregated data, can give historic information. Imagine having around 80% report accuracy for data ranging back to years !
Consolidated Report :
Consolidated report is a single page report which will list the traffic graph for the selected interface or IP group with the top 10 Application, Source, Destination and Conversation on IN and OUT basis. Consolidated Report can be generated by clicking on the Quick View icon present in the Interface View for each interface or we can drill down to specific interface or IP group then click on More Reports present at the right corner of the user interface. The reports help get a quick view on the traffic stats from each of the interfaces thus helping to avoid drill downs to the interface and then checking the top applications one by one.
Compare Reports:
Compare Report help you compare the traffic pattern over time or with different devices, networks or locations. You can get a picture on the traffic pattern for different devices or have an idea of the traffic pattern for the same device over time. To know more about Compare Report in NetFlow Analyzer, check out this blog.
Most of the reports we have talked about may be needed on a daily basis. Instead of having to generate the report everyday, you can have the reports emailed to you and this is where our Schedule Reports help.
Schedule option lets users
create reports about the information they need and have them emailed
on a daily, weekly or monthly basis. The reports can be send to
multiple email addresses and users can set time filters for daily
reports and exclude the reporting on weekends. To know more about
Schedule Report in NetFlow Analyzer visit this Blog.
With a better knowledge on the reports available in NetFlow Analyzer, I hope you can get more out of the product.
Demo | Download 30-day Trial | Twitter | Customers
Regards
Praveen Kumar
You may have seen that NetFlow Analyzer generates various kinds of report which can be used for Network troubleshooting , more visibility on traffic patterns, trend analysis, etc. In continuation with Data storage pattern blog , we are going to discuss on sub minute visibility available on the NetFlow Analyzer.
These days bandwidth utilization is being monitored very closely by the network admin to make sure that the network resources is properly utilized. This sub minute visibility in NetFlow Analyzer helps network admin to identify the hosts ,application, etc consuming bandwidth for each and every minute thus helping in identifying the cause of even short duration spikes.
NetFlow Analyzer shows the IN and OUT traffic passing through an interface for each minute and each and every transaction will be accounted. As a networker, you may have seen bandwidth choke occurring very randomly and for short time periods. It is to help with such short term troubleshooting that the product has sub minute visibility feature. If you are using NetFlow and NetFlow Analyzer, login to product and check the interface where the traffic passes through. If you see some short term spikes in the graphs occurring very randomly, click on 'Show Data Points' to see the traffic details for each and every minute for the selected time period and look out for the minute where the traffic pattern has suddenly changed.
Click on the minute for which you see a change in pattern and there you are ! A conversation report with the list of conversations that happened during that minute showing you the top talker. Check out some of the spikes that has so occurred and you can find out the common culprit.. maybe a host downloading some large file, a FTP by someone to his home PC, or maybe a possible large scale DNS scan (which could possible by a bot).
Based on the report, you can find out which application or host is utilizing the bandwidth and can introduce ACLs or QoS policies to stop or limit access through the interface.
The sub minute visibility report, which is generated when clicking on the data point, is generated from the raw data, As explained in the Data storage pattern blog, raw data consists of each and every flow from the interface, giving port level information and helping in better network troubleshooting.
Regards
Praveen Kumar
Many of you out there who uses NetFlow Analyzer or is evaluating NetFlow Analyzer would certainly want to know how the product stores its data and does all the historic reporting.
NetFlow Analyzer processes the NetFlow data exported from the devices and stores it in the database for traffic analysis and reporting. NetFlow Analyzer's flexible data storage pattern is intended to achieve detailed data storage forever without having an impact on the hard disk space and also provide real time reporting.
Data stored on NetFlow Analyzer will help you to achieve following things:
1. Troubleshooting Network spikes
4. Billing
6. Understanding Traffic Pattern and much more.
Coming to the data storage, NetFlow Analyzer stores two types of data, Raw data and Aggregated data.
Raw Data Storage:
Raw data is each and every flow exported from the monitored interfaces of the routers. All the flows exported from the routers is stored in the NetFlow Analyzer database as raw data. Since, the raw data is each and every flow from the routers, it consumes lot of disk space and so is set to be stored for maximum of 30 days. Raw data storage is determined by the amount of flows the product receives from the monitored routers. To make calculation easier, the product itself can suggest how long one can store the raw data based on the free space available in the installation directory and the flow rate.
Raw data storage can be configured on the product by clicking on Product Settings --> Storage Settings --> Raw data Storage. There are also options available to alert you when free disk space goes below specified percentage and to automatically delete the older raw data when disk space goes below a specified percentage.
The raw data is used in the product when generating 'Troubleshoot' reports and the last 2 hours reports will be generated from the raw data. The raw data has complete port level information which helps in detailed analysis of traffic.
Aggregated Data:
Apart from the raw data storage, NetFlow Analyzer stores aggregated data which is stored for ever in the database. The aggregation mechanism will happen simultaneously at the back end along with the raw data storage. The aggregated data is stored based on top 100 fields of the application and conversation for every 10 minute interval and is further aggregated as time goes on.
The aggregation of NetFlow data collected is done to avoid high disk space usage without impact on reporting and performance. The aggregated data on NetFlow Analyzer is used for historical reporting, capacity planning and trend analysis.
Following explanation will help you to understand how Application data on NetFlow Analyzer is aggregated and stored in various tables.
Aggregation Mechanism for Application data:
Older data is repeatedly rolled up into less granular times (10 minute, 1 hour, 6 hour, 24 hour and weekly). The top 100 records of application based on octet value is stored for every 10 minute interval. As time goes, this data is further aggregated to an hourly table.
When we select time period 10:00 to 10:59, NetFlow Analyzer stores top 100 Application for each 10 minutes (10:00, 10:10, 10:20, 10:30, 10:40 and 10:50), this data will be under 10 minute table. From this six 10 minutes data, the 600 records pertaining to 10:00, 10:10, 10:20, 10:30, 10:40 and 10:50 would be aggregated and the top 100 would be moved to the 1 hour table pertaining to 10:00.
In the same manner, aggregation happens to the hourly table and the data is moved to 6 hour table then to daily table and finally weekly tables. Most recent data is stored with 10 minute granularity and data older than 90 days is stored with 1 week granularity.
The 10 minute table will have most recent data and data older than 25 hours is cleaned up. Following is how the data are repeatedly rolled out.
10 minute granular data is stored for 25 hours (beyond which the older data is deleted)
1 hour granular data is stored for 30 days
6 hour granular data is stored for 30 days
24 hour granular data is stored for 90 days
1
week granular data is stored forever
In
the same way as applications, conversations are also aggregated and
stored in the database for historic reporting. The Application,
Source, Destination, Conversation and QoS reports generated for more
than last 2 hour period will be generated from the Aggregated data.
The granularity of data represented will change based on the time
period you select.
1 Minute traffic Data Storage:
Apart from the raw data and aggregated data, NetFlow Analyzer stores 1 minute traffic data which is used for real time reporting purpose. The aggregation mechanism for the traffic data happens as the same way we explained for Application data. The traffic report generated for any time period which is less than 24 hour is generated with 1 minute granularity which will give you a detail picture of each and every transaction going IN and OUT.
One minute data storage can be configured on the product by clicking on Product Settings---------> Storage Settings-----> One Minute Data Storage Settings.
Hope this blog gives you a better understanding about the data storage pattern in NetFlow Analyzer and will help you use the product better.
Interactive Demo | Product overview video | Twitter | Customers
Regards
Praveen Kumar
NetFlow Analyzer and RADIUS!
Whats is Radius ?
Remote Authentication Dial In User Service (RADIUS) is a networking protocol that provides centralized Authentication, Authorization, and Accounting management for computers to connect and use a network resources.
RADIUS enables centralized management of authentication data, such as user names and passwords. When a user attempts to login to a RADIUS client, such as a NetFlow Analyzer, the NetFlow Analyzer sends the authentication request to a RADIUS server, which is the centralized authentication server. The communication between the RADIUS client and the RADIUS server is authenticated and encrypted through the use of a shared secret, which is not transmitted over the network.
Configuring NetFlow Analyzer for Radius Authentication:
In order to configure users to access NetFlow Analyzer via Radius Server Authentication, we need to configure the radius server settings within the product. To configure Radius Server Credentials, the option is under Admin Operation ----------> Product Settings ---------> Advanced Settings Tab.
Following credentials need to be configured for Radius Server Authentication on NetFlow Analyzer:
Radius Server IP : IP address of the Radius server
Radius Server Authentication Port : Port through which the radius server is listening for authentication requests from NetFlow Analyzer
Radius Server Protocol : Protocol used for authentication purpose
NetFlow Analyzer support variety of Authentication Protocol for Radius Server Authentication, They are;
PAP : Password Authentication Protocol provides a simple method for the peer to establish its identity using a 2-way handshake.
CHAP : Challenge-Handshake Authentication Protocol (CHAP) authenticates a user or network host to an authenticating entity.
MSCHAP : MS-CHAP is the Microsoft version of the Challenge-handshake authentication protocol, CHAP.
MSCHAP2 : Another version of Microsoft version of the Challenge-handshake authentication protocol, CHAP.
Radius Server Secret : Secret that is specified on the Radius Server
Authentication Retries : Number of retries for authentication
Once the Radius server settings is configured on the NetFlow Analyzer, the next step is creation of user accounts.
User Creation on NetFlow Analyzer:
We can create users for NetFlow Analyzer from User Management page. Here, you need to enter a user name available in the RADIUS server and select the option to authenticate via Radius.When the created user tries to login to NetFlow Analyzer, he will authenticated via Radius Server. The Radius Server reads the request from the NetFlow Analyzer and checks the user name and password on its database and if the credentials are passed, the user will be directed to the NetFlow Analyzer web console.
With this type of secure authentication, we do not need to create user name and password locally in NetFlow Analyzer. RADIUS server authentication provides secure authentication and accounting. It is also possible to integrate RADIUS server with Active Directory so that you also get the capability for using user accounts from AD in NetFlow Analyzer.
Thanks
Praveen Kumar
NetFlow Analyzer Technical Team
Interactive Demo | Product overview video | Twitter | Customers
"It does an excellent job of accumulating our data flows so I
can accurately research problems in the WAN/LAN. Since It only keeps the
headers it is very efficient regarding storage. The the groups work
well to help fine tune Application performance."
Find below the TOP 10 reasons for having close to 4000 enterprises use NetFlow Analyzer for bandwidth monitoring, traffic analysis and much more...
Download | Interactive Demo | Product overview video | Twitter | Customers
cheersRecent improvement in the communication and broadband technology
has made ISP's to offer better billing model to their customers based
on their bandwidth usage. ISP's have their own standard technology to
bill the customers which is billing based on 95th percentile. Some of
the ISP's do offer billing based on the 90th percentile to attract
customers, but as of now the industrial standard of billing the
bandwidth usage is based on 95th percentile.
In this blog, we
are going to have brief look on 95th percentile, NetFlow Analyzer
reports with 95th percentile calculation and billing reports based on
95th percentile in NetFlow Analyzer.
95th Percentile :-
The 95th percentile is the standard of billing model and
it has a specific meaning. In order to calculate the traffic rate for
which you will be billed, ISP sorts the samples taken during your
billing period, then ignores the highest five percent of those
samples.
Traffic Graph of NetFlow Analyzer.
NetFlow
Analyzer calculates and displays 95th percentile for the interfaces
and IP groups. Given below are some example how NetFlow Analyzer
calculates and shows the 95th percentile for both IN and OUT traffic.
In the following screen shot for the time period of 20 minutes, there is about 20 data points(1 minute granularity) for both IN and OUT. These data points for both IN and OUT are separately sorted in a descending order to calculate the 95th Percentile.
Given
below is the calculation which shows how 95th Percentile is derived
for the IN traffic, for the OUT traffic the methodology is same
as IN traffic.
IN Data Points = (114.06, 137.09,
159.53, 159.6, 160.06, 182.24, 182.45, 182.75, 205.06, 205.74,
227.96, 228.33, 228.39,228.71, 228.76, 250.98, 251.11, 251.4, 251.74,
273.87 )
Now the data points gathered are sorted in
decesending order as below,
INData Points = (273.87,
251.7,251.4, 251.11, 250.98, 228.76, 228.71, 228.39, 228.33, 227.96,
205.75, 205.06, 182.75, 182.45, 182.24, 160.06, 159.6, 159.53,
137.09, 114.06)
From this 20 data points the top 5% of the
point is been ignored and the one next is considered as 95th
percentile IN. The data point ignored is 273.87 and the 95th
Percentile is 251.7
NetFlow Analyzer traffic graphs are based on 1 minute granularity,
the above example calculation for the 95th percentile is for traffic
graph on NetFlow Analyzer which is based on 1 minute granularity.
NetFlow Analyzer have billing functionality which is peculiarly
designed for ISP and Enterprise to bill their users and customers
based on their usage.
Billing Reports on NetFlow
Analyzer:-
NetFlow Analyzer offers a functionality to
users and ISP to bill the departments / clients based on their usage.
For this, we need to create a bill plan on NetFlow Analyzer and
associate the interfaces or IP group to the bill plan. Once the bill
plan is created, NetFlow Analyzer gathers the traffic usage of the
interface or IP group associated to bill plan for the billing period
and generates the billing report based on the 95th percentile. The
one important thing being that billing reports in NetFlow Analyzer is
based on 5 minute granularity for the whole billing period. In the
billing module, you can select the opiton to generate billing report
based on 95th percentile combined for both IN and OUT traffic or
separately.
Below given is an example screen shot of NetFlow Analyzer billing report based on 95th Percentile by merging IN and OUT.
NetFlow Analyzer, though the name says NetFlow, can work with quite a number of flow formats like sFlow, jFlow, NetStream, IPFIX etc. This blog will give you a brief idea on sFlow technology and also guide you on how to use NetFlow Analyzer with sFlow from HP Procurve devices.
What is sFlow?
sFlow is a monitoring technology which allows you to capture the traffic data from a switched or routed network to give complete visibility into the use of network bandwidth. This data helps in performance optimization, accounting/billing for usage, defense against security threats, capacity planning and much more.
sFlow datagrams are exported based on sampling due to which impact on the device CPU/Memory and available bandwidth is minimal. Based on a defined sampling rate, 1 out of N packets (where N is the sampling rate) is captured and sent to the NetFlow Analyzer for traffic analysis by the device. Though, this type of sampling does not provide 100% accurate statistics, it does provide a result with quantifiable accuracy.
sFlow analysis with NetFlow Analyzer:
NetFlow Analyzer can work with any devices which are capable of exporting NetFlow, sFlow and other compatible flow which are completely vendor dependent. You can check out the list of flow formats and devices with which NetFlow Analyzer can work from here.
HP Procurve and sFlow:
Just like Cisco has NetFlow and other vendors have thier flow formarts, some vendors use a technolgy called sFlow. HP Procurve devices are capable of exporting sFlow datagrams which can be used for bandwidth monitoring and traffic analysis. NetFlow Analyzer is capable of analyzing the sFlow datagram exported from the HP Procurve to give you the traffic statiscs on each active ports.
sFlow export on the HP procuve device can be configured using two different methods, We can enable sFlow on the HP device either by logging in to the router and configuring them for sFlow export. But this is available only in the older device models or OS.
On the new HP devices, sFlow can be enabled only through SNMP. To make the sFlow configuration on HP device a simple task, NetFlow Analyzer provides scripts to enable and disable the sFlow export. So, lets see how we can use the script and enable sFlow.
sFlow Enable utility:
The script to enable sFlow, named as sFlowEnable.bat (for Windows and .sh for Linux), is present under <\AdventNet\ME\NetFlow\troubleshooting> directory.
The usage for the script is as follows:
SFlowEnable.bat switchIp snmpPort snmpWriteCommunity collectorIP collectorPort samplingRate
Example:-
C:\AdventNet\ME\NetFlow\troubleshooting>sFlowEnable.bat 192.168.188.30 161 private 192.168.133.1 9996 4096
Once sFlow is enabled on the HP devices, NetFlow Analyzer server will receive the packets and the product will capture the packets to automatically generate the reports. You also need to ensure that no access control lists (ACLs) or firewalls block the NetFlow packets (on UDP 9996) and that even the software firewalls on the server are allowing the packets to reach the NetFlow Analyzer installation.
After enabling the sFlow on the HP devices, we need ensure a few points to get the accurate traffic statistics about the device in NetFlow Analyzer.
The first and foremost is the sampling rate. We suggest setting the sampling rate to 4096. We have observed from various setups and from our existing customers feedback that the sampling rate of 4096 gives the most accurate traffic statistics in NetFlow Analyzer.Most of the other sFlow collectors in the market suggest the sampling rate to 256 which means more number of exported sFlow datagrams. With a sampling rate of 4096, you get the additional benefit that the device is not being overloaded by sampling large number of datagrams and exporting to the NetFlow Analyzer.
Next point we need verify is the "sFlow receiver timeout". This determines how long sFlow remains active on the exporting device. When the value has expired, sFlow also gets disabled on the device forcing you to re-enable sFlow export. Due to this, we recommend setting the sFlow Receiver Timeout to the maximum possible value, which is 2147483647 seconds which is 68 years ! The command to be used on the HP device for setting the sFlow receiver timeout is:
setmib sFlowRcvrOwner.1 -D NetFlow Analyzer IP sFlowRcvrTimeout.1 -i 2147483647
sFlow Disable Utility:
Of course. We have thought about that too. Just in case you want to export sFlow to different server or stop the flows for some time or whatever be the reason, NetFlow Analyzer provides you the script to disable sFlow export on the HP device.
The disable can be done using the script sFlowDisable.bat (for Windows and .sh for Linux) and the file is present under <\AdventNet\ME\NetFlow\troubleshooting > directory. The usage of the script is as below:
SFlowDisable.bat switchIp snmpPort snmpWriteCommunity
Example :-
C:\AdventNet\ME\NetFlow\troubleshooting>sFlowDisable.bat 192.168.188.30 161 private
Go ahead and try our 30 day trial to see for yourself on how well NetFlow Analyzer works with sFlow and HP devices.
Thanks
Praveen Kumar
Download | Interactive Demo | Product overview video | Twitter | Customers
Corp