Quite a number of organizations uses some form of DSL connection for cost effective connectivity to the Internet of which ADSL is gaining more popularity due to the advantages it provides like higher security, IP Address conservation, per session accounting, etc. The ADSL connection requires the device to have a Dialer interface which establishes the connection after which a Virtual Access Interface will be created and the PPPoE session will run on this Virtual Access Interface. The Virtual Access Interface thus created inherits the properties of the Dialer interface.

Many users who use NetFlow data to monitor such interfaces would have seen that the Dialer Interface reports only outbound traffic and a Virtual Interface is automatically discovered and reporting inbound traffic. Let us see what is the reason for this and how NetFlow Analyzer can help.

As stated, it is the Dialer Interface created by the user that establishes the connection to the DSL provider and is the actual interface available on the router. Just for your information, the process of how a PPPoE connection is established is outlined below:

1. The router broadcasts a PPPoE Active Discovery Initiation (PADI) packet.
2. When the ISP's access concentrator receives a PADI packet, it sends a PPPoE Active Discovery Offer (PADO) packet to the client.
3. The host then looks through the many PADO packets it receives (as the PADI was a broadcast) and chooses one based on a few criterion.
4. The host then connects to the ISP's concentrator by sending a PPPoE Active Discovery Request (PADR) packet.
4. The access concentrator the accepts the connection by sending a confirmation packet to the client.

Once the confirmation is received, a Virtual Access Interface which inherits the properties of the Dialer interface is created and the session will run on this interface. Here, the traffic will leave the router through the Dialer Interface. This is how Cisco has implemented routing via dialer interfaces. It is to this interface on the router that the default route points thus taking the OUT traffic through the Dialer interface. When traffic comes in, it enters the network through the Virtual Access Interface as this is the interface that established the DSL connection.

To monitor the interfaces for traffic and bandwidth analysis, NetFlow can be enabled only on the interfaces that appears in the configuration. ie. the Dialer Interface along with the other physical interfaces and logical interfaces on the router. The Virtual Interface will automatically inherit the Dialer interface's properties when the DSL connection is to be established and will not show up in the configuration table.

When NetFlow data is exported, the IN traffic is captured on the Virtual Access Interface and the OUT traffic is captured on the Dialer Interface as this is how traffic has traversed.

A NetFlow cache entry with Dialer and Virtual Interface traffic will be as below:

IN TRAFFIC               OUT TRAFFIC
SrcIf    SrcIPaddress    DstIf    DstIPaddress    Pr    SrcP    DstP    Pkts
Fa0/0    192.16.3.7       Di0       20.4.10.14      06    043B     0747      2
Fa0/0    192.16.3.7       Di0       83.18.4.58     11     7B9A     05D2     1
Fa0/0    142.12.3.9       Di0       64.3.93.8       06     0BD0     01BB    1
Vi2        82.14.5.1        Local    91.63.6.3       32     8D41      B1A4    11
Vi2        84.20.12.46     Local   91.63.6.3       32     0E87      9CDC    170
Vi2        82.14.5.1        Local    91.63.6.3       2F    0000       0000     11
Fa0/0    192.16.3.7       Di0       92.37.54.12   06     070F      0DBB     4
Vi2        83.18.1.8        Fa0/0    91.63.6.3      11     05D2      7B9A     1
Vi2        92.3.4.72        Fa0/0    91.63.6.3      06     0DBB     070F      8
Vi2        8.23.15.46      Local    91.63.6.3       2F     0000      0000     170
Vi2        13.11.23.2      Fa0/0    91.3.6.3        11     D0A2     7B9A     1
Vi2        64.2.18.8        Fa0/0    91.3.6.3        06     01BB     0BD0     1
Fa0/0    19.16.3.7        Di0       13.11.23.23    11     7B9A     D0A2     2
Fa0/0    12.16.3.7        Di0       21.12.23.25    11     7B9A     AAF5     3

* All the IP Address have been changed and are randomly entered.

As you can see, NetFlow enabled on the Virtual Access interface has captured the IN traffic (categorized under SrcIf which is Source Interface) for the DSL connection and since traffic exits the router via the Dialer Interface due to Cisco's routing, the OUT traffic (categorized under DstIf which is Destination Interface) for the DSL is captured from the Dialer interface. In order to see the combined traffic statistics for the DSL connection, you need to combine the graphs for the Dialer Interface and the Virtual Interface.

Looking at a report for the interfaces, you can see that the graphs shows IN traffic for the Virtual Access Interface and the OUT traffic for the Dialer Interface and its not an easy job imagining them to be one especially when you want to see detailed reports on application, source, destination and both the IN and OUT traffic points.

The Interface Grouping feature in NetFlow Analyzer lets you group together different interfaces either from the same router or different devices to show the combined traffic statistics in a single graph. To create an Interface Graph, navigate to Device Group (option from Product Settings) and from here click on the Interface Group tab. From this link, you can select the interfaces to be grouped. You will be given an option to enter the Interface Group speed and here enter the speed of the Dialer interface (Virtual Access Interface wll have the same speed as it inherits the Dialers properties) and save the group.



The interface group created will show the combined graphs for both interfaces thus helping you get a clearer picture on the IN and OUT traffic for DSL link and also help in generating a complete report rather than having separate reports generated for each interface and then combining them. NetFlow Analyzer ensures that its not just the bandwidth monitoring that is made wasy, but the report generation too.

And a great thanks to Alec Waters who updated us about the behavior of ADSL connection through his post in our forums. You can follow Alec Waters on ManageEngine community from here.

Download | Interactive Demo | Product overview video | Twitter | Customers

Regards,
Don Thomas Jacob

Many of you out there who uses NetFlow Analyzer or is evaluating NetFlow Analyzer would certainly want to know how the product stores its data and does all the historic reporting.

NetFlow Analyzer processes the NetFlow data exported from the devices and stores it in the database for traffic analysis and reporting. NetFlow Analyzer's flexible data storage pattern is intended to achieve detailed data storage forever without having an impact on the hard disk space and also provide real time reporting.

Data stored on NetFlow Analyzer will help you to achieve following things:

1. Troubleshooting Network spikes

2. Capacity Planning

3. Historical Reporting

4. Billing

5. Trend Analysis

6. Understanding Traffic Pattern and much more.

Coming to the data storage, NetFlow Analyzer stores two types of data, Raw data and Aggregated data.

Raw Data Storage:

Raw data is each and every flow exported from the monitored interfaces of the routers. All the flows exported from the routers is stored in the NetFlow Analyzer database as raw data. Since, the raw data is each and every flow from the routers, it consumes lot of disk space and so is set to be stored for maximum of 30 days. Raw data storage is determined by the amount of flows the product receives from the monitored routers. To make calculation easier, the product itself can suggest how long one can store the raw data based on the free space available in the installation directory and the flow rate.

Raw data storage can be configured on the product by clicking on Product Settings --> Storage Settings --> Raw data Storage. There are also options available to alert you when free disk space goes below specified percentage and to automatically delete the older raw data when disk space goes below a specified percentage.

The raw data is used in the product when generating 'Troubleshoot' reports and the last 2 hours reports will be generated from the raw data. The raw data has complete port level information which helps in detailed analysis of traffic.

Aggregated Data:

Apart from the raw data storage, NetFlow Analyzer stores aggregated data which is stored for ever in the database. The aggregation mechanism will happen simultaneously at the back end along with the raw data storage. The aggregated data is stored based on top 100 fields of the application and conversation for every 10 minute interval and is further aggregated as time goes on.

The aggregation of NetFlow data collected is done to avoid high disk space usage without impact on reporting and performance. The aggregated data on NetFlow Analyzer is used for historical reporting, capacity planning and trend analysis.

Following explanation will help you to understand how Application data on NetFlow Analyzer is aggregated and stored in various tables.

Aggregation Mechanism for Application data:

Older data is repeatedly rolled up into less granular times (10 minute, 1 hour, 6 hour, 24 hour and weekly). The top 100 records of application based on octet value is stored for every 10 minute interval. As time goes, this data is further aggregated to an hourly table.

When we select time period 10:00 to 10:59, NetFlow Analyzer stores top 100 Application for each 10 minutes (10:00, 10:10, 10:20, 10:30, 10:40 and 10:50), this data will be under 10 minute table. From this six 10 minutes data, the 600 records pertaining to 10:00, 10:10, 10:20, 10:30, 10:40 and 10:50 would be aggregated and the top 100 would be moved to the 1 hour table pertaining to 10:00.

In the same manner, aggregation happens to the hourly table and the data is moved to 6 hour table then to daily table and finally weekly tables. Most recent data is stored with 10 minute granularity and data older than 90 days is stored with 1 week granularity.

The 10 minute table will have most recent data and data older than 25 hours is cleaned up. Following is how the data are repeatedly rolled out.

10 minute granular data is stored for 25 hours (beyond which the older data is deleted)

1 hour granular data is stored for 30 days

6 hour granular data is stored for 30 days

24 hour granular data is stored for 90 days

1 week granular data is stored forever

                                                         In the same way as applications, conversations are also aggregated and stored in the database for historic reporting. The Application, Source, Destination, Conversation and QoS reports generated for more than last 2 hour period will be generated from the Aggregated data. The granularity of data represented will change based on the time period you select.

1 Minute traffic Data Storage:

Apart from the raw data and aggregated data, NetFlow Analyzer stores 1 minute traffic data which is used for real time reporting purpose. The aggregation mechanism for the traffic data happens as the same way we explained for Application data. The traffic report generated for any time period which is less than 24 hour is generated with 1 minute granularity which will give you a detail picture of each and every transaction going IN and OUT.

One minute data storage can be configured on the product by clicking on Product Settings---------> Storage Settings-----> One Minute Data Storage Settings.

Hope this blog gives you a better understanding about the data storage pattern in NetFlow Analyzer and will help you use the product better.

Interactive Demo | Product overview video | Twitter | Customers

Regards

Praveen Kumar

Turns out to be a no contest !

NetFlow Analyzer with its capabilities to report on data ranging from the last minute to forever with new major features added almost every six months in new releases is one of the safest value for money tools. Check out our 30 day, full feature trial by downloading from here.

For those who needs to verify that the data reported by NetFlow is indeed correct, a combination of SNMP and NetFlow based solution will help. For this, try our product called OpManager which can give you not just SNMP based bandwidth reports, but can also report on device health and utilization, monitor all your network devices and do a lot more. You can even integrate NetFlow Analyzer and OpManager to get NetFlow reports from the OpManager GUI.

So, instead of having just one of the technologies, use the power of both to get the best out of your network.

Interactive Demo | Product overview video | Twitter | Customers

Regards,
Don Thomas Jacob

NetFlow Analyzer and RADIUS!

Whats is Radius ?

Remote Authentication Dial In User Service (RADIUS) is a networking protocol that provides centralized Authentication, Authorization, and Accounting management for computers to connect and use a network resources.

RADIUS enables centralized management of authentication data, such as user names and passwords. When a user attempts to login to a RADIUS client, such as a NetFlow Analyzer, the NetFlow Analyzer sends the authentication request to a RADIUS server, which is the centralized authentication server. The communication between the RADIUS client and the RADIUS server is authenticated and encrypted through the use of a shared secret, which is not transmitted over the network.

Configuring NetFlow Analyzer for Radius Authentication:

In order to configure users to access NetFlow Analyzer via Radius Server Authentication, we need to configure the radius server settings within the product. To configure Radius Server Credentials, the option is under Admin Operation ----------> Product Settings ---------> Advanced Settings Tab.

Following credentials need to be configured for Radius Server Authentication on NetFlow Analyzer:

Radius Server IP                                : IP address of the Radius server
   
Radius Server Authentication Port      : Port through which the radius server is listening for authentication requests from NetFlow Analyzer
   
Radius Server Protocol                       : Protocol used for authentication purpose

NetFlow Analyzer support variety of Authentication Protocol for Radius Server Authentication, They are;

PAP           : Password Authentication Protocol provides a simple method for the peer to establish its identity using a 2-way handshake.

CHAP         : Challenge-Handshake Authentication Protocol (CHAP) authenticates a user or network host to an authenticating entity.

MSCHAP    : MS-CHAP is the Microsoft version of the Challenge-handshake authentication protocol, CHAP.

MSCHAP2  :  Another version of Microsoft version of the Challenge-handshake authentication protocol, CHAP.

Radius Server Secret                         : Secret that is specified on the Radius Server

Authentication Retries                       : Number of retries for authentication

                                                    
Once the Radius server settings is configured on the NetFlow Analyzer, the next step is creation of user accounts.

User Creation on NetFlow Analyzer:

We can create users for NetFlow Analyzer from User Management page. Here, you need to enter a user name available in the RADIUS server and select the option to authenticate via Radius.When the created user tries to login to NetFlow Analyzer, he will authenticated via Radius Server. The Radius Server reads the request from the NetFlow Analyzer and checks the user name and password on its database and if the credentials are passed, the user will be directed to the NetFlow Analyzer web console.

                                                 
                                             
                        
With this type of secure authentication, we do not need to create user name and password locally in NetFlow Analyzer. RADIUS server authentication provides secure authentication and accounting. It is also possible to integrate RADIUS server with Active Directory so that you also get the capability for using user accounts from AD in NetFlow Analyzer.

Thanks

Praveen Kumar

NetFlow Analyzer Technical Team

Interactive Demo | Product overview video | Twitter | Customers

"It does an excellent job of accumulating our data flows so I can accurately research problems in the WAN/LAN. Since It only keeps the headers it is very efficient regarding storage. The the groups work well to help fine tune Application performance."

Find below the TOP 10 reasons for having close to 4000 enterprises use NetFlow Analyzer for bandwidth monitoring, traffic analysis and much more...

Download | Interactive Demo | Product overview video | Twitter | Customers

These reports will help you find if the network changes you implemented brought a positive or negative effect on the Internet link and you can even find if application performance has been affected using comparison reports with IP Groups. The advantages of using NetFlow and NetFlow Analyzer does not end there. To know more on the features and how else NetFlow Analyzer can help ypu, download and try the 30 day trial with free technical support today.

Download | Interactive Demo | Product overview video | Twitter | Customers

Regards,
Don Thomas Jacob

Recent improvement in the communication and broadband technology has made ISP's to offer better billing model to their customers based on their bandwidth usage. ISP's have their own standard technology to bill the customers which is billing based on 95th percentile. Some of the ISP's do offer billing based on the 90th percentile to attract customers, but as of now the industrial standard of billing the bandwidth usage is based on 95th percentile.

In this blog, we are going to have brief look on 95th percentile, NetFlow Analyzer reports with 95th percentile calculation and billing reports based on 95th percentile in NetFlow Analyzer.

95th Percentile :-

The 95th percentile is the standard of billing model and it has a specific meaning. In order to calculate the traffic rate for which you will be billed, ISP sorts the samples taken during your billing period, then ignores the highest five percent of those samples.

Traffic Graph of NetFlow Analyzer.

NetFlow Analyzer calculates and displays 95th percentile for the interfaces and IP groups. Given below are some example how NetFlow Analyzer calculates and shows the 95th percentile for both IN and OUT traffic.

In the following screen shot for the time period of 20 minutes, there is about 20 data points(1 minute granularity) for both IN and OUT. These data points for both IN and OUT are separately sorted in a descending order to calculate the 95th Percentile.

Given below is the calculation which shows how 95th Percentile is derived for the IN traffic, for the OUT traffic the  methodology is same as IN traffic.

IN Data Points = (114.06, 137.09, 159.53, 159.6, 160.06, 182.24, 182.45, 182.75, 205.06, 205.74, 227.96, 228.33, 228.39,228.71, 228.76, 250.98, 251.11, 251.4, 251.74, 273.87  )

Now the data points gathered are sorted in decesending order as below,

INData Points = (273.87, 251.7,251.4, 251.11, 250.98, 228.76, 228.71, 228.39, 228.33, 227.96, 205.75, 205.06, 182.75, 182.45, 182.24, 160.06, 159.6, 159.53, 137.09, 114.06)

From this 20 data points the top 5% of the point is been ignored and the one next is considered as 95th percentile IN. The data point ignored is 273.87 and the 95th Percentile is 251.7

NetFlow Analyzer traffic graphs are based on 1 minute granularity, the above example calculation for the 95th percentile is for traffic graph on NetFlow Analyzer which is based on 1 minute granularity. NetFlow Analyzer have billing functionality which is peculiarly designed for ISP and Enterprise to bill their users and customers based on their usage.

Billing Reports on NetFlow Analyzer:-

NetFlow Analyzer offers a functionality to users and ISP to bill the departments / clients based on their usage. For this, we need to create a bill plan on NetFlow Analyzer and associate the interfaces or IP group to the bill plan. Once the bill plan is created, NetFlow Analyzer gathers the traffic usage of the interface or IP group associated to bill plan for the billing period and generates the billing report based on the 95th percentile. The one important thing being that billing reports in NetFlow Analyzer is based on 5 minute granularity for the whole billing period. In the billing module, you can select the opiton to generate billing report based on 95th percentile combined for both IN and OUT traffic or separately.

Below given is an example screen shot of NetFlow Analyzer billing report based on 95th Percentile by merging IN and OUT.

Traffic analysis involves monitoring the network to find out who and what used the bandwidth and at what time. The analysis also involves having a detailed understanding on the network protocol distribution. One may ask why is there the need to identify the protocols in the network when you see the applications being used and their related conversations.

The protocol distribution helps network administrators find the bandwidth used by each protocol in the network. This helps find if any unwanted (read as: not mean to be used) protocols are being used in the network and based on this, the network administrator can reallocate this bandwidth to more critical applications using other protocols.

It also helps you determine if any inactive application protocol is being used in the network taking away valuable bandwidth. To give a real example, an administrator was expecting to see only negligible bandwidth usage by L2TP traffic in his network. He looked at the protocol distribution graph and what he found was L2TP occupying about 10% of the total traffic. Now, that is called sacrilege in network terminology !

Again, having a track on the network protocol distribution can even help quickly solve network problems. When the network is slow, instead of analyzing each application one by one, you can take a look at the protocol distribution to find if there is any unexpected change in the pattern and then analyze the protocol to find what application is involved in bandwidth.

And is it not much more easier to identify non compliance traffic based on protocol first and then drilling down to find the application and conversations involved rather than checking out for each applications in the list of thousands of applications?

Since Cisco and many of the major vendors in the market have already come up with NetFlow or a  similar flow format technology, one does not have to wonder how to obtain such an information from the routing or switching devices. All you need is configure your device to export NetFlow packets to ManageEngine NetFlow Analyzer which supports almost all the major flow formats, and the product will capture the flow packets to generate the reports. Now that is called Up and Running in a matter of minutes.

It really does not do a big deal if you can just see the protocol distribution in the network. What you need is the ability to see the source and destination associated with each conversation corresponding to a protocol and this is exactly what NetFlow Analyzer can also do. Check out the screen shots to see protocol distribution reports available in NetFlow Analyzer.

Download | Interactive Demo | Product overview video | Twitter | Customers