Quite a number of organizations uses some form of DSL connection for cost effective connectivity to the Internet of which ADSL is gaining more popularity due to the advantages it provides like higher security, IP Address conservation, per session accounting, etc. The ADSL connection requires the device to have a Dialer interface which establishes the connection after which a Virtual Access Interface will be created and the PPPoE session will run on this Virtual Access Interface. The Virtual Access Interface thus created inherits the properties of the Dialer interface.
Many users who use NetFlow data to monitor such interfaces would have seen that the Dialer Interface reports only outbound traffic and a Virtual Interface is automatically discovered and reporting inbound traffic. Let us see what is the reason for this and how NetFlow Analyzer can help.
As stated, it is the Dialer Interface created by the user that establishes the connection to the DSL provider and is the actual interface available on the router. Just for your information, the process of how a PPPoE connection is established is outlined below:
1. The router broadcasts a PPPoE Active Discovery Initiation (PADI) packet.
2. When the ISP's access concentrator receives a PADI packet, it sends a PPPoE Active Discovery Offer (PADO) packet to the client.
3. The host then looks through the many PADO packets it receives (as the PADI was a broadcast) and chooses one based on a few criterion.
4. The host then connects to the ISP's concentrator by sending a PPPoE Active Discovery Request (PADR) packet.
4. The access concentrator the accepts the connection by sending a confirmation packet to the client.
Once the confirmation is received, a Virtual Access Interface which inherits the properties of the Dialer interface is created and the session will run on this interface. Here, the traffic will leave the router through the Dialer Interface. This is how Cisco has implemented routing via dialer interfaces. It is to this interface on the router that the default route points thus taking the OUT traffic through the Dialer interface. When traffic comes in, it enters the network through the Virtual Access Interface as this is the interface that established the DSL connection.
To monitor the interfaces for traffic and bandwidth analysis, NetFlow can be enabled only on the interfaces that appears in the configuration. ie. the Dialer Interface along with the other physical interfaces and logical interfaces on the router. The Virtual Interface will automatically inherit the Dialer interface's properties when the DSL connection is to be established and will not show up in the configuration table.
When NetFlow data is exported, the IN traffic is captured on the Virtual Access Interface and the OUT traffic is captured on the Dialer Interface as this is how traffic has traversed.
A NetFlow cache entry with Dialer and Virtual Interface traffic will be as below:
IN TRAFFIC OUT TRAFFIC
SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts
Fa0/0 192.16.3.7 Di0 20.4.10.14 06 043B 0747 2
Fa0/0 192.16.3.7 Di0 83.18.4.58 11 7B9A 05D2 1
Fa0/0 142.12.3.9 Di0 64.3.93.8 06 0BD0 01BB 1
Vi2 82.14.5.1 Local 91.63.6.3 32 8D41 B1A4 11
Vi2 84.20.12.46 Local 91.63.6.3 32 0E87 9CDC 170
Vi2 82.14.5.1 Local 91.63.6.3 2F 0000 0000 11
Fa0/0 192.16.3.7 Di0 92.37.54.12 06 070F 0DBB 4
Vi2 83.18.1.8 Fa0/0 91.63.6.3 11 05D2 7B9A 1
Vi2 92.3.4.72 Fa0/0 91.63.6.3 06 0DBB 070F 8
Vi2 8.23.15.46 Local 91.63.6.3 2F 0000 0000 170
Vi2 13.11.23.2 Fa0/0 91.3.6.3 11 D0A2 7B9A 1
Vi2 64.2.18.8 Fa0/0 91.3.6.3 06 01BB 0BD0 1
Fa0/0 19.16.3.7 Di0 13.11.23.23 11 7B9A D0A2 2
Fa0/0 12.16.3.7 Di0 21.12.23.25 11 7B9A AAF5 3
* All the IP Address have been changed and are randomly entered.
As you can see, NetFlow enabled on the Virtual Access interface has captured the IN traffic (categorized under SrcIf which is Source Interface) for the DSL connection and since traffic exits the router via the Dialer Interface due to Cisco's routing, the OUT traffic (categorized under DstIf which is Destination Interface) for the DSL is captured from the Dialer interface. In order to see the combined traffic statistics for the DSL connection, you need to combine the graphs for the Dialer Interface and the Virtual Interface.
Looking at a report for the interfaces, you can see that the graphs shows IN traffic for the Virtual Access Interface and the OUT traffic for the Dialer Interface and its not an easy job imagining them to be one especially when you want to see detailed reports on application, source, destination and both the IN and OUT traffic points.
The Interface Grouping feature in NetFlow Analyzer lets you group together different interfaces either from the same router or different devices to show the combined traffic statistics in a single graph. To create an Interface Graph, navigate to Device Group (option from Product Settings) and from here click on the Interface Group tab. From this link, you can select the interfaces to be grouped. You will be given an option to enter the Interface Group speed and here enter the speed of the Dialer interface (Virtual Access Interface wll have the same speed as it inherits the Dialers properties) and save the group.
The interface group created will show the combined graphs for both interfaces thus helping you get a clearer picture on the IN and OUT traffic for DSL link and also help in generating a complete report rather than having separate reports generated for each interface and then combining them. NetFlow Analyzer ensures that its not just the bandwidth monitoring that is made wasy, but the report generation too.
And a great thanks to Alec Waters who updated us about the behavior of ADSL connection through his post in our forums. You can follow Alec Waters on ManageEngine community from here.
Download | Interactive Demo | Product overview video | Twitter | Customers
Regards,
Don Thomas Jacob
Turns out to be a no contest !
NetFlow Analyzer with its capabilities to report on data ranging from the last minute to forever with new major features added almost every six months in new releases is one of the safest value for money tools. Check out our 30 day, full feature trial by downloading from here.
For those who needs to verify that the data reported by NetFlow is indeed correct, a combination of SNMP and NetFlow based solution will help. For this, try our product called OpManager which can give you not just SNMP based bandwidth reports, but can also report on device health and utilization, monitor all your network devices and do a lot more. You can even integrate NetFlow Analyzer and OpManager to get NetFlow reports from the OpManager GUI.
So, instead of having just one of the technologies, use the power of both to get the best out of your network.
Interactive Demo | Product overview video | Twitter | Customers
Regards,
Don Thomas Jacob
"It does an excellent job of accumulating our data flows so I
can accurately research problems in the WAN/LAN. Since It only keeps the
headers it is very efficient regarding storage. The the groups work
well to help fine tune Application performance."
Find below the TOP 10 reasons for having close to 4000 enterprises use NetFlow Analyzer for bandwidth monitoring, traffic analysis and much more...
Download | Interactive Demo | Product overview video | Twitter | Customers
cheers
This blog may need prior reading of my first blog about Flexible NetFlow. We have already discussed about the advantages of Flexible NetFlow and migration from traditional NetFlow versions to FNF. To make this transition smooth Cisco provides the option of pre-defined flow records which can be used to configure Flexible NetFlow without investing a lot of time. And as I mentioned earlier it also helps your existing NetFlow V9 collector to parse exported data. However to use Flexible NetFlow to its fullest potential or to monitor a specific network behavior, you should create your own customized records.
Let’s see how to configure Flexible NetFlow to export flow statistics. Flexible NetFlow export can be configured in three easy steps.
1. Configure the exporter
2. Configure the Flow Monitor with the pre-defined Flow Record and Flow Exporter attached to the monitor.
3. Add the Flow Monitor to the interface to monitor either ingress (input) or egress (output traffic).
1. Configuring Exporter
Flow exporter can be configured with a unique name. Multiple Flow exporter profiles can be configured. Below is the configuration to configure Flow Exporter.
flow exporter <exporter name>
destination <ip address of ME NFA>
transport udp <port number>
Example configuration:
flow exporter me_nfa_analyzer
destination 192.168.1.1
transport udp 9996
2. Flow Monitor and Flow record configuration
Flow record configuration defines the fields exported via NetFlow protocol. Flexible pre-defined flow records are based on the original NetFlow ingress or egress caches. Cisco provides a unique keyword to identify the pre-defined records and these records can associated with a Flexible NetFlow Flow record configuration. The Flexible NetFlow "netflow-original" and netflow ipv4 original-input are predefined records and these two records can be used interchangeably to export the basic key fields and time stamp fields. Flow monitors can also include packet sampling information if sampling is required.
flow monitor <monitor name>
record netflow-original
exporter <exporter name>
cache timeout active <seconds>
cache timeout inactive <seconds>
Example Configuration:
flow monitor me_nfa_monitor
record netflow-original
exporter me_nfa_analyzer
cache timeout active 60
3. Adding Flow Monitor to the interface
Flow Monitor has to be attached to a specific physical or logical interface to export flow statistics for that particular interface. Below is the configuration to attach flow monitor to a specific interface.
interface <interface name>
ip flow monitor <monitor_name> input
Example Configuration:
interface serial0/0
ip flow monitor me_nfa_monitor input
And the above configuration can be verified by "show flow monitor" command. As I mentioned earlier Flexible NetFlow has numerous advantages and has the power of supporting new performance monitoring statistics as soon as they are available. Flexible NetFlow is an evolving technology available in Cisco devices to help with visibility into how network assets are being used and the network behavior.
Please find more information on FNF here.
ManageEngine constantly studies the market and user demands to support new technologies. In fact ManageEngine NetFlow Analyzer is the first tool to support multiple bandwidth and performance monitoring technologies like NetFlow, NBAR and CBQoS in the market. And currently ManageEngine NetFlow Analyzer supports Flexible NetFlow without any issues. Please write your questions to netflowanalyzer-support@manageengine.com. We are happy to assist you at any moment.
Thanks
Raj
Download | Interactive Demo | Product overview video | Twitter | Customers
Hello,
Some of our community folks using ME NetFlow Analyzer to monitor their Juniper firewalls SSG 500 series. It supports policy based netflow/JFlow export.
Can you share us the netflow/JFlow configuration to enable NetFlow/JFlow on these firewalls?
Thanks
Raj
Flexible NetFlow is the next generation flow export technique promoted by Cisco Systems. As the word depicts it is highly flexible based on user requirements and to monitor specific network behaviour. Traditional NetFlow used a fixed seven tupple of IP information to identify a flow most of the time. Advantages of Flexible NetFlow
1. Flexibility to choose the desired export fields.
2. Reduce the number of flows and allows CPU to perform efficient routing and switching
3. Convergence of multiple accounting technologies into one accounting mechanism
Flexible NetFlow and NetFlow V9
The export protocol of choice for Flexible NetFlow is the NetFlow Version 9 export protocol, but unfortunately and to date, NetFlow Version 5 has been a much more widely used protocol because of the legacy Cisco IOS® Software images that are still around that supported the NetFlow v5 export protocol only and worked very well. However Cisco claims the future is going to be Flexible NetFlow. And believe it this migration is going to very smooth since Flexible NetFlow can also be configured to export some predefined flow records using the NetFlow Version 5 protocol format for backward compatibility. This helps your existing collectors can work with Flexible NetFlow until you find a real requirement to use additional fields offered by Flexible NetFlow.
Flexible NetFlow Configuration
Traditional NetFlow configuration is pretty much straight forward. Flexible NetFlow consists of components that can be used together in several variations to perform traffic analysis and data export, and the new command-line interface (CLI) configuration follows the same traditional logic.In this user-defined flow records and the component structure of Flexible NetFlow make it easy to create various configurations for traffic analysis and data export on a networking device with a minimum number of configuration commands.
Flexible NetFlow consists of components that can be used together in several variations to perform traffic analysis and data export, and the new command-line interface configuration follows the same traditional logic.
Let's see this components in detail
Flow Monitor:
A Flexible NetFlow Flow Monitor describes the NetFlow cache or information stored in the cache. The Flow Monitor contains the Flow Records or key and non-key fields within the cache. Also, part of the Flow Monitor is the Flow Exporter which contains information about the export of NetFlow information including the destination address of the NetFlow collector. The Flow Monitor includes various cache characteristics including the timers for exporting, the size of the cache and if required, the packet sampling rate.
Flow Record:
A Flow Record is a set of key and non-key NetFlow field values used to characterize flows in the NetFlow cache. Flow Records may be pre-defined for ease of use or customized and user defined. A typical pre-defined record will aggregate flow data and allow users to target common applications for NetFlow. User defined records will allow selection of specific key or non-key fields in the Flow Record. The user defined field is the key to Flexible NetFlow allowing a wide range of information to be characterized and exported by NetFlow. It is expected that different network management applications will support specific user defined and pre-defined Flow Records based on what they are monitoring (ie: security detection, traffic analysis, capacity planning).
Flow Exporter:
The Flexible NetFlow Exporter allows the user to define where the export can be sent, the type of transport for the export and properties for the export. Multiple exporters can be configured per Flow Monitor or the same exporter can be used by multiple monitors.
The following figure shows the flow monitor and it components.
In our next blog we are going to use a pre-defined (defined in IOS itself) flow record to export netflow records using Flexible Netflow. In the meanwhile if you have any queries. please write to netflowanalyzer-eesupport@manageengine.com
Thanks
Raj
Download | Interactive Demo | Product overview video
We have posted a number of blogs to share information on how to use
NetFlow technology and NetFlow Analyzer to manage your network better.
Those blogs will definitely continue to give you more ideas to put the
product to better usage but we will also discuss about some of the
common issues that you may have come across in the product and how they
can be resolved.
NetFlow Analyzer
generates traffic reports based on the NetFlow packets exported from
the router. Based on the information in the NetFlow packets, the
product displays the traffic passing through the interfaces of the
exporting device.
One issue that is frequently reported is that the traffic utilization shown in NetFlow Analyzer is more than the actual traffic on the interface. Reports
showing more than actual utilization or more than 100 % utilization can
be resolved quickly by checking a few points on the exporting device
and the product.
Incorrect active timeout:
The
traffic reports in NetFlow Analyzer is shown with a 1 minute
granularity, ie. NetFlow Analyzer shows details of the traffic for each
minute. By default, the active timeout on the NetFlow exporting devices
is 30 minutes, which means that the information about the traffic that
passed through the interface in the previous 30 minutes is exported at
the 30th minute.
Since NetFlow Analyzer reports traffic
every minute, the export of 30 minutes information all at once leads to
the product's reports showing a spike every 30 minutes. The incorrect
traffic details for that minute leads to showing incorrect speed which
thus leads to worng utilization calculation. To avoid this, simply
check if the active timeout on the router is set to 1 minute using the
command "ip flow-cache timeout active 1""
Multiple NetFlow commands:
NetFlow can be enabled on the router using any one of the three commands:
ip
route-cache flow : - This command can be applied on all main
interfaces and will automatically enable NetFlow on the sub interfaces
too. This command accounts for the IN traffic across an interface.
ip
flow ingress :- Some of the newer IOS supports this command
which also accounts for the IN traffic across an interface. The
difference is that this command needs to be applied on a sub-interface
level
ip flow egress :- The same as 'ip flow ingress' but this command accounts for the OUT traffic across an interface.
NetFlow
can be enabled on the interfaces of the router by applying any one of
the above mentioned command, but most of the netwrok admin enable
either "ip flow ingress" or "ip route-cache flow" on the interfaces for
traffic accounting. When all these commands are applied on the
interfaces, it causes the same traffic to be counted multiple times
again causing the product to show incorrect traffic stats and thus
incorrect utilization reports.
Incorrect link speed in NetFlow Analyzer:
NetFlow Analyzer calculates the utilization based on the link speed. For
example, if the link has capability to handle 1 Mbps and the actual
traffic passing through an interface is about 512 Kbps, the utilization graph in NetFlow Analyzer displays the traffic percentage as 50 %. Here
is the formula which explains the utilization calculation on NetFlow
Analyzer.
Utilization = Actual Speed/Link Speed * 100
So,
if the link speed is not updated properly in NetFlow Analyzer, the
utilization shown in NetFlow Analyzer will be different than the
actual. NetFlow Analyzer can determine the interface speed if you set
the appropriate SNMP Port and Community for the router on NetFlow Analyzer. This can be done from the 'Set SNMP Parameters' icon on the
'Interface View' right next to the router name or you can set the
interface speed manually for each interface on NetFlow Analyzer (from
the Edit Settings icon on the 'Interface View' next to the interface
name). You can refer to this blog for more details.
Non dedicated burstable bandwidth:
Certain ISPs allows
you to use over the allocated bandwidth depending on the other
customers sharing that link. So, even though the max bandwidth is
2Mbps, the ISP may allow you to use even more based on availability.
This also affects the accurate reporting on NetFlow Analyzer causing
incorrect bandwidth utilization values and even more than 100%.
ESP and GRE traffic:
This is another reason for traffic
to get double counted in NetFlow Analyzer. With NetFlow data, the
tunnel traffic will be accounted as the normal traffic before
encryption and again as the encrypted traffic. NetFlow Analyzer have an
option to filter this kind of encrypted tunnel traffic from the
reports. This option is availble under Product Settings - Advance
Settings - ESP or GRE Filter.
To know more about the about ESP and GRE traffic double count, check this link.
If none of the above resolves the issue, please find the technical explanation on what could still be causing this:
Any
analyzer tools calculates the OUT traffic of an interface based on the
IN traffic of the interface that sends traffic to it. When traffic is
passing from higher speed interface to lower speed interface, the
calculation of OUT traffic from a higher speed IN traffic causes
incorrect traffic utilization to be shown on the OUT traffic.
The
above reason for more than 100 % utilization on OUT traffic can be
resolved by enabling only "ip flow egress" on all the interfaces.
If you have any further queries on this, kindly send us a email at netflowanalyzer-support@manageengine.com.
Thanks
Praveen
Download | Interactive Demo | Product overview video
Being a niche player in the SAAS market, Zoho brings an amazing level of engineering expertise to ManageEngine in building highly secure and scalable distributed applications. And hopefully you know, Adventnet has recently changed its name to Zoho Corp and formed three divisions namely ManageEngine, Zoho, and WebNMS.
Growing network needs complicate the job of network administrators and bring in new challenges. Network Administrators need robust,cutting-edge network management tools to quickly troubleshoot network incidents and increase the network performance. However considering the economic situation, it is very important to choose the right application which can leverage on network performance management data from multiple technologies and of course at an affordable cost.
ManageEngine NetFlow Analyzer team constantly interacts with its customers, technology companies and VARS to prioritize the road map. Whenever a new technology is introduced in the product, all existing customers see an immediate value by means of simple free upgrade instead of paying a hefty price. Here the ROI includes cutting bandwidth upgrade costs due to increased visibility using ManageEngine NetFlow Analyzer, avoid unauthorized bandwidth usage and increase the efficiency of business critical applications with almost zero implementation cost.
Multiple technologies - Single Solution:
Cisco NetFlow:
Cisco's NetFlow technology exports flow records from any IOS capable routers and switches. The exported flow records contain information about protocols, ports, source, destination IP addresses and much more.
NetFlow Analyzer provides several instant reports to monitor bandwidth including top talkers, top protocols, top conversations, and more. Apart from these pre-defined bandwidth reports, NetFlow Analyzer also includes options to search for specific bandwidth usage details based on IP address, host name, protocol, and more.
Bandwidth Monitoring without Probes
NetFlow Analyzer does network bandwidth monitoring using NetFlow. NetFlow exports are collected, correlated, and analyzed to get granular details to monitor bandwidth usage across each WAN link. There is no need for hardware probes to monitor bandwidth usage. NetFlow Analyzer is an all software solution which is suitable for both Windows and Linux.
Real-time Bandwidth Monitoring
Bandwidth monitoring reports for each interface shows the current, average, and peak bandwidth usage patterns across each NetFlow-enabled interface. With these bandwidth usage statistics you can get instant visibility into how much bandwidth was used up by hosts, applications, and conversations across a specific interfaces.
Application-wise Bandwidth Distribution
To monitor bandwidth utilized by different applications, NetFlow Analyzer gives you instant visibility into which applications are using up maximum bandwidth. You can also drill down to see the top sources, destinations and conversations using the bandwidth. With such granular detail, network troubleshooting and problem resolution take far less time than with traditional tools.
Cisco NBAR:
Cisco NBAR (Network Based Application Recognition) engine runs on the IOS and does deep packet inspection to identify applications riding on regular ports. For example TCP 80 can be identified as kazza2, BitTorrent, Napster etc. The respective utilization, volume and speed can be polled through SNMP protocol over time.
NBAR reports are very useful to set the Quality of Service (CB-QoS) policies. NBAR and QoS policies can work together to prevent bandwidth stealing applications and increase the efficiency of business critical applications.
Cisco CB-QoS (Class Based - Quality of Service):
We have discussed a lot about deploying CB-QoS policies for improved network performance. You can find CB-QoS blog series in this link. Cisco CB-QoS is the simplest way to prioritize network traffic.
Having insights over pre and post policy metrics, network administrators can modify their CB-QoS policy configuration for improved performance and to avoid any impact to business critical applications due to misconfiguration.
This is why we call ManageEngine NetFlow Analyzer is a powerful traffic analysis and forensic solution for a network of any size. Try our 30 days all feature version and write your queries to netflowanalyzer-support@manageengine.com
Thanks
Raj