Cisco Nexus 7000 and NetFlow Analyzer – Bandwidth Monitoring for Data Center Switches

Data Centers need no introduction to anyone associated with IT. With more businesses and services moving to SAAS models, the importance of Data Center networks scaling up and delivering the best possible performance has increased. It is to handle such important environments that Cisco introduced the Nexus series devices of which Nexus 7000 is the flagship model. The Nexus 7000 series is highly scalable and has a switching capacity of up to 15 tbps..Yes, 15 Terabits per second !

The Cisco Nexus devices runs on the new NX-OS software. This software, designed specifically for the Nexus devices, has quite a lot of innovative features and capabilities, more of which you can read about from here .

Instead of writing more on Cisco Nexus and its features, let me quickly get to what the title talks about. Traffic management and bandwidth monitoring in data centers is as important as having a device capable of handling huge traffic. With many business critical applications being hosted from the data center, bandwidth choke by non critical applications, network spikes, incorrect application priorities, possible virus attacks and such are unacceptable. You might have already read that NetFlow is one of the best available bandwidth monitoring technologies available, considering packet capture and SNMP . Cisco Nexus, like all their major devices, supports NetFlow export and so bandwidth monitoring is not a feature you will miss out in your data center.

When dealing with such high traffic volumes, you also need a flow monitoring software that is capable of handling such high rates. ManageEngine has NetFlow Analyzer to help in such scenarios. NetFlow Analyzer has capabilities to handle up to 20,000 interfaces through its various editions .

Now to enabling NetFlow on the Nexus device:

NetFlow configuration on Nexus is not done the same way as on an IOS device or on the Cisco CAT devices. Cisco Nexus supports the new flexible NetFlow with some twist !

Fexible NetFlow requires a flow record, flow monitor and flow exporter to be configured for NetFlow data to be exported. You can check this blog to know more about each component of flexible NetFlow and how to configure flexible NetFlow.

Unlike other devices, Nexus requires NetFlow to be enabled from the global config and then configure the rest of the NetFlow specific details depending on user requirements and flow collector details. From the global config mode, enable NetFlow using the command ‘feature netflow’. After enabling NetFlow feature, one should create flow records, flow exporters and flow monitors

Flow Records:

Flow records define the information included in a flow with key and non key fields. A flow is considered as a unique and thus part of the same conversation if all the key fields match. If a flow has one key field different from another flow, then it is considered a new flow and thus a new conversation. NetFlow v5 or traditional NetFlow has fixed key and non-key fields. The key fields used in NetFlow v5 are:

Source IP address
Destination IP address
Source port number
Destination port number
Layer 3 protocol type
ToS byte
Input logical interface (ifIndex)

Flexible NetFlow gives users the ‘flexibility’ to define the key fields and the additional fields (non-key fields) that has to be included in a flow record. A user can define key fields of flexible NetFlow using ‘match’ commands in the NX-OS. Some of the match commands available are:

match ip protocol
match ip tos
match ipv4 destination address
match ipv4 source address

One can also define the additional fields that has to be included in the exported flows. This is done using the ‘collect’ statement. Some of the collect statements available in NX-OS are as below:

collect counter bytes
collect counter packets
collect routing destination as
collect routing source as

If you prefer to export NetFlow from Nexus with the same the fields as available in NetFlow version 5, you do not need to create a flow record. You can instead attach the ‘netflow-original’ record to a flow monitor which can then be associated to an interface. Before we move to flow monitor, lets go to flow exporters.

Flow Exporter:

The flow exporter has information on the exported NetFlow version, interface through which flows are exported, the destination of NetFlow packets and the port used for sending the exported NetFlow packets. With flexible NetFlow, you can even define DSCP values for the flows exported through the network.

Nexus even supports sampling in flexible NetFlow. Sampling feature lets a user specify that only 1 out of ‘n’ packet is taken by NetFlow for gathering statistics based on the sampling interval set on the device. This helps reduce NetFlow load in very low bandwidth networks. We do not recommend sampling unless you are experiencing bandwidth chokes due to NetFlow export as sampling can reduce the visibility of traffic details. This is because, say sampling is set to 1 in 100, only 1 in 100 packets are taken for traffic analysis and based on the information in this 1 packet, traffic for the rest is constructed.

If you still prefer to go ahead with sampling, simply create a sampler and set the sampling interval. This is done from the global configuration mode:

Nexus(config)#sampler MENFASample
Nexus(config-flow-sampler)#mode 1 out-of 100

Flow Monitor:

Now you need to associate the flow record and exporter to the interfaces from which NetFlow collection is to be done. This is where flow monitor comes into the picture. A flow monitor has to be created and to this, the flow record and flow exporter is associated. In case you are using sampling, then the sampler also has to be associated with the flow monitor. This flow monitor is then associated with each interface of the device from which you need NetFlow stats.

The active and inactive timeout which is used to specify how frequently long lived flows and expired flows are exported from the device cache is done from the global configuration mode. The timeout values are necessary as the default values are not in tandem with NetFlow Analyzer’s lowest reporting granularity of 1 minute.

With the cache timeout added, the NetFlow configuration on a Cisco Nexus is finished and all you need is to look at NetFlow Analyzer for your reports.

Nexus(config)# interface ethernet 2/1
Nexus(config-if)# ip flow monitor NFAmonitor input [sampler name]   // Sets the flow monitor to capture IN traffic. Enter sampler name if used.
NOTE: Apply the above command on every interface of the switch.
Nexus(config-flow-monitor)# copy running-config startup-config

With NetFlow enabled and being exported from your Nexus, you start monitoring your data center traffic to ensure application and service uptime. Most importantly, to monitor data centers where uptime is not a priority but a necessity, what else other than NetFlow Analyzer?