In our previous blog, post we discussed the problems classified by the Advanced Security Analytics Module (ASAM) in NetFlow Analyzer along with how each problem is classified into a number of security events. This blog is about how the ASAM identifies each event in the suspect flow category. We discussed in detail the first three events on the suspect flow category in the previous blog.

This blog with cover TCP-based security events. TCP is the most reliable protocol to send and receive data between servers and clients. Since TCP is reliable, it assures orderly delivery of transmitted data. To understand how this reliable protocol can cause security threats on a network, we need to understand a few basics about this protocol.

Three-way handshake

TCP is connection-oriented. Before data transmission, a reliable connection must be established with the destination host. Once the connection is established and it has been acknowledged, data transmission happens between the source and destination hosts. The connection establishment, acknowledgment, and connection termination are actively carried by TCP flags.

Flags

URG: Urgent pointer field significant

ACK: Acknowledgment field significant

PSH: Push function

RST: Reset the connection

SYN: Synchronize sequence numbers

FIN: No more data from sender

In order to establish a connection, host A sends a SYN packet to host B. Host B receives the SYN and sends back a SYN-ACK message to host A. Host A receives the SYN-ACK message and sends an ACK message back to host B, which will establish connection between these two hosts for data transmission, forming the three-way handshake.

TCP-based events

  1. Excess empty TCP packets
  2. Excess short TCP handshake packets
  3. TCP null violations
  4. TCP SYN violations

Excess empty TCP packets

A TCP packet’s size should be above 40 bytes, including the header and the payload. If a TCP packets is transmitted with header information and without a payload, it’s called an empty TCP packet. TCP packets without a payload will be exactly 40 bytes and with the TCP FLAGS value IN (25–27, 29–31).

Excess short TCP handshake packets

TCP packets with a size between 40 to 44 bytes and TCP flags value IN (19/ASF, 22/ARS, 23/ARSF) are called short TCP handshake packets.

TCP null violation

In a TCP-based connection, the TCP flag determines what action is being performed. If the TCP flag is set to “0”, then it leads Null violation. A flow with TCP flag set to “0” is considered as TCP null violation.

TCP SYN Violation

Let’s take an example of a web server hosted on a network with a request to the web server coming from an external IP or network.

The external IP sends a SYN packet to the web server to establish a connection. The web server acknowledge the SYN by sending a SYN-ACK message and waits for an ACK message from the external IP. The web server maintains a queue of connections waiting to be completed. The queue will be emptied if the ACK message from the source host reaches the web server a millisecond after the SYN-ACK message.

This scenario can distorted by a TCP SYN attack or TCP SYN violations.

The source host that is contacting the web server will generate TCP SYN packets with random source addresses. While waiting for the ACK response to the SYN ACK message, a connection queue of finite size on the web server keeps track of connections waiting to be completed. This queue typically empties quickly since the ACK response is expected to arrive a few milliseconds after the SYN ACK message. The web server sends a SYN ACK message back to the random source address and adds an entry to the connection queue. Since the SYN ACK message is destined for an incorrect or non-existent host, the last part of the three-way handshake is never completed and the entry remains in the connection queue until a timer expires. This is TCP SYN violations or attack. TCP flows with TCP flags with a value equaling 2/SYN is called a SYN violation.

Download | Interactive demo | Customers | Bandwidth monitoring | Network security | CBQoS monitoring