Flow Technologies Explained
The ever-increasing development of IP networks as well as the reliance on network services for business applications, has increased the demand for manageable high capacity networks. Thus, network operators are implement monitoring technologies to provide network-wide visibility, allowing them to increase network performance for their users. These technologies are specifically created to provide network operators an access to IP flow information, enabling better control and network usage management throughout their network. This article will discuss the basics of flow technologies: the most popular, their differences, and there uses in modern networks.
What Is Flow?
Network flow monitoring technologies utilize packet sampling and are used in large data networks that contain switches, routers and firewalls. These devices collect “flow data” and export that information to “flow collectors” which are responsible for reception, storage and pre-processing of flow data.
A flow or a network flow is defined as a unidirectional or bidirectional sequence of packets between two endpoints with common attributes. Network devices differentiate flows depending upon their architecture, implementation, and device configuration. Network devices including Switches and Routers collect flow information on the traffic that is received or sent through them. This information can then be logged locally or sent to an external information collector that will provide fine-grained analysis.
The data collected provides vital information about traffic in the network and is used to aid billing, traffic engineering, capacity planning, and traffic analysis for policy decisions.
Below are the most popular technologies that capture NetFlow IP operational data from network links or devices.
NetFlow
Introduced by Cisco, NetFlow is the most widely used traffic monitoring system, and is considered the de facto industry standard for flow data statistics in network communication.
NetFlow enables network devices to look at the packets arriving on the interfaces and captures traffic statistics per flow. Each packet within a Switch/Router that is forwarded is examined for a certain set of IP packet attributes that identify the packet amongst others. An IP “Flow” is made up of a set of 5 attributes and can have up to 7 total. The flow data is sampled, then saved locally, externally or in a flow cache. NetFlow cache entries are created by the first packet of a flow, maintained for similar flow characteristics, and exported to collectors periodically based on flow timers or flow cache management.
The “Flow” groups can be created using these attributes:
- Source/Destination IP address
- Source/Destination ports
- Protocol interface and Class of Service.
All packets that match the criteria of an already established flow group are added to that group, if a group has not been created, and the packet is unique, a new group is created. This is an effective and scalable solution to monitor all sessions and flows utilizing your network devices.
A typical NetFlow monitoring set-up can be seen in the simple diagram below with its main components, and a simple breakdown of the NetFlow Process.
- Flow Export or Transport Mechanism: aggregates packets into flows and sends data to the Collector for reporting and analysis.
- Flow Collector: Typically, a server that does the actual data processing and traffic analysis.
- Analysis application: Analyzes received flow data can be software based, such as Solarwinds Network Traffic Analyzer
- NetFlow Cache (sometimes referred to as Data source or Flow Cache): Stores the IP Flow information.
NetFlow version 9, which is now an IETF standard known as IP Information Export (IPFIX), is the new standard for transporting information from Switches and Routers to a collector.
So, in review, NetFlow works by collecting all IP Traffic statistics on interfaces of a Cisco router or switch where NetFlow is enabled. Then exports the statistics as NetFlow records toward at least one NetFlow collector. These records will then be processed to perform traffic analysis and present the info in a user-friendly format. By analyzing these flow records, a picture of network traffic flow and volume can be built.
JFlow or Juniper Flow
J-Flow is Juniper Networks proprietary flow monitoring implementation. It is virtually identical with Cisco NetFlow, in that it records all network activity from the enabled ports on routers and switches. Juniper Routers and Switches have been designed around a flow-based architecture. So by default, these devices inspect the network and transport layer attributes of incoming packets and create flow sessions for a set of seven IP attribute values like CISCO NetFlow above.
Like flow-based architecture, Juniper network devices creates a J-flow table (J-Flow cache) for a set of network and transport layer attributes, and can vary with different J-Flow versions. A flow record is created in the J-Flow table when the first packet of a flow is processed. It is then maintained within cache while the flow is active. Each flow record in the table contains key fields that can be used later for exporting data to a collection device. As a flow record becomes active in a device, all packets with similar characteristics are tracked and counted, and certain fields in the flow record are updated. The flow record or J-Flow table information is then exported to a flow collector server periodically, based on flow timers. The collector contains a history of flow information exported by different devices.
This means that Juniper Flow and Cisco’s NetFlow are essentially identical and both perform Flow and Packet sampling analysis and allow a deeper view into the network’s performance and status.
SFLOW or Sampled Flow
SFlow is a completely different flow technology. This was developed by InMon Inc. and has become an industry standard defined in RFC 3176. It is a multi-vendor sampling technology embedded within routers and switches. This provides an uninterrupted monitoring capability of application level traffic flows at wire speed on all interfaces at the same time. The Sflow monitoring system consist of an sFlow agent which is embedded in routers, switches or in a standalone probe, connected to a central sFlow Collector.
The sFlow agent is software that combines interface counters and flow samples into sFlow datagrams and sends these to a server called, “sFlow Collector” via UDP(port 6343). These sFlow datagrams contain the “sFlow” information for current version, its originating agent’s IP address, sequence number, and how many samples it contains. Packet sampling is typically performed by the switching/routing ASICs, providing wire-speed performance that is faster than Cisco’s NetFlow, originally developed using IP routing accelerate techniques. Furthermore, sFlow data contains complete packet header and switching/routing information with advanced reporting detail in layers 2 too 7 of the OSI Model.
The biggest benefit of sFlow comes from its infinite scalability in large networks under heavy loads. However, this innovative statistical approach comes at a slight disadvantage in accuracy, granularity, and timing precision because it samples once every n-th packet from a traffic stream. sFlow also lacks accurate time stamping since sampled packets are immediately forwarded as they are picked up from the data stream.
Although “sFlow” seems poised to be an industry standard, only some network device companies have integrated it into their products such as Alcatel, Allied Telesis, Netgear, Foundry Networks, Hewlett-Packard (HP), Hitachi and NEC.
Overall NetFlow and SFlow technologies are extremely valuable and can provide amazingly detailed data and statistics. As new network devices are made, and NetFlow technology becomes easier to use and deploy, more companies will leverage this data to improve, fix, and optimize network of all shapes and sizes.