NetStream Packets
Currently, NetStream exports flow information in UDP packets in Version 5 (V5), Version 8 (V8), or Version 9 (V9) format. Later versions are in the trial phase and have not been put into commercial use. Each data packet contains a header and one or more flow records.
NetStream exported packets of original flows can be in either V5 or V9 format. NetStream exported packets of aggregated flows can be in either V8 or V9 format.
Differing from earlier versions, the V9 format is based on templates, which makes the traffic statistics to be exported more flexibly. In addition, it is earlier to add new fields and generate new records. The V9 format is incompatible with the V5 and V8 formats.
Figure 1 shows the structure of a NetStream packet.
Header of a NetStream V5 Packet
Table 1 describes fields in the header of a NetStream V5 packet.
Field |
Description |
|---|---|
version |
Version number of the format in which the NetStream packet is exported. For V5, the value is 0x05. |
count |
Number of flow records in the current NetStream packet. The value is in the range from 1 to 30. |
SysUptime |
Period from when the time the system is booted to the time when the NetStream packet is generated, in milliseconds. |
unix_secs |
Number of seconds from 00:00:00 on January 1, 1970 to the time when the NetStream packet is generated. |
unix_nsecs |
Number of nanoseconds less than one second in the packet generation period. |
flow_sequence |
Sequence numbers of the exported flow records: In the first NetStream packet, the value of this field is 0 and the value of count is c1. In the second NetStream packet, the value of this field is c1 and the value of count is c2 (count = c2). In the third NetStream packet, the value of this field is c2 + c1. ... In the (n - 1)th NetStream packet, the value of this field is fs(n - 1) and the value of count is c(n - 1). In the nth NetStream packet, the value of this field is fs(n - 1) + c(n - 1). This field helps identify whether any NetStream packet is lost. |
engine_type |
Type of the flow switching engine, which is set to the device type. |
engine_id |
Slot ID where the switching engine resides. |
Information Carried in a NetStream V5 Packet
Figure 3 shows the information carried in a NetStream V5 packet.
The system constructs two types of UDP packets according to the statistics about the incoming and outgoing traffic. These two types of UDP packets are both in V5 format.
Header of a NetStream V8 Packet
Table 2 describes fields in the header of a NetStream V8 packet.
Field |
Description |
|---|---|
version |
Version number of the format in which the NetStream packet is exported. For V8, the value is 0x08. |
count |
Number of flow records in the current NetStream packet, instead of the total number of flows. |
SysUptime |
Period from the time when the system is booted to the time when the NetStream packet is generated, in milliseconds. |
unix_secs |
Number of seconds from 00:00:00 on January 1, 1970 to the time when the NetStream packet is generated. |
unix_nsecs |
Number of nanoseconds less than one second in the packet generation period. |
flow sequence |
Sequence numbers of the exported flow records: In the first NetStream packet, the value of this field is 0 and the value of count is c1. In the second NetStream packet, the value of this field is c1 and the value of count is c2. In the third NetStream packet, the value of this field is c2 + c1. ... In the (n - 1)th NetStream packet, the value of this field is fs(n - 1) and the value of count is c(n - 1). In the nth NetStream packet, the value of this field is fs(n - 1) + c(n - 1). This field helps identify whether any NetStream packet is lost. |
engine type |
Type of the flow switching engine, which is set to the device type. |
engine id |
Slot ID where the switching engine resides. |
aggregation |
Aggregation mode, which can be:
|
aggregation version |
Version number of the format in which the aggregated NetStream packet is exported. The value is 0x02. |
sampling Interval |
Sampling interval. The value is configured in the system view. |
reserved |
Reserved field. The value is all 0s. |
Header of a NetStream V9 Packet
Table 3 describes fields in the header of a NetStream V9 packet.
Field |
Description |
|---|---|
version |
Version number of the format in which the NetStream packet is exported. For V9, the value is 0x09. |
count |
Number of FlowSet records (including records in the template FlowSet and data FlowSet) in the NetStream packet. |
system up time |
Period from the time when the system is booted to the time when the NetStream packet is generated, in milliseconds. |
unix_secs |
Number of seconds from 00:00:00 on January 1, 1970 to the time when the NetStream packet is generated. |
package sequence |
Cumulative sequence number of a NetStream packet. This field helps identify whether any NetStream packet is lost. |
source id |
Source ID. The value is 4 bytes and is used to guarantee the uniqueness for all flows exported from a device. (This field is similar to the engine type and engine id fields in the header of a NetStream V5 or V8 packet). The value can be user-defined. The first two bytes are reserved for future extension and are currently set to all 0s. The third byte uniquely identifies a device that exports NetStream packets, which is set to the device type. The fourth byte specifies the ID of slot where the NetStream-enabled card resides. |
Concepts Related to the V9 Format
Template FlowSet
Describes the flow information in exported NetStream packets. The template is encapsulated into NetStream packets by a NetStream-enabled device and is sent to the NSC to reach an agreement on flow information parsing between the device and the NSC. Being the core of the V9 format, a template FlowSet consists of multiple template records. Based on the received template, the NSC can parse the flow information carried in the exported NetStream packets, without the need of pre-defining a parsing format. This greatly increases the flexibility and scalability of the NetStream records, and facilitates the development of third party software and the extension of the NetStream function.
Template record
Corresponds to a data record in an exported NetStream packet. The flow information in data records is parsed according to the corresponding template record.
Template ID
Uniquely identifies a template. A data record contains the template ID used to select a template.
Data FlowSet
Indicates a combination of one or more data records.
Data record
Corresponds to one NetStream record.
NetStream Packet Exported in V9 Format
As shown in Figure 6, a NetStream packet exported in V9 format consists of the packet header, template FlowSet, and data FlowSet.
Template FlowSets and data FlowSets are independent of each other. The data records in a data FlowSet are parsed by the NSC according to a known template. That is, the NSC has already known the template corresponding to the template ID in a data record. The template FlowSet notifies the NSC of a template to be used for parsing subsequent exported NetStream packets.
Template FlowSets and data FlowSets are not necessarily contained in an exported NetStream packet. The possible combinations are as follows:
- An exported NetStream packet contains template FlowSets and data FlowSets. For the NSC, the template FlowSets and data FlowSets are independent of each other. The NSC saves the template in a received template FlowSet to parse the data records in subsequent data FlowSets.
- An exported NetStream packet contains only data FlowSets. If a template ID is pre-defined, the NetStream-enabled device exports NetStream packets carrying only data FlowSets to the NSC.
- An exported NetStream packet contains only template FlowSets. Generally, template FlowSets and data FlowSets are encapsulated into one exported NetStream packet to better use the network bandwidth. NetStream packets containing only template FlowSets are sent only in a few scenarios. When the device is already configured with templates and is restarted, the device sends NetStream packets containing only template FlowSets to the NSC so that it can send all templates to the NSC quickly. In addition, the templates have a validity period. When the validity period of a template elapses, the NSC deletes the template. In this case, the device needs to regularly transmit template FlowSets to the NSC. If no data FlowSet is generated during the transmission, the device sends NetStream packets containing only template FlowSets to the NSC.
Template FlowSet Format
Figure 7 shows the template FlowSet format.
In this example, the template FlowSet contains two template records. The following table describes the meaning of each field.
Field |
Description |
|---|---|
FlowSet ID |
Distinguishes template records from data records. The FlowSet ID of a template FlowSet is in the range from 0 to 255, and the FlowSet ID of a data FlowSet starts from 256. In this case, the NSC can identify the template FlowSets in exported NetStream packets. |
Length |
Indicates the total length of a FlowSet. An individual template FlowSet may contain multiple template IDs. The length value can be used to determine the position of the next FlowSet record, which could be either a template or a data FlowSet. |
Template ID |
Uniquely identifies a template on a device. When a device generates different template FlowSets to match the types of NetStream data to be exported, each template is assigned a unique ID. The templates that define the data record formats are numbered from 256, because the IDs from 0 to 255 are used as FlowSet IDs. |
Field Count |
Indicates the number of fields in this template record. A template FlowSet may include multiple template records. This field helps determine the end point of the current template record and the start point of the next one. |
Field Type |
Indicates the field type. The value can be defined by users. For example, when statistics are collected based on the destination IP address, protocol type, ToS, and MPLS label, each type of the obtained information is defined by a field type. |
Field Length |
Indicates the length of the field defined previously, in bytes. For the destination IP address, the value is 4, representing 4 bytes. |
Data FlowSet Format
Figure 8 shows the data FlowSet format.
In this example, the data FlowSet contains two data records. The data FlowSet ID is the ID of the template used to parse the two data records. The following table describes the fields in the data FlowSet.
Field |
Description |
|---|---|
flowset ID = template ID |
The FlowSet ID corresponds to the previously described template ID. |
length |
Indicates the length of the data FlowSet. |
record n -field n |
Indicates a collection of field values in the data FlowSet. |
Padding |
The padding field is 32 bits long and is inserted to the end of the FlowSet. Note that the length field includes those padding bits. |
Relationship Between the Flow Data and the V9 Template Format
Figure 9 shows the relationship between the flow data and the V9 template format.
