WO2024248658A1 - System and method for analysing an incoming stream of traffic - Google Patents

Abstract

The invention relates to computing systems and to the field of providing information security by means of the preliminary analysis of an incoming stream of traffic to determine whether the traffic belongs to a pre-established connection. The technical result is that of broadening the existing range of technical means and systems for protecting the information security of a computing network using an effective system for analysing a stream of traffic to determine whether the traffic belongs to a predetermined connection, said result being achieved by a system for analysing a stream of traffic to determine whether the traffic belongs to a predetermined connection, parsing packets to obtain data from the packets on a channel level, a network level, a transport level and an application level, splitting the data obtained from the packets into clusters, making a decision on each of the packets in said clusters using set rules for network packet processing, detecting deviations of network traffic that is compliant with said processing rules from a normal packet traffic profile, and refining the set rules for network packet processing.

Description

System and method for analyzing incoming traffic flow

AREA OF TECHNOLOGY

The invention relates to computing technology and the field of ensuring information security through preliminary analysis of the incoming traffic flow regarding the traffic's belonging to a previously established connection.

LEVEL OF TECHNOLOGY

The prior art includes means of protecting the perimeter of a local (corporate) computer network, known as a firewall, network screen, firewall, firewall - a complex of hardware or software that monitors and filters network packets passing through it in accordance with specified rules.

An analogue of the proposed solution is the "System for protecting computer networks from unauthorized access", disclosed in patent RU2607997, IPC G06F 15/163, published on 11.01.2017, wherein the system for protecting computer networks from unauthorized access is a firewall filter, connected between two computer networks in such a way that all information exchange between said networks is limited using filtering rules, wherein the firewall filter contains at least two network interfaces for exchanging data between clients of the first computer network and the second computer network of the two above-mentioned computer networks, wherein it additionally contains a traffic processing unit including a control device that provides for the input of traffic filtering rules and the storage of information about the filtering rules, a traffic analysis device that provides for the verification of the compliance of incoming information with the filtering rules, as well as a switching device through which the said network interfaces are connected to each other and that provides for the passage of information permitted by the filtering rules between the network interfaces and the blocking of information not permitted by the filtering rules, wherein the filtering rules prohibit the transit transmission of any packets between the said network interfaces except for those that have permitted features and addressing parameters in their headers, the form of the information part of the packet corresponding to a template stored in the memory of the firewall filter, as well as the parameters of the request or response corresponding to a set of permitted values stored in the memory of the firewall filter.

Known is the "Computer Network with a Firewall and a Firewall", disclosed in patent RU2214623, IPC G06F 15/163, G06F 15/173, published on 20.10.2003, which relates to the field of information security and, in particular, concerns hardware and software components of firewalls used to prevent unauthorized access and information exchange between different subscribers of computer networks. The technical result consists in increasing information security due to the non-use or complete concealment of network interfaces of the security device. The firewall for local area networks contains at least two network interfaces for packet switching of data between segments of the computer network, performed in accordance with the program packet filtering. After processing a packet in accordance with the filtering rules, the firewall retains unchanged the information about the physical and logical addresses of the sender of each packet, contained in their headers. The control program does not assign logical addresses to network interfaces and does not transmit information about their physical addresses to the network segments associated with them. To ensure that changes are made to the filtering rules, the firewall contains a special control interface, and any changes to the filtering parameters can be made exclusively through the control interface.

The main disadvantages of known analogues are that in order to protect network nodes from penetration through vulnerabilities or to protect against users downloading malicious programs, including viruses, as well as to protect against internal threats and data leaks, analogues do not provide for the use of a method for monitoring established connections and analyzing the incoming traffic flow to determine whether it belongs to a previously established connection.

DISCLOSURE OF INVENTION

The technical result of the invention consists in expanding the arsenal of technical means and systems for ensuring information security of a computer network by using an effective system for analyzing the traffic flow regarding the belonging of the traffic to a previously established connection.

The specified technical result is achieved by a system of traffic flow analysis on the traffic belonging to a previously established connection, packet parsing to obtain data from packets at the channel, network, transport and application levels, and splitting received data from packets to clusters, making a decision on each packet in clusters based on established rules for processing network packets, identifying deviations in network traffic corresponding to the processing rules from the normal packet traffic profile, and refining the established rules for processing network packets.

In a preferred embodiment, the traffic flow analysis system comprises a network traffic reception and preliminary analysis unit, a new device detection unit, a deep packet inspection (DPI) device, a signature unit, and a memory unit, wherein the reception and preliminary analysis unit is connected to the deep packet inspection (DPI) device and is configured to perform preliminary analysis of the incoming traffic flow. If the deep packet inspection (DPI) device determines that the traffic belongs to an already established connection, i.e. such packets with synchronization flags (Synchronize sequence numbers, SYN) and acknowledgement field is significant (ACK) have already passed through the control unit, then the packet can be passed without filtering.

In one embodiment, the system includes a graphical user interface to which the results of the work are sent for review by the system administrator.

If the packet is related to a new connection, it is sent to the DPI device for packet parsing. The DPI device parses packets with the Internet protocol (IP) specification to obtain data at the channel, network, transport, and application layers.

In one embodiment, the DPI device is capable of extracting known application protocols in use, tunneled IP addresses, usernames, file names, HTTP links and request URLs, server return codes.

In one embodiment of the traffic flow analysis system, the network traffic receiving and preliminary analysis unit is designed with the ability to buffer data for temporary storage.

Examples of data extracted by the DPI device include: obtaining physical addresses of hosts (MAC address), network protocol type (IPv4 and IPv6), IP addresses of source and destination hosts, transport protocol type (IP, ICMP, RIP, DDP, ARP, etc.), source and destination port numbers, as well as LLC, service flag values, and TCP "window" values.

Examples of application layer data include remote access and resource sharing protocols, HTTP, SMTP, FTP, HTTPS, TELNET, SSH, DNS and other well-known protocols.

The DPI device has a built-in signature database with characteristics for matching analyzed traffic packets, which allows you to accurately identify the application or protocol being analyzed.

To supplement the feature signature database with new applications and protocols, the feature signature database is designed to be updatable and refined.

The use of a system and method with preliminary analysis of the incoming traffic flow allows for a significant reduction in the time required to check data packets, especially in those cases where this is most important.

For example, a delay of more than 150 milliseconds is unacceptable for two-way voice conversations and may cause a connection break. An acceptable time interval for no traffic can also be set, by default it is usually 25 seconds. If the time is exceeded, this traffic will be related to a new connection.

In addition, each traffic type and/or protocol can be configured to have its own acceptable time interval for no traffic, at the user's discretion. In this way, application-level data and associated metadata for each network packet can be obtained.

BRIEF DESCRIPTION OF DRAWINGS

Fig. 1/3 shows a block diagram of a traffic processing system using a firewall system in accordance with the present invention.

Fig. 2/3 shows a block diagram of a method for processing traffic using a firewall system in accordance with the present invention and information flows between system blocks.

Fig. 3/3 shows the architecture of a firewall system according to the present invention.

IMPLEMENTATION OF THE INVENTION

The invention is a complex of hardware and software tools that ensure the implementation of a traffic flow analysis system, which, in accordance with Fig. 1/3, contains the following blocks:

• 11 - block for receiving and preliminary analysis of network traffic;

• 12 - deep packet inspection device (DPI);

• 13 - data clustering block;

• 14 - memory block; • 15 - registration log block;

• 16 - graphical user interface, contained in one of the implementation options of the system.

The network traffic reception and preliminary analysis unit 11 is connected to the DPI device 12 and is designed with the possibility of preliminary analysis of the incoming traffic flow. If it is established that the traffic belongs to an already established connection, i.e. such packets with the SYN and ACK flags have already passed through the network traffic reception and preliminary analysis unit 11, then the packet can be passed without filtering.

The 12 DPI device parses packets with the IP protocol specification to obtain data at the channel, network, transport and application levels. The 12 DPI device is designed with the ability to extract from the traffic the application protocols used, tunneled IP addresses, user names, file names, HTTP links and request URLs, and server return codes.

Examples of data extracted from the traffic flow include: obtaining physical addresses of hosts (MAC address), network protocol type (IPv4 and IPv6), IP addresses of source and destination hosts, transport protocol type (IP, ICMP, RIP, DDP, ARP, etc.), source and destination port numbers, as well as LLC, service flag values, TCP "window" values. Examples of application-level data include remote access and resource sharing protocols such as HTTP, SMTP, FTP, HTTPS, TELNET, SSH, DNS, etc.

In this way, application layer data and associated metadata of each network packet can be obtained. The 12 DPI device has its own unique characteristics, which are recorded in the built-in signature database. Matching a sample from databases with analyzed traffic allow to accurately identify the application or protocol. But since new applications and protocols appear periodically, the signature database also needs to be updated to ensure high identification accuracy.

Block 13 of data clustering serves to divide the data packages into groups (clusters) of similar objects to simplify further data processing and decision-making. In this case, each group of data can be applied to its own analysis method.

The functioning of the firewall system is determined by a set of rules, such as Suricata (Suricata is an open source intrusion detection system (IDS) and intrusion prevention system (IPS) developed by the Open Information Security Foundation (OISF)), which can be stored in the memory block 14. For each cluster, its own processing rules are created and installed, different from other clusters. In this way, chains are formed, into which clusters are redirected for subsequent processing.

To operate the system, it is sufficient to create one general rule for a specific cluster, and then redirect clusters from it to separate chains. Clusters can also be redirected from one own chain to another. This allows for optimization of network traffic processing, since each packet from the cluster is processed according to its own rules. In addition, information on any packet is simultaneously sent to block 15 of the registration log. Also, the traffic flow analysis system is configured so that packets are not discarded, but sent to block 13 for subsequent re-clustering if any discrepancies with the rules are detected. Based on temporary storage of files in the buffer (not shown in the figure), block 11 for receiving and preliminary analysis of network traffic checks that such a packet has already passed through, and marks it so that when re-clustering, the packet is placed in another cluster during sorting, and only after re-processing is it discarded. Periodically, the established rules for processing a network packet are refined depending on the passing traffic and the clusters that are formed.

In a preferred embodiment of the invention, the firewall system comprises a graphical user interface 16. In this case, it is possible not to discard packets, but to send them to the graphical user interface 16, where they will be checked by the system administrator. The system administrator can take all the information about the network traffic directly from the registration log block 15, he can also skip the packet manually and send it to the network traffic reception and preliminary analysis block 11, and then enter clarification about the packet into the registration log block 15.

Log block 15 collects data on each packet (including discarded ones) that comes from the 12 DPI device.

In a preferred embodiment of the invention, the data for analysis is metadata. Storing metadata without actual content is approximately 1/100th the compression ratio compared to raw network traffic and significantly increases the amount of information stored about packets.

Another advantage of the proposed system is that information about the identified atypical packets in block 13 clustering can be immediately sent to block 15 of the registration log.

In one use case, the traffic flow analysis system is used as a method and/or intrusion detection system (IDS). A method for analyzing a traffic flow implemented using a network traffic reception and preliminary analysis unit, in which the incoming traffic is received and analyzed, which is connected to a new device detection unit, a DPI deep packet inspection device, a signature unit, and a memory unit, characterized in that the reception and preliminary analysis unit is connected to the DPI deep packet inspection device, a memory unit, and a new device detection unit and a signature unit connected to it, wherein the traffic reception and analysis unit is connected to the new device detection unit.

The method for detecting an intrusion in accordance with Fig. 2/3 is implemented in a hardware and software complex, which is a traffic flow analysis system, which includes the following blocks:

•21 - block for receiving and parsing network traffic;

•22 - block for detecting new devices;

•23 - signature block;

•24 - memory block.

Memory block 24 is designed to store various information and sets of rules in accordance with the information security policy and is connected to the new device detection block and the signature block.

The network traffic reception and parsing unit 21 serves to collect network traffic at the channel level, i.e. traffic coming from the global Internet or traffic that passes within the enterprise network, for example, via a local network, and for its further parsing in accordance with the structure of the IP/TCP stack packet. To parse the packet, the means for parsing packets at the network and transport, or at the network, transport and application levels can be used. First, the incoming traffic is divided into TCP, UDP or other transport streams, after which parsers mark them and break them down into high-level protocols and their fields, normalizing them if necessary. The resulting decoded, unzipped, and normalized protocol fields are then sequentially checked by blocks 22 and 23 for the presence of network attacks or malicious activity in the network traffic.

The new device detection unit 22 serves to identify and control network devices between which data exchange and transmission is performed in the network. A network device can be identified in the network by its network address, for example, an IP address, MAC address, URI (Uniform Resource Identifier) or any other network identifier. Thus, network traffic is transmitted only between identified devices. The decision to pass or block outgoing or incoming traffic is made by the new device detection unit 22 based on the presence of the network device identifier in the memory unit 24. If the identifier characterizing the network device is contained in the memory unit 24, then the traffic is passed, otherwise, the traffic is blocked.

To detect network attacks, the intrusion detection system contains a signature block 23, which, when checking a packet, accesses a memory block 24 with signatures of known attacks.

A signature is understood as a continuous finite sequence of bytes, necessary and sufficient for unambiguous identification of threats. In the event that any code of the program being viewed matches a known virus code in memory block 24, a decision is made to block the packet. In the preferred embodiment, the contents of the packages being examined are compared not directly, but by their checksums, which allows for a significant reduction in the number of stored records of known viruses. A specialist in this field of technology should be It is clear that in order to increase the effectiveness of threats, the database of known viruses must be constantly updated, since new malicious codes and threats appear every day.

The network attack signature is almost identical to the virus signature and is a set of features that allow distinguishing a network attack from other types of network traffic. For example, the TCP packet flags SYN and FIN may be set at the same time. This combination of flags is used in many attack programs to bypass filters and monitors that only check for the setting of a single SYN flag. The presence of the string "GET.cgi-bin./etc/passwd" in the data area of the HTTP packet indicates the exploitation of a PathTraversal vulnerability. Finally, the destination port 139 and the OOB (Out of Band) flag may be set in the TCP packet header, which is a sign of a WinNuke attack.

In accordance with Fig. 3/3, additional functions can also be implemented in the traffic flow analysis system in accordance with the present invention:

•30 - packet filtering;

•31 - routing;

•32 - intrusion detection function (ISO or IDS according to industrial protocols);

•33 - creating a failover cluster;

•34 - streaming antivirus;

•35 - management and configuration of network interfaces;

•36 - virtual private network (VPN);

•37 - integration with Active Directory;

•38 - Domain Name System (DNS); •39 - web server function.

Claims

CLAUSE OF INVENTION 1. A traffic flow analysis system comprises a network traffic reception and preliminary analysis unit, a new device detection unit, a DPI deep packet inspection device, a signature unit and a memory unit, characterized in that the reception and preliminary analysis unit is connected to the DPI deep packet inspection device and is designed with the ability to preliminary analyze the incoming traffic flow, wherein if the DPI deep packet inspection device determines that the traffic belongs to an already established connection, i.e. such packets with SYN synchronization and ACK confirmation flags have already passed through the control unit, then the packet can be passed without filtering. 2. The system according to claim 1, in which a graphical user interface is additionally implemented. 3. The system according to claim 1, wherein if the packet relates to a new connection, it is sent to a DPI device for parsing packets, which parses packets with an Internet protocol specification to obtain data at the channel, network, transport and application levels. 4. The system according to claim 1, wherein the DPI device is capable of extracting known application protocols used, tunneled IP addresses, user names, file names, HTTP links and request URLs, server return codes. 5. The system according to claim 1, in which the traffic flow analysis systems, the network traffic reception unit and preliminary analysis unit are designed with the ability to buffer data for their temporary storage. 6. A method for analyzing a traffic flow implemented using a network traffic reception and preliminary analysis unit, in which incoming traffic is received and analyzed, which is connected to a new device detection unit that identifies and controls network devices that exchange and transmit data in the network, a DPI deep packet inspection device, with the help of which samples from the database are compared with the analyzed traffic and an application or protocol is determined, which then sequentially check the received decoded, unzipped and normalized protocol fields using a signature unit and a memory unit for the presence of network attacks or malicious activity in the network traffic, characterized in that the reception and preliminary analysis unit is connected to the DPI deep packet inspection device and is configured to preliminary analyze the incoming traffic flow, wherein if the DPI deep packet inspection device determines that the traffic belongs to an already established connection, i.e. such packets with SYN synchronization flags and ACK confirmation flags have already passed through the control unit, then the packet can be passed without filtering. 7. The method according to claim 6, in which the results of the work are sent to the user's graphical interface. 8. The method according to claim 6, wherein if the packet relates to a new connection, it is sent to a DPI device for parsing packets, which parses packets with an Internet protocol specification to obtain data at the channel, network, transport and application levels. 9. The method according to claim 6, wherein the DPI device extracts known used application protocols tunneled by IP- addresses, usernames, file names, HTTP links and request URLs, server return codes. 10. The method according to claim 6, in which the traffic flow analysis systems, the network traffic reception and preliminary analysis unit, buffer data for their temporary storage.