WO2025190164A1 - Mapping relationship acquiring method and apparatus - Google Patents

Abstract

Provided in the embodiments of the present application are a mapping relationship acquiring method and an apparatus. The method comprises: on the basis of target data, acquiring an attribute of the port number of a first port, the target data comprising a port number field and a background field, the port number field comprising the port number of the first port, and the background field comprising the attribute of the port number; performing feature extraction on the attribute of the port number so as to obtain a feature tag set of the port number, the feature tag set comprising one or more feature tags of the port number; and, on the basis of the port number and the feature tag set, acquiring a mapping relationship, the mapping relationship comprising a key-value pair, the key in the mapping relationship comprising the port number, and the value in the mapping relationship comprising the feature tag set.

Description

Method and device for obtaining mapping relationship

This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 11, 2024, with application number 202410288365.2 and invention name “Method and Device for Obtaining Mapping Relationship”, the entire contents of which are incorporated by reference into this application.

Technical Field

The present application relates to the field of computers, and in particular to a method and device for obtaining a mapping relationship.

Background Art

A port can be understood as the outlet for a communication device to communicate with the outside world. Ports include both virtual ports and physical ports. A communication device can have multiple ports, each of which can be associated with a specific application or service within the device. A port number is data that uniquely identifies a port. In machine learning systems targeting data communication data, port numbers are extremely important features. By converting port numbers into feature labels, these feature labels can be used as input for machine learning algorithms to perform tasks such as application identification and anomaly detection.

To obtain the feature label of the port number, the port number itself is usually directly encoded. For example, when using one-hot encoding, if there are d different port numbers in all samples, a d-dimensional binary vector is used as the feature label of the port number.

However, the feature tag obtained by directly encoding the port number itself cannot represent the properties of the port number, causing the feature tag of the port number to lose the information of the port number. In addition, the feature tags of different port numbers cannot represent the correlation between different ports.

Summary of the Invention

This application provides a method and device for obtaining a mapping relationship, which can avoid information loss. The technical solution is as follows.

In a first aspect, a method for obtaining a mapping relationship is provided, the method comprising: obtaining the attributes of the port number of a first port based on target data, the target data comprising a port number field and a background field, the port number field comprising the port number of the first port, and the background field comprising the attributes of the port number; performing feature extraction on the attributes of the port number to obtain a feature tag set of the port number, the feature tag set comprising one or more feature tags of the port number; obtaining a mapping relationship based on the port number and the feature tag set, the mapping relationship comprising a key-value pair, the key in the mapping relationship comprising the port number, and the value in the mapping relationship comprising the feature tag set.

In the method provided in the first aspect, due to the co-occurrence relationship between the port number and the background field, the port number itself is not directly encoded, but the attributes of the port number are obtained from the background field related to the port number, and the feature tag set of the port number (or the encoding of the port number) is obtained based on the attributes of the port number, so that the feature tag set of the port number can represent the attributes of the port number (or the semantics or purpose of the port number), thereby avoiding the information loss caused by the inability of the feature tag set of the port number to represent the attributes of the port number. In addition, if two port numbers have common attributes, there will be the same part in the feature tag sets of the two port numbers. In other words, the encodings of the two port numbers are partially the same or similar. Therefore, the similarity between the feature tag sets of the two port numbers can represent the closeness of the association relationship between the two port numbers, thereby more fully representing the correlation between different ports.

In some embodiments, the target data includes static data, which refers to data that is unrelated to the actual connection usage of the port. The attributes of the port number include usage information of the port number, and the usage information includes at least one of an identifier of an application communicated through the port number, an identifier of a protocol communicated through the port number, or an identifier of a service communicated through the port number. The feature extraction of the attributes of the port number to obtain a feature tag set of the port number includes: feature extraction of the usage information of the port number to obtain a static tag set of the port number, and the static tag set of the port number includes one or more static tags of the port number.

By using the usage information of the port number as the static label of the port number, since the number of applications related to a port number is usually small, for example, in most cases a port number corresponds to only one application, and in a few cases a port number corresponds to multiple applications, but even when a port number corresponds to multiple applications, the number of applications corresponding to a port number is basically only in the single digit. Therefore, by using the usage information corresponding to the port number as the static label set of the port number, the size of the static label set of the port number or the dimension of the feature vector does not exceed 10, which is much smaller than the size of tens of thousands that the static label set of the port number will reach when using the one-hot encoding method or label encoding method, thereby avoiding the dimensionality disaster.

In some implementations, the static data includes a port registry, and the port registry includes at least one of an IANA port registry and/or a private protocol port registry.

Since the static label of the port number can be obtained by using the port registry, the calculation overhead is relatively small and there is no need to update the port registry, thereby reducing the overhead generated by additional updating of the port registry.

In some implementations, before acquiring the attribute of the port number of the first port based on the target data, the method further includes: performing redundancy removal processing on an original port registry to obtain the port registry.

By performing redundancy processing on the original port registry, the data volume of the port registry is reduced, thereby reducing the amount of data required to be processed when extracting feature tags from the port registry, thereby improving the efficiency of extracting feature tags.

In some implementations, performing redundancy removal on the original port registry includes: if it is detected that a background field in the original port registry includes a keyword indicating duplication, deleting the row containing the background field from the original port registry.

Since the presence of a keyword representing duplication in a row of the port registry usually indicates that the row is redundant with a row that appeared earlier, the row containing the keyword representing duplication is deleted from the original port registry, thereby avoiding the unnecessary increase in the amount of processed data caused by extracting feature labels from multiple repeated rows of content, and further improving the efficiency of extracting feature labels.

In some embodiments, the de-redundancy processing of the original port registry includes: if it is detected that there are terms with the same meaning but different expressions in multiple background fields in the original port registry, determining the target words for the terms, and replacing the terms appearing in the multiple background fields with the target words.

By unifying the expressions of terms with the same meaning but different expressions in the port registry, it is possible to avoid unnecessary increase in the amount of processed data by extracting feature labels from multiple lines of content with inconsistent expressions but the same meaning, thereby further improving the efficiency of extracting feature labels.

In some embodiments, the target data includes dynamic data, and the dynamic data refers to data related to the actual connection usage of the first port. The attributes of the port number include an object identifier, and the object identifier is used to identify an object that has an interactive relationship with the first port. The feature extraction of the attributes of the port number to obtain a feature tag set of the port number includes: feature extraction of the object identifier to obtain a dynamic tag set of the port number, and the dynamic tag set of the port number includes one or more dynamic tags of the port number.

Since the dynamic label of the port number is obtained by identifying the object that has an interactive relationship with the port number, the feature label sets of different ports that interact with the same object will have similarities, so that the port number label can more fully characterize the association relationship between different port numbers, and the port number label contains richer information. In addition, the number of objects that have an interactive relationship with a port number is usually limited. Even if there are a large number of objects that have an interactive relationship with a port number, the dimension of the dynamic label set can be reduced by retaining objects that appear more frequently than a predetermined frequency from the objects that have an interactive relationship with the port number. This prevents the size of the dynamic label set from reaching tens of thousands as in the one-hot encoding or label encoding method, thus avoiding the dimensionality disaster.

In some embodiments, the dynamic data includes log records of a first communication device, wherein the log records are used to record data generated by communication between the first port of the first communication device and the second communication device, and the object identifier includes at least one of the network address of the second communication device, the identifier of the area where the second communication device is located, the first port in the second communication device that communicates with the first port, and/or the identifier of an application in the second communication device that communicates with the first port.

The address of the remote device, the identifier of the area where the remote device is located, the interface of the remote device, the identifier of the area where the remote device is located, the port of the remote device or the identifier of the application program can all serve as feature tags of the port, making the data form of the feature tag more diversified, the application scenarios adapted are richer, and the flexibility is improved.

In some embodiments, the attributes of the port number also include the number of interactions between the object and the first port, and the feature extraction of the object identifier to obtain a dynamic tag set for the port number includes splicing or fusing the object identifier and the number of interactions to obtain a feature tag set for the port number.

Since the dynamic label of the port retains the number of interactions between the port and objects such as remote IP addresses and remote applications, the dynamic label of the port can provide richer background information. For example, the similarity of the feature labels of different ports that interact with the same object with similar frequencies will be similar, so that the similarity between the feature labels of two port numbers can more fully characterize the closeness of the association relationship between the two port numbers.

In some embodiments, the dynamic data also includes threat records, the background field in the threat record includes an attack type field, the attributes of the port number include the type of network attack targeting the first port and the number of times the first port is attacked by the type of network attack, and the feature extraction of the attributes of the port number to obtain a feature tag set of the port number includes: feature extraction of the network attack and/or the number of attacks to obtain a dynamic tag set of the port number, and the dynamic tag set of the port number includes one or more dynamic tags of the port number.

Through the above method, the feature tag sets of different ports attacked by the same type of network attack will have a higher similarity, and the feature tag sets of different ports attacked with similar frequency will have a higher similarity, so that the feature tag set of the port can more fully represent the semantics of the port.

In some embodiments, the feature extraction of the attributes of the port number to obtain a feature tag set of the port number includes: feature extraction of the attributes of the port number in static data to obtain a static tag set of the port number; feature extraction of the attributes of the port number in dynamic data to obtain a dynamic tag set of the port number; and fusion of the static tag set of the port number with the static tag set of the port number to obtain a feature tag set of the port number.

By extracting feature labels by fusing dynamic paths with static paths, not only is information loss and dimensionality disaster avoided, but because the feature label set includes two data sources, dynamic data and static data, the feature label extraction process takes into account both static and dynamic data, solving the problem of difficulty in balancing dynamic and static data, and enabling feature labels to more fully characterize the semantics of ports.

In some embodiments, obtaining the attributes of the port number of the first port based on the target data includes: determining the similarity between a background field in the target data and the port number; and extracting the attributes of the port number from the background field if the similarity is higher than a similarity threshold.

The above method is equivalent to verifying the correlation between the background field and the port number. The background field that is not closely related to the port number does not need to participate in the feature label extraction process. Only the background field that is closely related to the port number will participate in the feature label extraction process. This reduces the interference of the background field that is not closely related to the port number and reduces the number of dimensions of the feature label.

In some embodiments, acquiring the attribute of the port number of the first port based on the target data includes: determining a hash degree of a background field in the target data; and extracting the attribute of the port number from the background field if the hash degree is higher than a hash degree threshold.

The above method is equivalent to verifying the hash degree of the background field. Background fields with insufficient hash degree do not need to participate in the feature label extraction process. Background fields with sufficiently high hash degree will participate in the feature label extraction process, thereby improving the distinction between different port number feature labels and reducing the probability of duplication of different port number feature labels.

In some embodiments, the method further includes: comparing the feature tag set of the first port with the feature tag set of the second port to obtain the similarity between the feature tag set of the first port and the feature tag set of the second port; and determining the similarity between the first port and the second port based on the similarity between the feature tag set of the first port and the feature tag set of the second port.

The above method supports quantitative and relatively accurate evaluation of the similarity between different ports.

In some embodiments, the target data includes an alarm log set, the alarm log set includes a first alarm log and a second alarm log, and the method further includes: determining the similarity between the first alarm log and the second alarm log based on the similarity between the feature tag set of the first port and the feature tag set of the second port, the feature tag set of the first port is determined based on the background field in the first alarm log, and the feature tag set of the second port is determined based on the background field in the second alarm log; clustering the alarm log set based on the similarity between the first alarm log and the second alarm log.

Since the similarity between the feature tag sets based on the port is used to determine the similarity between different alarm logs where the port appears, the similarity between the alarm logs includes the similarity of the port dimension, and the similarity between the alarm logs is more accurate. Therefore, clustering the alarm logs based on the similarity of different alarm logs can more accurately determine the type of the alarm log.

In some embodiments, the target data includes network connection information of the first communication device in a normal state, and the method further includes: when it is detected that a second communication device establishes a connection with the first port of the first communication device, searching the mapping relationship based on the port number of the first port to obtain a feature tag set of the port number; if the feature of the second communication device hits the feature tag set of the port number, determining that the connection is a normal connection; if the feature of the second communication device does not hit the feature tag set of the port number, determining that the connection is an abnormal connection.

The mapping relationship between the port and the feature tag set determined by the network connection information under normal conditions can realize the role of the traffic baseline. If the characteristics of the second communication device belong to the feature tag set corresponding to the port, it indicates that the connection currently initiated by the second communication device to the port does not exceed the traffic baseline, and the second communication device is normally connected to the port of the first communication device in the historical time period; if the characteristics of the second communication device belong to the feature tag set corresponding to the port, it indicates that the connection currently initiated by the second communication device to the port deviates from the traffic baseline, then the connection between the second communication device and the first communication device is determined to be an abnormal connection, which helps to detect abnormal connections more accurately.

In some embodiments, the target data includes a training sample set, each sample in the training sample set includes a source address, a destination address, a source port number, a destination port number, a protocol type, and a type label, and the method further includes: searching the mapping relationship based on the source port number to obtain a feature label set of the source port number; searching the mapping relationship based on the destination port number to obtain a feature label set of the destination port number; splicing or fusing the feature label set of the source address, the feature label set of the destination address, the feature label set of the source port number, the feature label set of the destination port number, and the feature label set of the protocol type to obtain a feature label set of each sample in the training sample set; performing model training based on the feature label set of each sample in the training sample set and the type label of each sample in the training sample set to obtain a classification model.

In a second aspect, a device for obtaining a mapping relationship is provided, the device comprising:

an acquiring unit, configured to acquire an attribute of a port number of a first port based on target data, wherein the target data includes a port number field and a background field, the port number field includes the port number of the first port, and the background field includes the attribute of the port number;

a processing unit, configured to perform feature extraction on the attributes of the port number to obtain a feature tag set of the port number, wherein the feature tag set includes one or more feature tags of the port number;

The acquisition unit is further configured to acquire a mapping relationship based on the port number and the feature tag set, wherein the mapping relationship includes a key-value pair, the key in the mapping relationship includes the port number, and the value in the mapping relationship includes the feature tag set.

In some embodiments, the target data includes static data, which refers to data that is unrelated to the actual connection usage of the port. The attributes of the port number include usage information of the port number, and the usage information includes at least one of an identifier of an application communicated through the port number, an identifier of a protocol communicated through the port number, or an identifier of a service communicated through the port number. The processing unit is further used to perform feature extraction on the usage information of the port number to obtain a static tag set of the port number, and the static tag set of the port number includes one or more static tags of the port number.

In some implementations, the static data includes a port registry, and the port registry includes at least one of an IANA port registry and/or a private protocol port registry.

In some implementations, the processing unit is further configured to perform redundancy removal processing on the original port registry to obtain the port registry.

In some embodiments, the processing unit is further used to perform at least one of the following: if it is detected that a background field in the original port registry includes keywords representing repetition, deleting the row where the background field is located from the original port registry; if it is detected that there are terms with the same meaning but different expressions in multiple background fields in the original port registry, determining the target terms for the terms, and replacing the terms appearing in the multiple background fields with the target terms.

In some embodiments, the target data includes dynamic data, which refers to data related to the actual connection usage of the first port. The attributes of the port number include an object identifier, which is used to identify an object that has an interactive relationship with the first port. The processing unit is used to extract features from the object identifier to obtain a dynamic tag set for the port number, and the dynamic tag set for the port number includes one or more dynamic tags of the port number.

In some embodiments, the dynamic data includes log records of a first communication device, wherein the log records are used to record data generated by communication between the first port of the first communication device and the second communication device, and the object identifier includes at least one of the network address of the second communication device, the identifier of the area where the second communication device is located, the first port in the second communication device that communicates with the first port, and/or the identifier of an application in the second communication device that communicates with the first port.

In some implementations, the attributes of the port number further include the number of interactions between the object and the first port, and the processing unit is configured to concatenate or fuse the object identifier and the number of interactions to obtain a feature tag set for the port number.

In some embodiments, the dynamic data also includes a threat record, the background field in the threat record includes an attack type field, the attributes of the port number include the type of network attack targeting the first port and the number of times the first port has been attacked by the type of network attack, and the processing unit is used to perform feature extraction on the network attack and/or the number of attacks to obtain a dynamic label set for the port number, and the dynamic label set for the port number includes one or more dynamic labels for the port number.

In some embodiments, the processing unit is further used to perform feature extraction on the attributes of the port number in static data to obtain a static tag set of the port number; perform feature extraction on the attributes of the port number in dynamic data to obtain a dynamic tag set of the port number; and fuse the static tag set of the port number with the static tag set of the port number to obtain a feature tag set of the port number.

In some embodiments, the processing unit is further configured to determine a similarity between a background field in the target data and the port number; and if the similarity is higher than a similarity threshold, extracting the attribute of the port number from the background field.

In some implementations, the processing unit is further configured to determine a hash degree of a background field in the target data; and if the hash degree is higher than a hash degree threshold, extract the attribute of the port number from the background field.

In some embodiments, the device further comprises:

a comparing unit, configured to compare the feature tag set of the first port with the feature tag set of the second port to obtain a similarity between the feature tag set of the first port and the feature tag set of the second port;

The processing unit is further configured to determine a similarity between the first port and the second port based on a similarity between the feature tag set of the first port and the feature tag set of the second port.

In some embodiments, the target data includes an alarm log set, the alarm log set includes a first alarm log and a second alarm log, and the processing unit is further used to determine the similarity between the first alarm log and the second alarm log based on the similarity between the feature tag set of the first port and the feature tag set of the second port, the feature tag set of the first port is determined based on the background field in the first alarm log, and the feature tag set of the second port is determined based on the background field in the second alarm log; based on the similarity between the first alarm log and the second alarm log, the alarm log set is clustered.

In some embodiments, the target data includes network connection information of the first communication device in a normal state, and the processing unit is further used to, when detecting that a second communication device has established a connection with the first port of the first communication device, search the mapping relationship based on the port number of the first port to obtain a feature tag set of the port number; if the feature of the second communication device hits the feature tag set of the port number, determine that the connection is a normal connection; if the feature of the second communication device does not hit the feature tag set of the port number, determine that the connection is an abnormal connection.

In some embodiments, the target data includes a training sample set, each sample in the training sample set includes a source address, a destination address, a source port number, a destination port number, a protocol type, and a type label, and the apparatus further includes:

A searching unit, configured to search the mapping relationship based on the source port number to obtain a feature tag set of the source port number; and search the mapping relationship based on the destination port number to obtain a feature tag set of the destination port number;

The processing unit is also used to splice or fuse the feature label set of the source address, the feature label set of the destination address, the feature label set of the source port number, the feature label set of the destination port number, and the feature label set of the protocol type to obtain the feature label set of each sample in the training sample set; and perform model training based on the feature label set of each sample in the training sample set and the type label of each sample in the training sample set to obtain a classification model.

According to a third aspect, a communication device is provided, comprising: a processor, the processor being coupled to a memory, the memory storing at least one computer program instruction, the at least one computer program instruction being loaded and executed by the processor, so that the communication device implements the method described in the first aspect or any optional method of the first aspect.

In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and when the instruction is executed on a computer, the computer executes the method according to the first aspect or any optional method of the first aspect.

In a fifth aspect, a computer program product is provided, which includes one or more computer program instructions. When the computer program instructions are loaded and run by a computer, the computer executes the method described in the first aspect or any optional method of the first aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flowchart of a method for obtaining a mapping relationship provided in an embodiment of the present application;

FIG2 is a schematic structural diagram of a device for obtaining a mapping relationship provided in an embodiment of the present application;

FIG3 is a schematic structural diagram of a communication device provided in an embodiment of the present application.

DETAILED DESCRIPTION

In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

The following is an explanation of some terminology concepts involved in the embodiments of this application.

(1) Port number

A port number is used to identify an application or service port on a computer network. A port number is usually a 16-bit integer, ranging from 0 to 65535.

(2) Background field

Background fields refer to fields other than port numbers that are associated with port numbers. The content of the background field is dependent on the port number. For example, the background field includes an Internet Protocol (IP) address that interacts with the port, and the IP address needs to use the port to establish a network connection. For another example, the background field includes the identifier of the application that uses the port and the identifier of the business that uses the port. The background field is used to extract the feature label of the port number. For example, when a dynamic path is used, the interactive relationship (co-occurrence relationship) between the background field and the port number can be used to extract the feature label of the port number. The background field can come from static data or dynamic data. For example, the background field in static data includes fields related to the purpose of the port number, such as an application identification field or a business identification field. The background field in dynamic data includes an IP address field that interacts with the port or an attack type field corresponding to the port.

(3) Feature tag set of port number

The feature tag set of a port number includes one or more feature tags of the port number. The feature tag is used to identify the characteristics of the port number. In some embodiments, the feature tag is used to identify the purpose of the port number. For example, the feature tag is an identifier of the application corresponding to the port number. For another example, the feature tag can be in the form of a string or a number. The feature tag can be a static tag or a dynamic tag. In some embodiments, the hash degree of the feature tag is higher than the hash degree threshold.

(4) Static data

Static data, also known as static knowledge, refers to knowledge that is unrelated to the actual connection and usage of a port. In the embodiments of this application, static data primarily serves as a data source for feature tags. Static data includes the port number and its attributes. For example, static data includes a port number field and a background field, where the port number field is the key and the background field is the value, with the background field including the attributes of the port number.

The attributes of a port number can also be called its descriptive information, semantic information, usage information, or port category information. For example, the attributes of a port number include at least one of the usage of the port number, the application name bound to the port number, the service name bound to the port number, and the protocol type bound to the port number.

In some embodiments, the static data is a port registry. The port registry is used to record the attributes of the port. The port registry includes descriptive information of multiple dimensions of the port number. For example, the key of the port registry includes the port number (Port Number), and the value field of the port registry includes the service name, protocol type (Protocol Name), description (Description), assignment status (Assignment Status) and reference documents, etc. The port registry can be further divided into two categories. One type of port registry comes from a standardization organization, and the port registry from the standardization organization includes, for example, the IANA port registry. The other type of port registry comes from a private protocol, and the port registry from the private protocol is, for example, a private protocol port registry, for example, to assign corresponding services to port numbers that have not yet been assigned by IANA, and record the correspondence between the port number and the service through the private protocol port registry.

In some embodiments, the static data is a file other than the port registry that can characterize the purpose of the port number or contain port number attributes. For example, the static data is port usage information, operating system or application usage documentation, or communication device manufacturer information. The static data can be a formatted file (such as a table) or an unformatted file. This embodiment does not limit the type of static data used to extract the characteristics of the port number.

(5) Static tags

Static tags are feature tags of port numbers extracted from static data. For example, they can be based on the protocols supported by the port in the IANA port registry, such as Secure Shell (SSH), Hypertext Transfer Protocol (HTTP), and File Transfer Protocol (FTP). Alternatively, static tags can be processed based on the service description information corresponding to the port in the IANA port registry (such as File Transfer and Unassigned).

(6) Dynamic data

Dynamic data refers to data related to the actual connection usage of a port. For example, dynamic data includes data related to actual deployment scenarios. Examples of dynamic data include log records. For example, dynamic data may be traffic records generated by a communication device during the process of forwarding data streams. Another example is threat detection records generated by a security device during threat detection. In the embodiments of this application, dynamic data primarily serves as another data source for feature tags.

(7) Dynamic Tags

A dynamic label refers to a label for a port number extracted from dynamic data. For example, a port's dynamic label includes the type of network attack targeting that port. Another example is a port's dynamic label includes the set of IP addresses that interacted with the port for a predetermined period of time within a historical time period. Another example is a port's dynamic label includes the set of service types provided by the port for a predetermined period of time within a historical time period. For example, during the development of an application, port A is temporarily used to provide the application's services and is bound to the application. Subsequently, port A is no longer used to provide the application's services, but port B is used to provide the application's services and is bound to the application. In this scenario, after extracting the feature label set of port A and the feature label set of port B from the dynamic data, it can be determined that there is overlap between the feature label set of port A and the feature label set of port B. Based on this, it can be determined that the similarity between port A and port B is higher than the similarity between port A and port B.

(8) Internet Assigned Numbers Authority (IANA) Port Registry

IANA is the world's leading authority for assigning Internet addresses. IANA manages IP addresses, domain names, and many other parameters used on the Internet. Embodiments of the present application relate to a file provided by IANA that describes the relationship between Internet services and port numbers. This file, also known as the IANA Port Registry, describes the relationship between Internet services and port numbers.

The access link to the IANA port registry can be found at the following address.

https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml .

The following is an example of an application scenario of the embodiment of the present application.

Port numbers are one of the most frequently processed fields in various communication devices. In machine learning systems for data communication data, port numbers are extremely important features. However, unlike common features with continuous ranges (such as traffic values and number of connections), port numbers inherently have discrete ranges. For example, sample points A, B, and C have port numbers 79, 80, and 8080, respectively, while all other features are identical. The numerical difference between port numbers 79 and 80 is 1 (80 - 79 = 1), and the numerical difference between port numbers 80 and 8080 is 8000 (8080 - 80 = 8000). Although the numerical difference between <79, 80> (i.e., 1) is smaller than the difference between <80, 8080> (i.e., 8000), this does not mean that the similarity between sample points A and B is higher than that between sample points B and C. This is because the numerical difference between different port numbers does not represent the actual degree of difference between the port numbers in the semantic dimension.

Therefore, various popular continuous-value-oriented feature engineering techniques (i.e., processing and manipulating original features to transform them into features that better express the essence of the problem) cannot be directly applied to port numbers.

Methods for encoding port numbers generally include one-hot encoding and label encoding.

When using the one-hot encoding method, the port number is encoded as a binary vector. If there are d different port numbers in all samples, a d-dimensional feature vector is used to represent a port number. The value of each dimension of the feature vector of the port number is either 0 or 1. The d port numbers are sorted according to predetermined rules (such as alphabetical order or numerical size rules), and a dimension number is assigned to each port number in turn, with the number range from 1 to d. For the port number of each sample, the value of the dimension corresponding to the port number is set to 1, and the values of the remaining dimensions other than the dimension corresponding to the port number are set to 0. For example, if there are three port numbers {79, 80, 8080} in all samples, the three port numbers can be encoded as [1, 0, 0], [0, 1, 0] and [0, 0, 1] respectively.

When using label encoding, if all samples have d different port numbers, then d different numbers are used to represent the d port numbers. For example, if all samples have three port numbers {79, 80, 8080}, then the three port numbers can be encoded as 0, 1, and 2 respectively.

However, both the one-hot encoding method and the label encoding method have the following three defects.

(1) Information loss

Both one-hot encoding and label encoding ignore the semantic correlations that may exist between different ports. For example, after encoding port 79, port 80, and port 8080 using the above method, the similarity between the encoding of port 79 and port 80 is always the same as the similarity between the encoding of port 80 and port 8080.

(2) Curse of Dimensionality

Common systems have a total of 65,536 possible port numbers. Using one-hot encoding can result in a massive vector of tens of thousands of dimensions; using label encoding can result in tens of thousands of different values. Consequently, both one-hot encoding and label encoding methods dramatically increase the computational complexity of the corresponding machine learning system. To address this issue, only high-frequency port numbers are typically encoded individually, while the remaining low-frequency port numbers are grouped under "Other" and encoded as a whole. However, this inevitably results in information loss for low-frequency port numbers—a drawback that is particularly pronounced when the port number distribution is too skewed or too small.

(3) It is difficult to take into account both dynamic and static data

For example, either only static information is encoded (for example, port 80 is directly encoded as "80"), or only dynamic information is encoded (for example, encoding according to the frequency of the port's appearance in the data), failing to take into account both static data and dynamic data. As a result, the port encoding may be difficult to fully and accurately represent the port's semantics.

In view of this, in some embodiments of the present application, the port number itself is not directly encoded, but the background field is encoded by utilizing the co-occurrence relationship between the port number and the background field. For example, the attributes of the port number (such as the application related to the port number, the service related to the port number, the protocol related to the port number, the IP address that has an interactive relationship with the port number, and other data related to the port number, which can also be called the semantic information of the port number) are obtained from the background field other than the port number, and the feature tag set of the port number is obtained based on the attributes of the port number.

Since the feature tag set of the port number represents the attributes of the port number, for example, if two port numbers have common attributes, there will be the same part in the feature tags of the two port numbers. In other words, the encodings of the two port numbers are partially the same or similar. Therefore, the correlation (i.e., similarity) between the feature tags of the two port numbers can represent the closeness of the association between the two port numbers. Therefore, the feature tag of the port number retains the semantic information of the port number and avoids information loss.

In addition, the number of values that the port number attribute can take is usually much smaller than the number of values that the port number itself can take, thus effectively avoiding the curse of dimensionality.

Furthermore, to address the difficulty of balancing dynamic and static data, some embodiments of the present application extract a static tag set for the port number from static data via a static path, and extract a dynamic tag set for the port number from dynamic data via a dynamic path. The static and dynamic tag sets are then fused, and the fused feature tag set is used as the feature tag set for the port number. Because the fused feature tag set is sourced from both dynamic and static data, balancing dynamic and static data is achieved.

In the embodiment of the present application, the methods for obtaining the characteristic tags of the port number include static path, dynamic path and dynamic and static path fusion. These methods are respectively described with examples below.

A. Static path

A static path refers to a static label that extracts a port number from static data. For the definition of static data, refer to the previous description.

In some embodiments, the static tag includes an application identifier corresponding to the port number. For example, a set of application identifiers corresponding to the port number is obtained from static data, and the set of application identifiers is used as a static tag set for the port number, or the set of application identifiers is processed according to predetermined rules to obtain a static tag set for the port number. For example, in the case where a port number is bound to an application identifier, the set of static tags obtained from the static data may include an application identifier corresponding to the port number; in the case where a port number is bound to multiple application identifiers, the set of static tags obtained from the static data may include multiple application identifiers corresponding to the port number. The application identifier is used to identify the application or service bound to the port number (or the purpose of the port number). For example, the application identifier is an application name or a service name. The predetermined rule is, for example, to truncate the application identifier and use the truncated application identifier as the static tag; the predetermined rule is, for example, to vectorize the application identifier and use the application identifier in vector form as the static tag; the predetermined rule is, for example, to extract a field of a predetermined number of bits from the application identifier as the static tag;

In some embodiments, the static label includes a protocol identifier corresponding to the port number. For example, a set of protocol identifiers corresponding to the port number is obtained from static data, and the set of protocol identifiers is used as a static label set for the port number, or the set of protocol identifiers is processed according to a predetermined rule to obtain a static label set for the port number. The static label set for the port number includes one or more protocol identifiers corresponding to the port number. The protocol identifier is used to identify the protocol bound to the port number. For example, the protocol identifier is used to identify the application layer protocol bound to the port number, the transport layer protocol bound to the port number, or the network layer protocol bound to the port number. For example, the protocol identifier is a protocol name or a protocol number. For example, the protocol identifier includes TCP, User Datagram Protocol (UDP), Stream Control Transmission Protocol (SCTP), or SSH.

Taking a port registry as an example of static data, in some implementations, the port number is used as an index to search the port registry for a context field corresponding to the port number. The context field includes at least one of a Service Name field and a Description field. One or more application identifiers are extracted from the Service Name field corresponding to the port number, and one or more protocol identifiers are extracted from the Description field to obtain a static tag set for the port number.

When extracting a static tag set, the two static data, the IANA port registry and the private protocol port registry, can be used alone or in combination. In some embodiments of combining the IANA port registry and the private protocol port registry, the content of the background field corresponding to the port number is determined from the IANA port registry to obtain a static tag A; and the content of the background field corresponding to the port number is determined from the private protocol port registry to obtain a static tag B; and static tag A and static tag B are fused to obtain a static tag set for the port number. The fusion of static tag A and static tag B can be, for example, taking the union of static tag A and static tag B, or taking the weighted sum of static tag A and static tag B according to a predetermined weight.

For example, for port 80, using the port number (80) of port 80 as an index, the Service Name corresponding to port 80 is searched from the IANA port registry, and www and http are obtained. Therefore, the static label of port 80 is determined to include {www, http}. For another example, for port 750, the Service Name corresponding to port 750 in the IANA port registry includes Rfile, loadav, and kerberos-iv. Therefore, the static label of port 750 is determined to include {Rfile, loadav, kerberos-iv}. For another example, for port 79, using the port number (79) of port 79 as an index, the Service Name corresponding to port 79 is searched from the IANA port registry, and finger is obtained; using the port number (79) of port 79 as an index, the application name corresponding to port 79 is searched from the private protocol document, and private_79 is obtained; the union of finger and private_79 is taken, and the static label of port 79 is determined to include {finger, private_79}.

Since the application identifier or protocol identifier of the port number obtained through the static path can both characterize the semantics of the port (the purpose of the port), using the application identifier or protocol identifier as the static label set of the port number prevents the static label set of the port number from losing the semantics of the port. The similarity between the static label sets of different port numbers can characterize the semantic correlation of the port numbers. For example, ports A and B are bound to the same application, while ports A and C are bound to different applications. After determining the static label sets of ports A, B, and C based on the port-bound applications, the static label sets of port A and port B will have the same application identifier (or the static label sets of ports A and B will have overlapping parts), thereby characterizing the correlation between ports A and B. Similarly, the static label set of port A and the static label set of port C will not have the same application identifier (or the static label sets of ports A and C will have no overlapping parts), thereby characterizing the correlation between ports A and C. After determining the similarity between port A and port B based on the static label set of port A and the static label set of port B, and determining the similarity between port A and port C based on the static label set of port A and the static label set of port C, the similarity between port A and port B is greater than the similarity between port A and port C.

It can be seen that the method of extracting the static label set of the port number through the static path solves the problem of information loss on the one hand, and on the other hand, because the number of values of the application identifier and the number of values of the protocol identifier are far less than the number of values of the port number, it avoids dimensionality explosion.

In some embodiments, the description text of the port number is obtained, and the description text of the port number is vectorized using the term frequency-inverse document frequency (TF-IDF) method to generate a TF-IDF feature vector corresponding to each description text as a static label of the port number.

In some embodiments, the hash value of the background field is further considered when extracting the static label, thereby improving the discrimination of the static label and helping to distinguish different port number uses through different static labels. For example, for each background field related to the port number use in the port registration table, it is determined whether the hash value of the background field is higher than the hash value threshold. If the background field is related to the port number use and the hash value is higher than the hash value threshold, the static label of the port number is extracted from the background field. The hash value is used to characterize the number of different values in the background field. For example, if the hash value of a background field in the port registration table is higher than the hash value threshold, or the richness or diversity of the background field is relatively strong, the amount of information contained in the background field is relatively large. Based on the background field, it is helpful to distinguish port numbers with different uses, and the static label can be determined based on the content of the background field. On the other hand, if the hash value of a background field in the port registry is too low, for example, for the Transport Protocol field in the port registry, the value of the Transport Protocol field usually has only two types, one type is UDP and the other type is TCP. The number of types of values in the Transport Protocol field (2) is much smaller than the total number of port numbers. The hash value of the Transport Protocol field is too low, so there is no need to determine the value of the Transport Protocol field as a static label. For another example, for the Service Name field in the port registry, the Service Name field itself is relatively rich and is related to the purpose of the port number. Therefore, the static label of the port number can be determined based on the content of the Service Name field.

In some embodiments, after extracting the static label set for the port number using a static path, a set of mapping relationships is generated based on the port number and the static label set. Each mapping relationship in this set of mapping relationships is, for example, a key-value pair, where the key is the port number and the value is the static label set corresponding to the port number. For example, for the four ports of port 79, port 80, port 8080, and port 52297, the static labels corresponding to these four ports obtained based on the static path and the mapping relationships constructed are shown in Table 1 below.

Table 1

The static data shown in the table above comes from at least one of IANA and/or private protocols. For ease of illustration, the application names corresponding to port numbers provided by private protocols are prefixed with "private." For example, port number 52297 corresponds to two applications in the private protocol: one application is named private_52297a, and the other application is named private_52297b.

For example, using the application name corresponding to a port number as a static label and the IANA port registry as static data, port 88 corresponds to Kerberos, port 749 also corresponds to Kerberos, port 750 corresponds to Kerberos, rfile, and loadav, port 2007 corresponds to RAID and DecTalk, port 2015 corresponds to Cypress and RAID, and port 2017 corresponds to BootClient and Cypress. After extracting the application name corresponding to each port based on the IANA port registry, the resulting static labels for each port are shown in Table 2 below.

Table 2

By generating a mapping relationship between the port number and the static tag set, if further inference tasks need to be performed based on the static tag set of the port number, a hash mapping is performed using the port number as the key to obtain the static tag set corresponding to the port number. Due to the low computational overhead (the process of extracting the static tag set only involves scanning the background fields, with a complexity of O(N); the inference process only involves a simple hash, with a complexity of O(1), it is basically not limited by specific application scenarios and technical fields, and can be widely deployed in various communication products that apply machine learning algorithms.

In some implementations, considering that the port registry may contain redundant content, the port registry is first de-redundanted, and then the static tag of the port is extracted from the de-redundant port registry. By reducing the redundant content in the port registry, the data volume of the port registry is reduced, and the amount of data required to be processed when extracting feature tags from the port registry is reduced, thereby improving the efficiency of extracting feature tags.

Regarding the de-redundancy processing method, in some embodiments, each row of fields in the port registry is traversed and matched against predetermined rules. If a row of fields matches the predetermined rules, the row of fields is deleted. For example, semantic analysis is performed on the contents of each background field in the port registry. If it is determined that the content of a background field in the port registry contains duplicate semantics, the row containing the background field is deleted. In another example, natural language processing technology or regular expressions are used to detect whether the content of each background field in the port registry contains keywords that represent duplication, such as "duplicate". If it is detected that the content of a background field in the port registry contains keywords that represent duplication, the row containing the background field is deleted.

As an example, traverse each row of data in the IANA port registry and extract the notes field (field name Assignment Notes) in each row, that is, the content of the last column of the field. If the notes field in any row of the IANA port registry is identified as containing the word "duplicate" with duplicate semantics, then delete this row of fields from the IANA port registry. For example, in the IANA port registry, there are two rows of fields with the same content for port 80. The content of the last column (Assignment Notes) of these two rows includes "This is a duplicate of the "http" service and should not be used for discovery purposes." When it is detected that a row of fields for port 80 contains the word "duplicate", delete this row of fields.

In some embodiments, considering that there may be multiple expressions for the same term in the port registry, if it is identified that there are terms with the same semantics but inconsistent expressions in multiple fields in the port registry, the multiple fields are subjected to expression consistency processing. For example, a target expression is selected from the multiple expressions, only the target expression is retained, and the other expressions other than the target expression are replaced with the target expression. For example, when it is identified that the full English name and the English abbreviation of the same term exist in the port registry, one of the expressions of the full English name and the English abbreviation can be retained, such as choosing to retain only the full English name and replacing the English abbreviation of the term with the full English name, so that the expression of the same term is consistent.

For example, Table 3 below shows part of the content in the IANA port registry. The table below gives two ways of expressing the term FTP, one is the full English name File Transfer Protocol, and the other is the English abbreviation FTP. In the process of preprocessing the IANA port registry, the English abbreviation FTP that appears in the table below can be replaced with the full English name File Transfer Protocol to make the expression of the term FTP consistent in the entire IANA port registry and reduce content redundancy.

Table 3

Of course, extracting the static label from the port registry is only exemplary. In other implementations, the static label is extracted from other files outside the port registry that can characterize the purpose of the port number.

By using the application identifier corresponding to a port number as the static label for the port number, the number of applications associated with a port number is typically small. For example, in most cases, a port number corresponds to only one application, while in a few cases, a port number corresponds to multiple applications. However, even in cases where a port number corresponds to multiple applications, the number of applications corresponding to a port number is generally only in the single digit. Therefore, by using the application identifier corresponding to the port number as the static label set for the port number, the size of the static label set for the port number, or the dimension of the feature vector, does not exceed 10, which is much smaller than the tens of thousands that can be achieved when using one-hot encoding or label encoding, thus avoiding the curse of dimensionality. Furthermore, since the static label for the port number can be obtained using the port registry, the computational overhead is relatively low, and there is no need to update the port registry, thus reducing the overhead incurred by additional updates to the port registry.

In some embodiments, considering that the degree of correlation between different background fields and port numbers in static data such as a port registry may vary, a determination is first made based on the dimension of correlation as to whether a background field in the port registry is suitable for extracting a static label for the port number. If the similarity between the background field in the port registry and the port number exceeds a similarity threshold, the static label for the port number is extracted based on the background field. If the similarity between the background field and the port number is below the similarity threshold, there is no need to extract a static label for the port number based on the background field.

In some implementations, whenever it is found that the port registry is updated, the feature tag set of the port number is regenerated based on the updated port registry, thereby updating the feature tag set of the port number.

B. Dynamic Path

Dynamic paths refer to encoding port numbers based on dynamic data to obtain dynamic labels. For example, in the security field, when attackers perform specific types of network attacks, they often repeatedly interact with a few fixed types of ports. For example, please refer to Table 4 below. Each row in Table 4 represents an attack type, each column in Table 4 represents a port, and each element in Table 4 represents the number of times the corresponding port appears in the records of the corresponding attack type. For example, the value corresponding to port 53 and attack type 6 is 40, indicating that port 53 appears 40 times in the records of attack type 6. As can be seen from Table 4, port number 21 is only related to attack type 4, port number 53 appears in attack types 5 and attack type 9, and port number 80 appears in each attack type from attack type 5 to attack type 9.

Table 4

Given the relationship between ports and network attacks, in some implementations, the type of network attack targeting a port is determined based on the firewall's threat detection records, and a dynamic label for the port is obtained based on the network attack type corresponding to the port number. For example, the type of network attack corresponding to the port number is encoded to obtain a feature vector for the port number. For example, each dimension in the feature vector corresponds to an attack type, and the value of each dimension in the feature vector indicates whether the corresponding attack type has attacked the port.

For example, if there is a record in the threat detection record that the i-th network attack has attacked a port, the value of the i-th dimension in the feature vector of the port is 1; if there is no record in the threat detection record that the i-th network attack has attacked a port, the value of the i-th dimension in the feature vector of the port is 0.

For example, a feature vector for a port number is generated by encoding the type of network attack corresponding to the port number and the number of times the port number has been attacked by the network attack. The value of each dimension in the feature vector represents the frequency with which the port number has been attacked by the corresponding network attack. The higher the frequency with which a port is attacked by the corresponding network attack, the higher the value of the corresponding dimension in the feature vector for the port number. For example, if a threat detection record contains a record of a port being attacked by the i-th network attack, and the i-th network attack has attacked the port a total of k times, the value of the i-th dimension in the feature vector for the port is k.

In other embodiments, when encoding the port number, the frequency of occurrence of the port in the attack type is also considered. For example, different weights are assigned to ports that appear frequently and ports that appear infrequently in a specific attack type, and the characteristic vector of the port is determined by combining the weight corresponding to the frequency and the frequency of occurrence of the port in the attack type.

Among them, the encoding method based on network attacks is, for example, one-hot encoding, TF-IDF encoding, embedding vector encoding (Embedding), etc. This embodiment does not limit the specific encoding method used.

For example, port number 21 can be encoded as a 7-dimensional feature vector, where each dimension corresponds to an attack type. The 7-dimensional feature vectors correspond to attack type 4, attack type 5, attack type 6, and so on, attack type 10. Based on this encoding method, the feature vector for port number 21 is 1000000, the feature vector for port number 53 is 0100001, and the feature vector for port number 80 is 1111111.

Since the feature vector for port number 21 does not overlap with the feature vector for port number 53, the similarity between the feature vectors for port number 21 and port number 53 is small, indicating that the similarity between port number 21 and port number 53 is small. Since the feature vector for port number 53 and the feature vector for port number 80 overlap in the second and seventh dimensions, the similarity between the feature vector for port number 53 and port number 80 is large, indicating that the similarity between port number 53 and port number 80 is large. By encoding port numbers into feature vectors based on attack types, the feature vectors can retain the association between port numbers in terms of attack types, and the feature vectors are related to the attacks suffered by the corresponding ports in actual applications.

The fact that the feature vector obtained by encoding the port number is a binary vector is merely an example. In other implementations, the feature vector obtained by encoding the port number may also be a value other than 0 or 1.

In some implementations, considering that when a device runs an application, the destination ports accessed by the application typically follow certain patterns, the same application often frequently accesses a few fixed ports on the server. For example, the destination ports for a server accessed using NetBIOS typically include three fixed ports: port 137, port 139, and port 138. Therefore, if multiple ports interact with the same object (such as an application or remote device), these multiple ports may be associated.

Based on this, the identifier of the object that has an interactive relationship with the port of the first communication device can be obtained from the operation log record of the first communication device; and the dynamic label of the port number can be obtained based on the identifier of the object that has an interactive relationship with the port. The object that has an interactive relationship with the port of the first communication device is, for example, the second communication device or an application in the second communication device. Accordingly, the dynamic label of the port number is, for example, the IP address of the second communication device, a part of the field in the IP address of the second communication device, the Media Access Control Address (MAC) address of the second communication device, the identifier of the area where the second communication device is located, or the identifier of the application in the second communication device.

For example, the dynamic label of the source port number is determined based on the destination IP address that has an interactive relationship with the source port number. For another example, the dynamic label of the destination port number is determined based on the source IP address that has an interactive relationship with the destination port number.

As an example, the log fragment recorded in the communication device includes the content shown in Table 5. The source port column in Table 5 records the port number of each port in the communication device, and the destination address column in Table 5 records the destination address accessed by the corresponding port.

Table 5 Log snippet

In conjunction with the content shown in Table 5 above, the IP address that has interacted with the port can be selected as the background field, and the dynamic label of the port number can be obtained based on the IP address that has interacted with the port, thereby dynamically labeling each port. For example, the dynamic label of port number 56551 is {172.31.65.123}; the dynamic label of port number 443 is {172.31.65.124,172.31.65.123}; the dynamic label of port number 80 is {172.31.65.126}; the dynamic label of port number 62746 is {172.31.65.124}; and the dynamic label of port number 60628 is {172.31.65.124}.

Since the dynamic label of the port number is obtained based on the IP address that has an interactive relationship with the port number, on the one hand, the dynamic label can represent the semantics of the port number, and the similarity between the dynamic labels of different port numbers can represent the correlation between the semantics of different port numbers, thereby avoiding information loss. For example, from the log fragment shown in Table 5 above, since the three port numbers 443, 62746, and 60628 have an interactive relationship with the same IP address 172.31.65.124, it can be determined that the three port numbers 443, 62746, and 60628 have an association relationship, and the dynamic labels of the three port numbers 443, 62746, and 60628 overlap, and the dynamic labels of the three port numbers 443, 62746, and 60628 all include the IP address 172.31.65.124. Among them, the IP address that has an interactive relationship with port number 62746 is consistent with the IP address that has an interactive relationship with port number 60628, and the dynamic label of port number 62746 completely overlaps with the dynamic label of port number 60628, which also reflects that port number 62746 and port number 60628 are highly correlated. The IP address that has an interactive relationship with port number 443 is different from the IP address that has an interactive relationship with port number 80, and the dynamic label of port number 443 is different from the dynamic label of port number 80, which also reflects that port number 443 has a low correlation with port number 80. This shows that the method of encoding port numbers based on the IP addresses that have an interactive relationship with the port numbers can reflect the similarity or correlation between port numbers.

On the other hand, the number of IP addresses that interact with a port number is usually limited. Even if there are a large number of IP addresses that interact with a port number, the dimension of the dynamic label set can be reduced by retaining IP addresses that appear more frequently than a predetermined frequency from the IP addresses that interact with the port number. This prevents the size of the dynamic label set from reaching tens of thousands as in one-hot encoding or label encoding, thus avoiding the dimensionality disaster.

On the other hand, the dynamic tag of the port number can be updated through dynamically updated business data. This maintains the timeliness of the dynamic tag of the port number, allowing it to change as business data is updated, and making the characteristic tag set of the port number match the real-time nature of business data and the locality of business data.

Methods for updating the dynamic tag set of a port number include a timed update method and a rolling update method. For example, at predetermined time intervals, the log records generated by the communication device during the current time period are obtained, and the current dynamic tag set of the port number is obtained based on the log records generated by the communication device during the current time period; the historical dynamic tag set of the port number is updated based on the current dynamic tag set of the port number. For another example, considering that different service providers may have different applications for the same port, for example, the game service provider originally rented port A of the communication device to provide game services, and later the video service provider rented port A of the communication device to provide video services, resulting in different uses of port A of the communication device at different times. Based on this, when it is determined that the service provider corresponding to port A is updated from the first service provider to the second service provider, the log records generated during the period when the second service provider applied port A are obtained, and the current dynamic tag set of the port number is extracted based on the log records generated during the period when the second service provider applied port A; the historical dynamic tag set of the port number is updated based on the current dynamic tag set of the port number.

In some embodiments, the IP address interacting with the port is used directly as the port's characteristic tag. In other embodiments, the IP address interacting with the port is processed using a predetermined rule, and the resulting fields are used as the port's characteristic tag. For example, the first predetermined number of fields from the IP address interacting with the port are intercepted and used as the port's characteristic tag. For example, the first field from the IP address interacting with the port is intercepted and used as the port's characteristic tag.

By intercepting the first field of the IP address that has an interactive relationship with the port as the port's feature tag, the length of the port's feature tag is reduced, making the port's feature tag more concise and the amount of data in the port's feature tag less, thereby reducing the storage space required for the port's feature tag and the amount of data required to process when performing further reasoning tasks based on the port's feature tag. In addition, the first field in the IP address usually means the identifier of the network segment to which the device belongs. By using the first field of the IP address that has an interactive relationship with the port as the port's feature tag, the port's feature tag can represent the network segment where the remote device interacting with the port is located.

Of course, the first two or three fields of the IP address that interacts with the port can also be intercepted as the port's characteristic tag; or the number of fields to be intercepted from the IP address that interacts with the port can be determined based on the value of the first byte of the IP address that interacts with the port, and the number of fields intercepted from the IP address can be used as the port's characteristic tag. Alternatively, the hash value of the IP address that interacts with the port can be determined, and the hash value of the IP address can be used as the port's characteristic tag.

In some embodiments, the dynamic label of the port number is also used to characterize the number of interactions between an object such as a remote device or application and the port. For example, the dynamic label of the port number is obtained by concatenating the identifier of the object that has an interactive relationship with the port and the number of interactions between the object and the port. For example, the dynamic label of the port number includes a first field and a second field, the first field includes the identifier of the object that has an interactive relationship with the port, and the second field includes the number of interactions between the object and the port. For another example, the dynamic label of the port number is obtained by concatenating the identifier of the object that has an interactive relationship with the port and the number of interactions between the object and the port.

For example, the IP addresses that have an interaction relationship with the port and the number of interactions between the IP addresses and the port are determined from dynamic data such as log records. The IP addresses that have an interaction relationship with the port and the number of interactions between the IP addresses and the port are concatenated to obtain the dynamic label of the port. The dynamic label of the port includes the IP addresses that have an interaction relationship with the port and the number of interactions between the IP addresses and the port. For example, if port A has an interaction relationship with IP address 8.8.8.8 and port A has interacted with IP address 8.8.8.8 4 times, the dynamic label of port A is 8.8.8.8 4.

Because the dynamic label of a port retains the number of interactions between the port and objects such as remote IP addresses and remote applications, the dynamic label of the port can provide richer background information. The dynamic label of the port can more fully represent the semantics of the port, further improving the accuracy of reasoning tasks based on the dynamic label of the port.

In some embodiments, after obtaining the dynamic tag corresponding to the port number, a set of mapping relationships is maintained based on the port number and the dynamic tag. The key in each mapping relationship is still the port number, and the value in each mapping relationship is still the set of dynamic tags. The dynamic tag of the port number is obtained from other fields (referred to as "background fields") other than the port number in multiple records containing the port number. For example, the dynamic tag of the port number may include all IP addresses that have interacted with the port or all application names that have involved the port.

C. Dynamic and static path fusion

In some embodiments, a dynamic path and a static path are fused to extract characteristic tags. For example, a dynamic path is used to extract the characteristic tags of a port number to obtain a dynamic tag set; a static path is used to extract the characteristic tags of a port number to obtain a static tag set; and the static tag set is fused with the dynamic tag set to obtain a characteristic tag set of the port number.

Methods for fusing two feature label sets include a union method, a weighted sum method, and a set obtained by a single path. The union method is, for example, to obtain the union of a static label set and a dynamic label set to obtain a feature label set for the port number. The weighted sum method is, for example, to assign corresponding weights to the dynamic path and the static path respectively, and perform weighted summation on the static label set and the dynamic label set based on the weight corresponding to the dynamic path and the weight corresponding to the static path to obtain a feature label set for the port number. The single path method is, for example, to select one from the static label set and the dynamic label set as the feature label set for the port number. Which method to use to fuse the two feature label sets can be determined according to the actual scenario.

Since feature labels are extracted by fusing dynamic paths with static paths, not only information loss and dimensionality disaster are avoided, but also, since the feature label set includes two data sources, dynamic data and static data, the feature label extraction process takes into account both static data and dynamic data, allowing the feature labels to more fully represent the semantics of the port.

In any of the above-mentioned dynamic paths, static paths, and dynamic-static path fusion methods, before extracting feature labels from the background fields in the static data or/and the background fields in the dynamic data, it is optionally determined based on the dimension of correlation whether the background field is suitable for extracting the feature label of the port number. If the similarity between the background field and the port number is higher than the similarity threshold, the feature label of the port number is extracted based on the background field. If the similarity between the background field and the port number is lower than the similarity threshold, there is no need to extract the feature label of the port number based on the background field. Based on this, it is equivalent to verifying the correlation between the background field and the port number. The background field that is not closely related to the port number does not need to participate in the feature label extraction process, and the background field that is closely related to the port number will participate in the feature label extraction process. This reduces the interference of the background field that is not closely related to the port number on the one hand, and also reduces the number of dimensions of the feature label.

For example, in the process of extracting the static label of the port number from the port registry, the similarity between the background field of each dimension in the port registry and the port number is determined. If the similarity between the background field of a dimension in the port registry and the port number is higher than the similarity threshold, the static label of the port number is extracted based on the background field. If the similarity between the background field of a dimension in the port registry and the port number is lower than the similarity threshold, there is no need to extract the static label of the port number based on the background field. Based on this, fields in the port registry that have little relationship with the semantics of the port number, such as the person's name and assignment status, do not need to serve as the data source of the static label of the port number.

For example, in the process of extracting dynamic labels of port numbers from traffic logs, the traffic logs usually record the region where the source IP address from which the traffic comes is located. If it is determined that the similarity between the region where the source IP address is located and the destination port number is higher than the similarity threshold, the region where the source IP address is located can be used as a feature label of the destination port number.

For example, when extracting dynamic labels for port numbers from threat logs, a similarity analysis can be performed between the destination port number and the attack type to determine the similarity between the two. If the similarity between the attack type and the destination port number exceeds a similarity threshold, the attack type is used as the characteristic label for the destination port number.

The method for calculating the similarity between the background field and the port number is, for example, a method for calculating the similarity between two discrete variables, such as covariance, chi-square test method, correlation coefficient, Jacquard coefficient, etc.

The above lists various methods for extracting feature tags of port numbers. The following describes how to use feature tags of port numbers in combination with some specific application scenarios.

In some embodiments, the similarity between different port numbers is determined based on the similarity between the feature tag sets of different port numbers. For example, if the similarity between the feature tag sets of different port numbers is higher than a similarity threshold, it is determined that there is correlation between the different port numbers. If the similarity between the feature tag sets of different port numbers is lower than the similarity threshold, it is determined that there is no correlation between the different port numbers.

Taking the example of using the Jacquard coefficient to represent the similarity between feature tag sets of different port numbers, for example, the Jacquard coefficient between the feature tag sets of two port numbers is determined. If the Jacquard coefficient exceeds 50%, the two port numbers are determined to be correlated. If the Jacquard coefficient does not exceed 50%, the two port numbers are determined to be uncorrelated. The formula for calculating the Jacquard coefficient is as follows. In the following formula, A represents the feature tag set of one port number, B represents the feature tag set of another port number, and J represents the similarity between the feature tag sets of the two port numbers.

In other embodiments, the Jacquard coefficient is replaced by other algorithms such as correlation coefficient, cosine similarity or Euclidean distance that can characterize the similarity between sets of different discrete variables. This embodiment does not limit the specific algorithm used to determine the similarity between feature tag sets with different port numbers.

For example, when calculating the similarity between different port numbers based on the feature tags of the ports shown in Table 2, the applications corresponding to port 88 and port 749 are exactly the same, so the similarity between port 88 and port 749 is 1; there is one identical application among the applications corresponding to port 88 and port 750, and port 88 and port 750 correspond to a total of four applications, so the similarity between port 88 and port 750 is 1/3; the application corresponding to port 88 is different from the application corresponding to port 2007, the application corresponding to port 88 is different from the application corresponding to port 2015, and the application corresponding to port 88 is different from the application corresponding to port 2017, so the similarity between port 88 and port 2007 is 0, the similarity between port 88 and port 2015 is 0, and the similarity between port 88 and port 2017 is 0.

In some embodiments, when determining the similarity between different ports based on their feature tags, the feature tags of the ports may be weighted based on the number of interactions between the IP address and the port. The weight is used to represent the importance of the feature tag of the port, and the weight of the feature tag of the port is positively correlated with the number of interactions.

For example, port A has interaction relationships with both IP address 8.8.8.8 and IP address 4.4.4.4, and the number of interactions between port A and IP address 8.8.8.8 is greater than the number of interactions between port A and IP address 4.4.4.4; port B also has interaction relationships with both IP address 8.8.8.8 and IP address 4.4.4.4, and the number of interactions between port B and IP address 8.8.8.8 is less than the number of interactions between port A and IP address 4.4.4.4; port C also has interaction relationships with both IP address 8.8.8.8 and IP address 4.4.4.4, and the number of interactions between port B and IP address 8.8.8.8 is greater than the number of interactions between port A and IP address 4.4.4.4. In this example, when extracting feature tag sets for three ports, Port A, Port B, and Port C, and determining the similarity between different ports within the three ports based on the feature tag sets, it can be determined that the similarity between Port A and Port C is greater than the similarity between Port A and Port B. This is because the feature tag sets for the ports include the number of interactions between the IP address and the port. This number of interactions between the IP address and the port can be factored into the similarity comparison, giving a greater weight to IP addresses with a greater number of interactions.

In some further embodiments, text contained in the background field of static data and/or dynamic data is extracted, the text is segmented, the obtained words are sorted in descending order of frequency of occurrence, and the words with the highest frequency of occurrence are selected as the feature tag set of the port number.

In some implementations, after obtaining feature tag sets for different port numbers based on multiple log records, the similarity between the feature tag sets for different port numbers can be used to determine the similarity between the different log records. Based on the similarity between the different log records, the multiple log records can be clustered, thereby automatically discovering potential associations between the multiple log records and automatically grouping related log records together to facilitate storage and further processing of the log records. Taking alarm logs as an example, the clustering process for alarm logs is illustrated below through Example 1.

Example 1: Clustering

For example, for asset 1.3.5.7, attackers often use random ports (such as 52333, 49999, 56666, etc. in this example) as source ports to attack asset 1.3.5.7, thereby generating a set of firewall alarm logs as shown in Table 6 below.

Table 6

The alarm log correlation analysis module needs to identify related alarm logs among these isolated alarm logs and automatically group the related alarm logs together to facilitate review and disposal by operation and maintenance personnel.

For example, a network administrator sets the association rule such that when more than 50% of the fields in two alarm logs (i.e., more than two of the four fields) are correlated, the two alarm logs are considered to be associated. If the embodiment of the present application is not used, only the association between alarm logs 1 and 3 will be found; the other alarm logs will be considered to have no association.

By adopting the embodiment of the present application, a mapping relationship between ports and feature tags as shown in Table 7 below can be generated from a dynamic path based on the alarm log shown in Table 6 above. For example, for port 79, since port 79 appears in the alarm log numbered 1 as identified from the alarm log shown in Table 6 above, port 79 is the destination port, and the IP address of the IP address that has an interactive relationship with port 79 is the source IP address 192.1.1.1, the feature tag of port 79 is {192.1.1.1}. For port 80, as shown in Table 6 above, port 80 appears in both alarm logs 2 and 6. In alarm log 2, port 80 is the destination port, and the IP address interacting with port 80 is the source IP address 215.8.8.8. In alarm log 6, port 80 is the destination port, and the IP address interacting with port 80 is the source IP address 191.4.4.4. Therefore, the characteristic tag for port 80 is {215.8.8.8, 191.4.4.4}. Similarly, the mapping relationship between port numbers and characteristic tags is determined as shown in Table 7 below.

Table 7

In this example, in order to merge the dynamic path and the static path, for each port number that appears in the alarm log, the feature label set of the port number obtained using the static path and the feature label set of the port number obtained using the dynamic path are merged. The results are shown in Table 8 below.

Table 8

In this embodiment, the similarity between the feature tag sets of different ports can be determined by comparing the feature tag sets of different ports, and based on the similarity between the feature tag sets of different ports, the similarity between the alarm logs corresponding to different ports can be determined.

For example, using the Jaccard coefficient to determine the similarity between different feature tag sets, if the Jaccard coefficient of the feature tag sets of two ports exceeds 50%, a correlation is determined between the two ports. For example, if the Jaccard coefficient between the feature tag set of port 80 and the feature tag set of port 8080 is calculated to be 3/4, then ports 80 and 8080 are determined to be correlated. Similarly, it can be determined that alarm logs numbered 2, 4, 5, and 6 are all correlated, while alarm logs numbered 2, 4, 5, and 6 are not correlated with alarm logs numbered 1 or 3.

The method of determining the similarity of different alarm logs and then clustering the alarm logs based on the similarity between the feature tag sets of different ports is more consistent with the manual research and judgment of security experts than not using the method provided by this embodiment.

The method provided in this embodiment determines the feature tag set corresponding to the port based on the IP address that interacts with the port recorded in the alarm log. This feature tag set can more accurately determine the distance between the two ports. The distance between different samples is an important foundation for clustering tasks. Therefore, the port-dimensional distance provided in this embodiment not only provides a basis for clustering alarm logs, but also reduces information loss. Furthermore, by avoiding the curse of dimensionality, it also requires less memory space.

Example 2: Abnormal connection detection

In some implementations, based on normal historical connection information of the first communication device, a dynamic path is used to obtain a set of characteristic tags of multiple ports in the first communication device.

Normal historical connection information refers to the network connection information contained in the log recorded when the first network device was in normal status during the historical time period (such as no network attack, abnormal behavior, normal CPU memory and other operating conditions, and no security alarm was triggered). Normal historical connection information includes the local port number of the first network device and the identifier of the remote device that established a connection with the port corresponding to the local port number (such as an IP address or part of the IP address). The mapping relationship between the port and the feature tag set can realize the role of a traffic baseline.

For example, when a first communication device detects that a second communication device has established a connection with a port of the first communication device, the first communication device can determine whether the characteristics of the second communication device hit the characteristic tag set corresponding to the port; if the characteristics of the second communication device belong to the characteristic tag set corresponding to the port, indicating that the second communication device has been normally connected to the port of the first communication device in a historical time period, and the connection currently initiated by the second communication device to the port does not exceed the traffic baseline, the first communication device can allow a connection to be established with the second communication device; if the characteristics of the second communication device belong to the characteristic tag set corresponding to the port, indicating that the connection currently initiated by the second communication device to the port deviates from the traffic baseline, then it is determined that the connection between the second communication device and the first communication device is an abnormal connection, and the first communication device can refuse to establish a connection with the second communication device.

Using the IP address that interacts with a port as the port's characteristic tag is merely exemplary; in other embodiments, the IP address that interacts with a port is processed using predetermined rules, and the resulting fields are used as the port's characteristic tags. For example, the first predetermined number of fields from the IP address that interacts with a port are intercepted and used as the port's characteristic tag. For example, the first field from the IP address that interacts with a port is intercepted and used as the port's characteristic tag. Specifically, the first field in an IP address typically identifies the network segment to which the device belongs. By using the first field of the IP address that interacts with a port as the port's characteristic tag, IP addresses that interact with the same port and belong to the same network segment can be merged to obtain the network segment set corresponding to the port. When detecting abnormal connections based on the network segment set corresponding to the port, the network segment set corresponding to the port can be used as a trusted network segment set. For any data flow, if the data flow originates from any network segment in the network segment set and the destination port is the port corresponding to the network segment set, the data flow is determined to be trusted traffic. If the data flow comes from a network segment other than the network segment set and the destination port is a port corresponding to the network segment set, the data flow is determined to be abnormal traffic.

Exemplarily, normal historical connection information of the first communication device is shown in Table 9 below.

Table 9

Based on the traffic baseline, a mapping relationship between the port number of the first communication device and a set of characteristic tags is established using a dynamic path. For example, the first field of the IPv4 address connected to the port number of the first communication device is selected as the characteristic tag of the port number, resulting in the characteristic tag set shown in Table 10 below.

Table 10

The first communication device can perform abnormal connection detection based on the mapping relationship shown in Table 10 above. For example, the first communication device detects that the IP address 128.8.8.8 is connected to port 80 of the local machine, and uses port 80 as the key to look up the mapping relationship shown in Table 10 to obtain the traffic baseline {128, 255} corresponding to port 80; the first communication device matches the IP address 128.8.8.8 with the traffic baseline {128, 255} corresponding to port 80, and based on the fact that the IP address 128.8.8.8 does not exceed the traffic baseline (128∈{128, 255}), the connection between the IP address 128.8.8.8 and port 80 is not regarded as an abnormal connection; for another example, the first communication device detects that the IP address 128.8.8.8 is connected to port 79 of the local machine, and uses port 79 as the key to look up the mapping relationship shown in Table 10 to obtain the traffic baseline {8} corresponding to port 79; the first communication device matches the IP address 128.8.8.8 with the traffic baseline {8} of port 79, and based on the fact that the IP address 128.8.8.8 deviates from the traffic baseline The connection between IP address 128.8.8.8 and port 79 of the local machine is determined to be an abnormal connection.

Example 3: Classification

In some embodiments, a communication device obtains a training sample set, where each sample in the training sample set includes a port number and a context field. For example, each sample includes fields such as source address, destination address, source port, destination port, and protocol type. Furthermore, each sample in the training sample set is annotated with a type tag that identifies the type of the sample.

During the feature engineering step, the communications device converts each field contained in the sample into a corresponding feature label. For fields other than port numbers, such as IP addresses, the communications device can use corresponding feature engineering algorithms to convert them into feature labels. For source and destination ports, the communications device directly uses the mapping relationship generated via the static path to convert the port number into a feature label. The communications device concatenates or fuses the feature labels corresponding to the port numbers with those corresponding to non-port numbers to obtain the sample's feature label.

During training, the communication device converts each labeled sample into a series of feature labels using the aforementioned feature engineering process, and then trains a classification model using a decision tree algorithm. During inference, the communication device directly uses the same characteristic equation scheme to convert unlabeled samples to be predicted into a series of feature labels, and then uses the trained decision tree to classify the unlabeled samples.

Taking the application identification scenario as an example, a training sample set is generated based on a session table, with each sample in the training sample set being derived from a data flow record in the session table. Each sample includes fields such as the source address of a data flow, its destination address, source port, destination port, and protocol type. Furthermore, each sample in the training sample set is annotated with a type label corresponding to the data flow. The type label characterizes the corresponding application type, such as video application, gaming application, or voice application. During training, the communication device can convert each type-annotated sample into a series of feature labels according to the aforementioned feature engineering process, and then train an application identification model using a decision tree algorithm. During inference, the communication device directly uses the same characteristic equation scheme to convert unlabeled prediction samples into a series of feature labels. It then uses a trained decision tree to determine the corresponding application type based on the prediction sample.

In summary, the method provided in this embodiment solves problems such as information loss and dimensionality explosion by constructing a mapping relationship between port numbers and feature tags through dynamic paths and/or static paths.

In addition, the algorithm calculation is relatively simple and is applicable to existing network communication equipment.

In addition, the dynamic path and static path can be used alone or in combination.

Furthermore, dynamic and static paths are based on a variety of data sources, including the IANA port registry, private protocol port registries, traffic logs, and alarm logs, making them easily adaptable to various applications. For example, when the application type changes, the information source used to convert port numbers into feature tags and the specific processing method for this information can be selected based on the application type.

The above embodiments are summarized below with reference to the embodiment shown in FIG1 .

FIG1 is a flowchart of a method for obtaining a mapping relationship provided in an embodiment of the present application. The method shown in FIG1 is performed by a communication device, such as a network device, such as a switch, router, or firewall, which has forwarding capabilities. The communication device can also be a terminal, server, or other device with data processing capabilities. The method shown in FIG1 includes the following steps S110 to S140.

Step S110: The communication device obtains target data.

The target data includes a port number field and a background field. The port number field includes the port number of the first port, and the background field includes the attributes of the port number. The target data can come from a forwarding table stored on the communication device. The target data can also come from logs recorded on the communication device, such as traffic forwarding logs, security detection logs, etc. The target data can also be sent to the communication device by another device or input by the user.

In some embodiments, the target data includes static data, which refers to data unrelated to the actual connection usage of the port. For example, the static data includes a port registry, which includes at least one of an IANA port registry and/or a private protocol port registry.

In some implementations, before acquiring the attribute of the port number of the first port based on the target data, the communication device performs redundancy removal processing on the original port registration table to obtain the port registration table.

In some implementations, the redundancy removal method includes at least one of the following method 1 and/or method 2.

Method 1: If it is detected that a background field in the original port registration table includes a keyword indicating duplication, the row where the background field is located is deleted from the original port registration table.

Method 2: If it is detected that there are terms with the same meaning but different expressions in multiple background fields in the original port registration table, a target word for the term is determined, and the term appearing in the multiple background fields is replaced with the target word.

In some embodiments, the target data includes dynamic data, which includes log records of a first communication device, and the log records are used to record data generated by the communication between the first port of the first communication device and the second communication device. The object identifier includes at least one of the network address of the second communication device, the identifier of the area where the second communication device is located, the first port in the second communication device that communicates with the first port, and/or the identifier of an application in the second communication device that communicates with the first port.

Step S120: The communication device obtains the attribute of the port number of the first port based on the target data.

The first port refers to a port recorded in the target data. The first port is, for example, a port included in the communication device. The communication device parses the background field in the target data and extracts the attribute of the port number from the background field.

In some embodiments, the attributes of the port number include usage information of the port number, which includes at least one of an identifier of an application communicating through the port number, an identifier of a protocol communicating through the port number, or an identifier of a service communicating through the port number.

In some embodiments, before executing step S120, the communication device first determines the similarity between the background field in the target data and the port number; compares the similarity with a similarity threshold; and if the similarity exceeds the similarity threshold, executes step S120 to extract the attributes of the port number from the background field. If the similarity is not greater than the similarity threshold, step S120 is not executed.

In some embodiments, before executing step S120, the communication device first determines the hash value of the background field in the target data; compares the hash value with a hash value threshold; and if the hash value exceeds the hash value threshold, executes step S120 to extract the attributes of the port number from the background field. If the hash value does not exceed the hash value threshold, step S120 is not executed.

Step S130: The communication device extracts features from the attributes of the port number to obtain a feature tag set of the port number, where the feature tag set includes one or more feature tags of the port number.

In some implementations, the communication device performs feature extraction on usage information of the port number to obtain a static tag set for the port number, where the static tag set for the port number includes one or more static tags for the port number.

In some implementations, the attributes of the port number further include the number of interactions between the object and the first port. The communication device concatenates or fuses the object identifier and the number of interactions to obtain a feature tag set for the port number.

In some embodiments, the target data includes dynamic data, which refers to data related to the actual connection usage of the first port. The attributes of the port number include an object identifier, which is used to identify an object that has an interactive relationship with the first port. The communication device performs feature extraction on the object identifier to obtain a dynamic tag set for the port number, and the dynamic tag set for the port number includes one or more dynamic tags of the port number.

In some embodiments, the dynamic data also includes a threat record, the background field in the threat record includes an attack type field, the attributes of the port number include the type of network attack targeting the first port and the number of times the first port is attacked by this type of network attack, and the communication device performs feature extraction on the network attack and/or the number of attacks to obtain a dynamic label set for the port number, and the dynamic label set for the port number includes one or more dynamic labels for the port number.

In some embodiments, the communication device performs feature extraction on the attributes of the port number in the static data to obtain a static tag set for the port number; the communication device performs feature extraction on the attributes of the port number in the dynamic data to obtain a dynamic tag set for the port number; the communication device fuses the static tag set of the port number with the static tag set of the port number to obtain a feature tag set for the port number.

In step S140 , the communication device generates a mapping relationship based on the port number and the feature tag set, where the mapping relationship includes a key-value pair, the key in the mapping relationship includes the port number, and the value in the mapping relationship includes the feature tag set.

The method provided in this embodiment utilizes the co-occurrence relationship between the port number and the background field. Instead of directly encoding the port number itself, it obtains the attributes of the port number from the background field related to the port number, and obtains the feature tag set of the port number (or the encoding of the port number) based on the attributes of the port number, so that the feature tag set of the port number can represent the attributes of the port number (or the semantics or purpose of the port number), thereby avoiding the loss of information caused by the inability of the feature tag set of the port number to represent the attributes of the port number. In addition, if two port numbers have common attributes, there will be the same part in the feature tag sets of the two port numbers. In other words, the encodings of the two port numbers are partially the same or similar. Therefore, the similarity between the feature tag sets of the two port numbers can represent the closeness of the association relationship between the two port numbers, thereby more fully representing the correlation between different ports.

In some embodiments, the method further comprises:

Comparing the feature tag set of the first port with the feature tag set of the second port to obtain a similarity between the feature tag set of the first port and the feature tag set of the second port;

Based on the similarity between the feature tag set of the first port and the feature tag set of the second port, the similarity between the first port and the second port is determined.

In some embodiments, the target data includes an alarm log set, the alarm log set includes a first alarm log and a second alarm log, and the method further includes:

Determining the similarity between the first alarm log and the second alarm log based on similarity between a feature tag set of a first port and a feature tag set of a second port, wherein the feature tag set of the first port is determined based on a background field in the first alarm log, and the feature tag set of the second port is determined based on the background field in the second alarm log;

The alarm log set is clustered based on the similarity between the first alarm log and the second alarm log.

In some embodiments, the target data includes network connection information of the first communication device in a normal state, and the method further includes:

When it is detected that the second communication device establishes a connection with the first port of the first communication device, searching the mapping relationship based on the port number of the first port to obtain a feature tag set of the port number;

If the feature of the second communication device matches the feature tag set of the port number, determining that the connection is a normal connection;

If the feature of the second communication device does not match the feature tag set of the port number, it is determined that the connection is an abnormal connection.

In some embodiments, the target data includes a training sample set, each sample in the training sample set includes a source address, a destination address, a source port number, a destination port number, a protocol type, and a type label, and the method further includes:

Searching the mapping relationship based on the source port number to obtain a feature tag set of the source port number;

Searching the mapping relationship based on the destination port number to obtain a feature tag set of the destination port number;

splicing or fusing the feature label set of the source address, the feature label set of the destination address, the feature label set of the source port number, the feature label set of the destination port number, and the feature label set of the protocol type to obtain a feature label set for each sample in the training sample set;

Model training is performed based on the feature label set of each sample in the training sample set and the type label of each sample in the training sample set to obtain a classification model.

FIG2 is a schematic diagram of a mapping relationship acquisition device 200 provided in an embodiment of the present application. The device 200 includes:

An acquiring unit 210 is configured to acquire the properties of the port number of the first port based on target data, wherein the target data includes a port number field and a background field, the port number field includes the port number of the first port, and the background field includes the properties of the port number;

The processing unit 220 is configured to perform feature extraction on the attributes of the port number to obtain a feature tag set of the port number, where the feature tag set includes one or more feature tags of the port number;

The acquisition unit 210 is further configured to acquire a mapping relationship based on the port number and the feature tag set, where the mapping relationship includes a key-value pair, the key in the mapping relationship includes the port number, and the value in the mapping relationship includes the feature tag set.

In some embodiments, the target data includes static data, which refers to data that is unrelated to the actual connection usage of the port. The attributes of the port number include usage information of the port number, which includes at least one of an identifier of an application communicating through the port number, an identifier of a protocol communicating through the port number, or an identifier of a service communicating through the port number. The processing unit 220 is further used to perform feature extraction on the usage information of the port number to obtain a static tag set of the port number, and the static tag set of the port number includes one or more static tags of the port number.

In some implementations, the static data includes a port registry, where the port registry includes at least one of an IANA port registry and/or a private protocol port registry.

In some implementations, the processing unit 220 is further configured to perform redundancy removal on the original port registration table to obtain a port registration table.

In some embodiments, the processing unit 220 is further configured to perform at least one of the following: if it is detected that a background field in the original port registry includes keywords representing duplication, deleting the row containing the background field from the original port registry; if it is detected that multiple background fields in the original port registry contain terms with the same meaning but different expressions, determining a target term for the term, and replacing the term appearing in multiple background fields with the target term.

In some embodiments, the target data includes dynamic data, which refers to data related to the actual connection usage of the first port. The attributes of the port number include an object identifier, which is used to identify an object that has an interactive relationship with the first port. The processing unit 220 is used to extract features from the object identifier to obtain a dynamic tag set of the port number. The dynamic tag set of the port number includes one or more dynamic tags of the port number.

In some embodiments, the dynamic data includes log records of the first communication device, where the log records are used to record data generated by the communication between the first port of the first communication device and the second communication device, and the object identifier includes at least one of the network address of the second communication device, the identifier of the area where the second communication device is located, the identifier of the first port in the second communication device that communicates with the first port, and/or the identifier of the application in the second communication device that communicates with the first port.

In some implementations, the attribute of the port number further includes the number of interactions between the object and the first port. The processing unit 220 is configured to concatenate or fuse the object identifier and the number of interactions to obtain a feature tag set of the port number.

In some embodiments, the dynamic data also includes threat records, the background field in the threat record includes an attack type field, the attributes of the port number include the type of network attack targeting the first port and the number of times the first port is attacked by the type of network attack, and the processing unit 220 is used to perform feature extraction on the network attack and/or the number of attacks to obtain a dynamic label set for the port number, and the dynamic label set for the port number includes one or more dynamic labels for the port number.

In some embodiments, the processing unit 220 is also used to perform feature extraction on the attributes of the port number in the static data to obtain a static tag set of the port number; perform feature extraction on the attributes of the port number in the dynamic data to obtain a dynamic tag set of the port number; and fuse the static tag set of the port number with the static tag set of the port number to obtain a feature tag set of the port number.

In some implementations, the processing unit 220 is further configured to determine the similarity between the background field and the port number in the target data; if the similarity is higher than a similarity threshold, extract the attribute of the port number from the background field.

In some implementations, the processing unit 220 is further configured to determine a hash degree of a background field in the target data; if the hash degree is higher than a hash degree threshold, extract the attribute of the port number from the background field.

In some embodiments, the apparatus further comprises:

a comparing unit, for comparing the feature tag set of the first port with the feature tag set of the second port to obtain a similarity between the feature tag set of the first port and the feature tag set of the second port;

The processing unit 220 is further configured to determine the similarity between the first port and the second port based on the similarity between the feature tag set of the first port and the feature tag set of the second port.

In some embodiments, the target data includes an alarm log set, the alarm log set includes a first alarm log and a second alarm log, and the processing unit 220 is further used to determine the similarity between the first alarm log and the second alarm log based on the similarity between the feature tag set of the first port and the feature tag set of the second port, the feature tag set of the first port is determined based on the background field in the first alarm log, and the feature tag set of the second port is determined based on the background field in the second alarm log; based on the similarity between the first alarm log and the second alarm log, the alarm log set is clustered.

In some embodiments, the target data includes network connection information of the first communication device in a normal state. The processing unit 220 is further used to, when detecting that the second communication device has established a connection with the first port of the first communication device, search for a mapping relationship based on the port number of the first port to obtain a feature tag set of the port number; if the feature of the second communication device hits the feature tag set of the port number, determine that the connection is a normal connection; if the feature of the second communication device does not hit the feature tag set of the port number, determine that the connection is an abnormal connection.

In some embodiments, the target data includes a training sample set, each sample in the training sample set includes a source address, a destination address, a source port number, a destination port number, a protocol type, and a type label, and the apparatus further includes:

A search unit is configured to search for a mapping relationship based on a source port number to obtain a feature tag set of the source port number; and search for a mapping relationship based on a destination port number to obtain a feature tag set of the destination port number;

The processing unit 220 is also used to splice or fuse the feature label set of the source address, the feature label set of the destination address, the feature label set of the source port number, the feature label set of the destination port number, and the feature label set of the protocol type to obtain the feature label set of each sample in the training sample set; and perform model training based on the feature label set of each sample in the training sample set and the type label of each sample in the training sample set to obtain a classification model.

The device embodiment described in FIG2 is merely illustrative. For example, the division of the above units is merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The functional units in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit.

Each unit in the device 200 is implemented entirely or partially by software, hardware, firmware, or any combination thereof.

In conjunction with the communication device 300 described below, some possible implementations of the various functional units in the apparatus 200 using hardware or software are described below.

In the case of software implementation, for example, the acquisition unit 210 and the processing unit 220 are implemented by software functional units generated by at least one processor 301 in FIG. 3 after reading the program code stored in the memory 302 .

In the case of hardware implementation, for example, each of the above units in FIG. 2 is implemented by different hardware in the communication device. For example, processing unit 220 is implemented by a portion of processing resources in at least one processor 301 in FIG. 3 (e.g., one or two cores in a multi-core processor), or is implemented by a programmable device such as a field-programmable gate array (FPGA) or a coprocessor. Acquisition unit 210 is implemented by network interface 303 in FIG. 3 .

FIG3 is a schematic structural diagram of a communication device 300 provided in an embodiment of the present application.

The communication device 300 includes at least one processor 301 , a memory 302 , and at least one network interface 303 .

The processor 301 is, for example, a general-purpose central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), a neural-network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits for implementing the solution of the present application. For example, the processor 301 includes an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD is, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

Memory 302 may be, for example, a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, an optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Optionally, memory 302 exists independently and is connected to processor 301 via internal connection 304. Alternatively, memory 302 and processor 301 may be integrated together.

The network interface 303 uses any transceiver-like device for communicating with other devices or communication networks. The network interface 303 includes, for example, at least one of a wired network interface and a wireless network interface. The wired network interface is, for example, an Ethernet interface. The Ethernet interface is, for example, an optical interface, an electrical interface, or a combination thereof. The wireless network interface is, for example, a wireless local area network (WLAN) interface, a cellular network interface, or a combination thereof.

In some embodiments, processor 301 includes one or more CPUs, such as CPU0 and CPU1 shown in FIG. 3 .

In some embodiments, the communication device 300 may optionally include multiple processors, such as processor 301 and processor 305 shown in FIG3 . Each of these processors may be, for example, a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein may optionally refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

In some embodiments, communication device 300 further includes internal connections 304. Processor 301, memory 302, and at least one network interface 303 are connected via internal connections 304. Internal connections 304 include pathways that transmit information between these components. Optionally, internal connections 304 are boards or buses. Optionally, internal connections 304 are divided into address buses, data buses, control buses, and the like.

In some embodiments, the communication device 300 further includes an input/output interface 306 , which is connected to the internal connection 304 .

Optionally, the processor 301 implements the method in the above embodiment by reading the program code stored in the memory 302, or the processor 301 implements the method in the above embodiment by internally stored program code. In the case where the processor 301 implements the method in the above embodiment by reading the program code stored in the memory 302, the memory 302 stores the program code 310 that implements the method provided in the embodiment of the present application.

The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

A refers to B, which means that A is the same as B or A is a simple variant of B.

The terms "first" and "second" in the description and claims of the embodiments of this application are used to distinguish different objects, not to describe a specific order of objects, and should not be construed as indicating or implying relative importance. For example, the terms "first port" and "second port" are used to distinguish different ports, not to describe a specific order of ports, and should not be construed as implying that the first port is more important than the second port.

In the embodiments of the present application, unless otherwise specified, "at least one" means one or more, and "a plurality" means two or more. For example, a plurality of ports means two or more ports.

The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in accordance with the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. Available media can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)).

The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims (20)

A method for obtaining a mapping relationship, characterized in that the method includes: Acquire attributes of a port number of a first port based on target data, wherein the target data includes a port number field and a background field, the port number field includes the port number of the first port, and the background field includes attributes of the port number; Performing feature extraction on the attributes of the port number to obtain a feature tag set of the port number, wherein the feature tag set includes one or more feature tags of the port number; A mapping relationship is acquired based on the port number and the feature tag set, where the mapping relationship includes a key-value pair, the key in the mapping relationship includes the port number, and the value in the mapping relationship includes the feature tag set. The method according to claim 1 is characterized in that the target data includes static data, the static data refers to data unrelated to the actual connection usage of the port, the attributes of the port number include usage information of the port number, the usage information includes at least one of an identifier of an application communicated through the port number, an identifier of a protocol communicated through the port number, or an identifier of a service communicated through the port number, and the feature extraction of the attributes of the port number to obtain a feature tag set for the port number includes: Feature extraction is performed on the usage information of the port number to obtain a static tag set of the port number, where the static tag set of the port number includes one or more static tags of the port number. The method according to claim 2, characterized in that the static data includes a port registry, and the port registry includes at least one of an Internet Assigned Numbers Authority (IANA) port registry and/or a private protocol port registry. The method according to claim 3, characterized in that before obtaining the attribute of the port number of the first port based on the target data, the method further comprises: The original port registration table is de-redundanted to obtain the port registration table. The method according to claim 4, wherein the de-redundancy processing of the original port registry comprises at least one of the following: If it is detected that a background field in the original port registration table includes a keyword indicating duplication, deleting the row where the background field is located from the original port registration table; If it is detected that there are terms with the same meaning but different expressions in multiple background fields in the original port registration table, a target word for the term is determined, and the terms appearing in the multiple background fields are replaced with the target word. The method according to claim 1, wherein the target data includes dynamic data, the dynamic data refers to data related to the actual connection usage of the first port, the attributes of the port number include an object identifier, the object identifier is used to identify an object that has an interactive relationship with the first port, and the feature extraction of the attributes of the port number to obtain a feature tag set for the port number includes: Feature extraction is performed on the object identifier to obtain a dynamic tag set of the port number, where the dynamic tag set of the port number includes one or more dynamic tags of the port number. The method according to claim 1 is characterized in that the dynamic data includes log records of a first communication device, the log records are used to record data generated by the communication between the first port of the first communication device and the second communication device, and the object identifier includes at least one of the network address of the second communication device, the identifier of the area where the second communication device is located, the first port in the second communication device that communicates with the first port, and/or the identifier of the application in the second communication device that communicates with the first port. The method according to claim 1, wherein the attributes of the port number further include the number of interactions between the object and the first port, and the extracting features of the object identifier to obtain the dynamic tag set of the port number comprises: The object identifier and the number of interactions are concatenated or fused to obtain a feature tag set of the port number. The method according to claim 1, wherein the dynamic data further includes a threat record, the background field in the threat record includes an attack type field, the attributes of the port number include a type of network attack targeting the first port and a number of times the first port has been attacked by the type of network attack, and the feature extraction of the attributes of the port number to obtain a feature tag set for the port number includes: Feature extraction is performed on the network attack and/or the number of attacks to obtain a dynamic label set of the port number, where the dynamic label set of the port number includes one or more dynamic labels of the port number. The method according to claim 1, wherein extracting features from the attributes of the port number to obtain a feature tag set of the port number comprises: Performing feature extraction on the attributes of the port number in the static data to obtain a static tag set of the port number; Performing feature extraction on the attributes of the port number in the dynamic data to obtain a dynamic tag set of the port number; The static label set of the port number is merged with the static label set of the port number to obtain the feature label set of the port number. The method according to claim 1, wherein obtaining the attribute of the port number of the first port based on the target data comprises: Determining the similarity between the background field in the target data and the port number; If the similarity is higher than a similarity threshold, the attribute of the port number is extracted from the background field. The method according to claim 1, wherein obtaining the attribute of the port number of the first port based on the target data comprises: Determining the hash degree of the background field in the target data; If the hash degree is higher than a hash degree threshold, the attribute of the port number is extracted from the background field. The method according to claim 1, further comprising: comparing the feature tag set of the first port with the feature tag set of the second port to obtain a similarity between the feature tag set of the first port and the feature tag set of the second port; Based on the similarity between the feature tag set of the first port and the feature tag set of the second port, the similarity between the first port and the second port is determined. The method according to claim 1, wherein the target data includes an alarm log set, the alarm log set includes a first alarm log and a second alarm log, and the method further comprises: Determining similarity between the first alarm log and the second alarm log based on similarity between a feature tag set of a first port and a feature tag set of a second port, wherein the feature tag set of the first port is determined based on a background field in the first alarm log, and the feature tag set of the second port is determined based on the background field in the second alarm log; The alarm log set is clustered based on the similarity between the first alarm log and the second alarm log. The method according to claim 1, wherein the target data includes network connection information of the first communication device in a normal state, and the method further comprises: When it is detected that a second communication device establishes a connection with the first port of the first communication device, searching the mapping relationship based on the port number of the first port to obtain a feature tag set of the port number; If the feature of the second communication device matches the feature tag set of the port number, determining that the connection is a normal connection; If the feature of the second communication device does not match the feature tag set of the port number, it is determined that the connection is an abnormal connection. The method according to claim 1, wherein the target data comprises a training sample set, each sample in the training sample set comprises a source address, a destination address, a source port number, a destination port number, a protocol type, and a type label, and the method further comprises: Searching the mapping relationship based on the source port number to obtain a feature tag set of the source port number; Searching the mapping relationship based on the destination port number to obtain a feature tag set of the destination port number; splicing or fusing the feature label set of the source address, the feature label set of the destination address, the feature label set of the source port number, the feature label set of the destination port number, and the feature label set of the protocol type to obtain a feature label set for each sample in the training sample set; Model training is performed based on the feature label set of each sample in the training sample set and the type label of each sample in the training sample set to obtain a classification model. A device for obtaining a mapping relationship, characterized in that the device comprises: an acquiring unit, configured to acquire an attribute of a port number of a first port based on target data, wherein the target data includes a port number field and a background field, the port number field includes the port number of the first port, and the background field includes the attribute of the port number; a processing unit, configured to perform feature extraction on the attributes of the port number to obtain a feature tag set of the port number, wherein the feature tag set includes one or more feature tags of the port number; The acquisition unit is further configured to acquire a mapping relationship based on the port number and the feature tag set, wherein the mapping relationship includes a key-value pair, the key in the mapping relationship includes the port number, and the value in the mapping relationship includes the feature tag set. A communication device, characterized in that the communication device includes: a processor, the processor is coupled to a memory, the memory stores at least one computer program instruction, and the at least one computer program instruction is loaded and executed by the processor to enable the communication device to implement the method according to any one of claims 1 to 16. A computer-readable storage medium, characterized in that at least one instruction is stored in the storage medium, and when the instruction is executed on a computer, the computer executes the method according to any one of claims 1 to 16. A computer program product, characterized in that the computer program product comprises one or more computer program instructions, which, when loaded and executed by a computer, enable the computer to execute the method according to any one of claims 1 to 16.