HK1207221B - Method, apparatus and computer readable medium for node de-duplication in network - Google Patents

Description

Method, apparatus, and computer-readable medium for performing node deduplication in a network

Technical Field

Embodiments of the present invention generally relate to network traffic monitoring, analysis, and/or reporting. More specifically, some embodiments are directed to methods, systems, and computer programs for node deduplication, for example, of physical nodes monitored by a network monitoring system.

Background Art

Network management encompasses activities, methods, procedures, and tools related to the operation, administration, maintenance, and/or provisioning of networked systems. Functions that may be performed as part of network management may, for example, include planning, controlling, deploying, allocating, coordinating, and monitoring network resources. Further functions may relate to network planning, frequency allocation, load balancing, configuration management, fault management, security management, performance management, bandwidth management, routing analysis, and billing management.

As mentioned above, a subset of network management includes network monitoring of network traffic. Network administrators are interested in network traffic data for several reasons, including analyzing the impact of new applications on the network, troubleshooting network pain points, detecting slow or faulty network devices, detecting heavy users of bandwidth, and protecting the network. Various protocols have been developed for network traffic flow data. These protocols can include various types of information, such as source Internet Protocol (IP) address, destination IP address, source port, destination port, IP protocol, ingress interface, IP type of service, start and end time, number of bytes, and next hop.

As networks become larger and more complex, systems for monitoring, analyzing, and reporting traffic flow data must become more efficient at handling the increasing number of network devices and the amount of information generated about network traffic.

Summary of the Invention

Certain embodiments are directed to methods, devices, and computer program products for node deduplication. A method includes discovering nodes in a network by a network monitoring device. The method may further include collecting a list of Internet Protocol (IP) addresses, Media Access Control (MAC) addresses, Domain Name System (DNS) names, and system names (sysnames) for each of the nodes discovered in the network, comparing the IP address of each of the discovered nodes with the IP addresses of the current node and other discovered nodes, comparing the MAC address of each of the discovered nodes with the MAC addresses of the current node and other discovered nodes, comparing the DNS name of each of the discovered nodes with the DNS names of the current node and other discovered nodes, comparing the system name of each of the discovered nodes with the system names of the current node and other discovered nodes, and determining duplicate nodes of other discovered nodes and/or the current node based on the comparison of the IP addresses, MAC addresses, DNS names, and system names.

BRIEF DESCRIPTION OF THE DRAWINGS

For a proper understanding of the present invention, reference should be made to the accompanying drawings, in which:

FIG1 illustrates a block diagram of a system according to one embodiment;

FIG2 illustrates a block diagram of a system according to one embodiment;

FIG3 illustrates a flow chart of a method according to one embodiment;

FIG4 illustrates a flow chart of a method according to another embodiment;

FIG5 illustrates a block diagram of an apparatus according to an embodiment; and

FIG6 illustrates a flow chart of a method according to another embodiment.

DETAILED DESCRIPTION

It will be readily understood that the components of the present invention, as generally described and illustrated in the accompanying drawings, may be arranged and designed in a variety of configurations. Accordingly, the following detailed description of embodiments of systems, methods, apparatus, and computer program products for node deduplication, as represented in the accompanying drawings, is not intended to limit the scope of the invention, but merely represents selected embodiments of the invention.

The characteristics, structures or features of the present invention described throughout this specification may be combined in one or more embodiments in any appropriate manner. For example, the use of the phrases "certain embodiments", "some embodiments", etc., refers to the fact that the specific characteristics, structures or features described in the embodiments may be included in at least one embodiment of the present invention throughout this specification. Therefore, the appearance of the phrases "certain embodiments", "in some embodiments", "in other embodiments" or other similar language does not necessarily refer to the same group of embodiments throughout this specification, and the characteristics, structures or features may be combined in one or more embodiments in any appropriate manner. In addition, if desired, the different functions discussed below may be performed in different orders and/or simultaneously with each other. In addition, if desired, one or more of the functions may be optional or may be combined. In this regard, the following description should be considered as merely an illustration of the principles, teachings and embodiments of the present invention, and is not intended to limit them.

It should be noted that throughout this specification, the terms network device and network node, or simply device or node, may be used interchangeably to refer to any physical device capable of connecting and/or communicating on the Internet. Examples of these devices or nodes may include, but are not limited to, routers, switches, servers, computers, laptops, tablets, phones, printers, mobile devices, and any other current or future component capable of sending, receiving, or forwarding information over a communication channel.

FIG1 illustrates an example of a system according to one embodiment. The system includes a network monitoring system 100, a network monitoring system storage 110, a switch device 130, and one or more network devices 120. The network monitoring system storage 110 is capable of storing network monitoring data. The network monitoring system storage 110 can be a database or any other suitable storage device. The network devices 120 can be nodes in the network monitored by the network traffic monitor 100. It should be noted that any number and type of network devices 120 can be supported in the system. Therefore, the embodiment is not limited to the number and type of network devices shown in FIG1. In an embodiment, the network monitoring system storage 110 can store a discovery result database table that stores and organizes information about network devices discovered in the network.

In some cases, network devices can be accessible at multiple Internet Protocol (IP) addresses. For example, some devices can have more than one IP address at a time. Additionally, some nodes are dynamic in that they have IP addresses that change over time. This dynamic IP addressing creates problems for deduplication logic, as it may match an outdated primary IP address from a discovery result with the current IP address of a dynamic node.

Currently, systems often apply logic that equates one IP address to one network node. As a result, nodes with multiple IP addresses bound to them may not be identified as a single node. This behavior can lead to situations where a physical device is monitored more than once, which can significantly increase overhead and inefficiencies in network monitoring systems.

Therefore, users often want to enable devices that respond on multiple IP addresses to be monitored as a single node in the network. Embodiments of the present invention implement automatic network discovery that can detect this situation and avoid processing the same physical node multiple times (e.g., each time using a different IP address). One embodiment includes deduplication logic configured to automatically identify duplicate nodes so that a single node is not monitored multiple times.

Certain embodiments identify one or more pieces of information that serve as a node identifier required to uniquely identify a network node. With this unique node identifier, certain embodiments can then define logic that detects duplication among nodes found during the discovery process (e.g., the same physical network node under different IP addresses) and/or nodes that are already being monitored under different IP addresses.

In accordance with certain embodiments, the information used to uniquely identify a node includes information that can be easily polled, is available to most devices (i.e., vendor independent), is available to Internet Control Message Protocol (ICMP) nodes and other types of nodes (e.g., ICMP nodes can be considered nodes that are not reachable via SNMP or WMI), and is a minimal set of information that is efficient while still providing accurate results.

Certain embodiments can address at least two typical use cases. For example, one use case may include running automatic network discovery on a subnet that includes devices accessible at multiple IP addresses. As is well known, devices belonging to a subnet are addressed by the common, identical, most significant byte of their IP addresses. Another use case may include running automatic network discovery on a subnet that is already being monitored.

Certain embodiments are capable of identifying network nodes that are network duplicates even if one set of information was collected within a different time frame than another set of information. This may occur, for example, when a user decides to import discovery results for a periodic discovery profile. In this case, some embodiments may need to use information collected during discovery that may be out of date and compare it to new information that is being continuously collected for all monitored nodes. Thus, deduplication according to certain embodiments may occur during a discovery effort, where discovered nodes are compared to each other so that duplicates are removed, and/or may occur during a discovery results import, where discovered nodes are imported and compared to existing nodes so that duplicate nodes are identified and removed.

According to one embodiment, a data collection set consisting of the DNS name, system name, IP address, and MAC address for each network node is used to help identify duplicate nodes. One embodiment includes logic for comparing multiple pieces of information from the information collection set to each other (e.g., DNS with DNS, MACs with MACs, etc.). The logic can be implemented in a duplicate detector component. The result from the detection is a match index that indicates whether there is a match, no match, or unknown. Certain embodiments also provide logic for aggregating partial results from the detector and calculating a final conclusion as to whether the node is a duplicate. For example, in an embodiment, node deduplication can include several sub-iterations, each of which is responsible for discovering a selected IP range. At the end of each iteration, deduplication is performed to immediately omit the node (e.g., an endpoint) if it is considered to be a duplicate node. In the case of an unknown or non-duplicate result for an iteration, the node is passed to the next step or iteration for processing.

In an embodiment, two sets of duplicate detectors may be provided. One set of detectors may be used during the automatic discovery process to filter out newly discovered devices and remove duplicate nodes. The other set of detectors may be used during the import of discovery results to prevent duplicate nodes from being added to the monitored node group.

In one embodiment, the system is configured to collect a list of all DNS names, system names, IP addresses, and MAC addresses for all discovered nodes and store them, for example, as part of the results of a discovery effort. This information can be stored in the network system monitoring storage or database 110. As mentioned above, this information can be used during at least two stages of discovery. For example, the DNS name, system name, IP address, and MAC address can be used when running a discovery to check whether the currently discovered node is a duplicate of any other found node, and/or they can be used during the import of discovery results to compare the discovery results with existing nodes monitored by the system. In one embodiment, the MAC information is stored to a permanent storage, for example, as part of the discovery results.

FIG2 illustrates a system 200 according to one embodiment, which may include an IP address duplication detector 201, a DNS duplication detector 203, a MAC address duplication detector 202, and a system name duplication detector 204. In an embodiment, a database 210 may communicate with the IP address duplication detector 201, the DNS duplication detector 203, the MAC address duplication detector 202, and the system name duplication detector 204 so that each of the detectors can collect relevant addresses stored in the database 210. In an embodiment, the database 210 may be stored in the network traffic data storage 110 shown in FIG1. It should be noted that the system 200 may be implemented entirely in hardware, or in a combination of hardware and software.

Each of the duplicate detectors 201, 202, 203, 204 can have a priority for a defined order of execution (e.g., a lower number indicates an earlier execution), a weight indicating the reliability of the results provided by the duplicate detector (e.g., a weight of 0 will have no effect on the final result), and a veto power that is used as the highest priority to determine whether a node is a duplicate.

The DNS duplicate detector 203 can be configured to compare the DNS of the discovered node with all other discovered nodes and the current node being monitored. In an embodiment, the DNS duplicate detector 203 can conclude that a node is a duplicate if the DNS of the discovered node is the same as the DNS used by any monitored node or any other discovered node.

MAC addresses are generally designed to be unique (although there are situations where the same MAC address is used on two different devices, such as cloned virtual machines hosted in two separate virtual hosts). MAC address duplicate detector 202 can be configured to compare the MAC address of a discovered node with all previously collected MAC addresses of nodes, which can be stored in a node MAC address database table of database 210. MAC address duplicate detector 202 can be configured to conclude that a node is a duplicate node if a set of discovered MAC addresses is a subset of the currently monitored MAC addresses for the node or if a set of monitored MAC addresses is a subset of the discovered MAC addresses.

Table 1 below shows an example where nodes are considered equal based on MAC address duplication detector 202, where A, B, C... represent MAC addresses. Meanwhile, Table 2 below shows an example where nodes are not considered equal (eg, based on MAC addresses).

Node A Node B Node C Node D A A A A B B B B C C C D

Table 1

Node A Node B Node C Node D A A D A B B E D C D F

Table 2

Two nodes that both have exactly two MAC addresses, such as 0000000000000000 and 000000000000000E0, should be considered equal. For example, node A and node B are equal if and only if node A's MAC address list is a subset of node B's MAC address list or node B's MAC list is a subset of node A's MAC list. In this case, the system can rely on a different duplicate detector or a different deduplication method (e.g., system name matching) because the two nodes are equal based on the MAC address.

The system name duplication detector 204 can be configured to compare the system name of the node discovered with the nodes of all other discoveries and the current node being monitored. In an embodiment, the system name duplication detector 204 can conclude that if the system name of the node discovered is identical to the system name used by any monitored node or any other discovered node, then the node is a duplicate node. In an embodiment, the system name can, for example, be the system name for a Simple Network Management Protocol (SNMP) node or can be the full computer name of a Window Management Interface (WMI) node. It should be noted that the embodiment limits the data source to SNMP and/or WMI, and according to certain embodiments (e.g., CLI on a router/switch or remote login via SSH) other types of data sources are equally applicable.

As mentioned above, each duplicate detector can have an associated weight loaded from a settings database form. The weight represents the reliability of the results provided by the associated duplicate detector. It is possible to set the weight to -1 to disable the associated duplicate detector. Depending on the embodiment, the weight value can range from 0 to 100, where 0 represents the least reliable and 100 represents the most reliable.

As discussed above, all duplicate detectors (d ₁ , ..., d _n ) can be executed sequentially in the order defined by the priority. Each duplicate detector can set the 'IsAuthoritative' flag to true, which can then terminate the execution of subsequent duplicate detectors. In this case, the vote for the duplicate detector with the 'IsAuthoritative' flag set to true is considered final, and all other votes are ignored. According to one embodiment, if no 'IsAuthoritative' flag is set to true, the final result as to whether a node is a duplicate node is calculated as the sum of the values of all duplicate detector votes, as follows:

Final decision

d ₁ .IsDuplicate()*d ₁ .Priority+…+d _n .IsDuplicate()*d _n .Priority,

Where _d1 refers to the first duplicate detector, _d2 refers to the second duplicate detector, ... and _dn refers to the nth duplicate detector. Therefore, _{dn.IsDuplicate} () is a function representing the conclusion of the nth duplicate detector on whether a node is a duplicate node.

As shown in Table 3 below, each duplicate detector can return a list of node IDs for all duplicate nodes it finds. Each node ID can be assigned an associated match index (MatchIndex) that indicates the likelihood of a match. In an embodiment, the match index value ranges from 0 to 100, where 0 represents the minimum likelihood of a match and 100 represents the maximum likelihood of a match. According to an embodiment, the system 200 can be configured to group the duplicate node information described in Table 3 by node ID and sum the match indices for the same node ID. The system 200 can then be configured to select the node ID with the highest aggregate total match index for discarding.

Duplicate Detector Duplicate Node ID Matching Index DNS 5 90 Mac 1 60 Mac 5 80 System Name 5 85 Final Decision: 5 90

Table 3

Table 4 below shows an example result table according to one embodiment. In this example, each row of the table can represent a node. The 'DnsDuplicateDetector' column shows the conclusion of the DNS duplicate detector on whether the node is a duplicate node. Similarly, the 'MacAddressDuplicateDetector' column shows the conclusion of the MAC address duplicate detector on whether the node is a duplicate node, and the 'NameDuplicateDetector' column shows the conclusion of the system name duplicate detector on whether the node is a duplicate node. Subsequently, the final 'ExpectedResult' column shows the expected result for the node.

Table 4

Figure 3 illustrates a flow chart of a method for node deduplication according to one embodiment. In the example of Figure 3, at 300, one or more nodes are discovered. At 310, the associated IP address of the node is discovered. At 320, supported technologies for obtaining device information, such as SNMP, WMI, etc., are detected. At 330, information about the node is obtained via the (detected) supported technology. At 340, deduplication of the discovered nodes is performed. At 350, it is determined whether the node is a duplicate. If so, at 360, the duplicate node is omitted from the discovered result group. If the node is not a duplicate node, at 370, the discovered node is saved to a database. At 380, data associated with the discovered node is prepared for discovery import.

FIG4 illustrates a flow chart of a method for node deduplication according to another embodiment. In the example of FIG4 , at 400 , the results of the discovery process are saved. At 405 , the discovery results are loaded from the database and deserialized. At 410 , duplicate detection is primarily performed using the primary IP address associated with the node. At 415 , a check is performed to see if the node with the same IP address is already monitored by any engine. If the node with the same IP address is already monitored, then at 420 , the existing node information is updated. If the node with the same IP address is not already monitored, then at 425 , deduplication detection is performed on the ignored nodes. At 430 , a determination is made as to whether the node is a duplicate. If so, at 435 , the condition is recorded and the duplicate node is discarded. If the node is not determined to be a duplicate, then at 440 , a duplicate check is performed on all monitored nodes. At 445 , a determination is made as to whether the node is a duplicate. If the node is determined to be a duplicate, then at 450 , the condition is recorded and the duplicate node is discarded. If the node is not determined to be a duplicate, then at 455 , the node information is saved to permanent storage. At 460, the result of the discovery import is a list of nodes along with associated IP addresses and MAC addresses.

FIG5 illustrates a block diagram of an apparatus 10 in which one embodiment of the present invention may be implemented. The apparatus 10 may include a bus 12 or other communication mechanism for conveying information between components of the apparatus 10. The apparatus 10 also includes a processor 22 coupled to the bus 12 for processing information and executing instructions or operations. The processor 22 may be any type of general-purpose or special-purpose processor. The apparatus 10 further includes a memory 14 for storing information and instructions for execution by the processor 22. The memory 14 may be comprised of any combination of random access memory ("RAM"), read-only memory ("ROM"), static storage (e.g., a magnetic disk or optical disk), or any other type of machine- or computer-readable medium. The apparatus 10 further includes a communication device 20, such as a network interface card or other communication interface, to provide access to a network. As a result, a user may interface with the apparatus 10 directly or remotely via a network or any other method.

Computer-readable media can be any available media that can be accessed by processor 22 and includes both volatile and nonvolatile media, removable and non-removable media, and communication media. Communication media may include computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

Processor 22 is further coupled to a presentation device 24, such as a display, monitor, screen, or web browser, via bus 12 for displaying information, such as network traffic information, to a user. User input components 25, such as a keyboard, computer mouse, or web browser, are further coupled to bus 12 to enable a user to interface with device 10. Processor 22 and memory 14 may also be coupled to a database system 30 via bus 12, thereby enabling access to and retrieval of information stored in database system 30. In one embodiment, database system 30 is network monitoring system storage 110 shown in FIG. Although only a single database is shown in FIG. 5 , any number of databases may be used according to certain embodiments.

In one embodiment, the memory 14 stores software modules that provide functionality when executed by the processor 22. The modules may include an operating system 15 that provides operating system functionality for the device 10. As discussed above, the memory may also store one or more duplicate detectors 16 that support node deduplication functionality. The one or more duplicate detectors 16 may include, for example, an IP address duplicate detector 201, a DNS duplicate detector 203, a MAC address duplicate detector 202, and a system name duplicate detector 204, as discussed above and depicted in FIG2 . The device 10 may also include one or more other functional modules 18 to provide additional functionality.

The database system 30 may include a database server and any type of database, such as a relational database or a flat file database. The database system 30 may store data about network traffic flows for each entity in the network and/or any data associated with the device 10 or its associated modules and components.

In some embodiments, the processor 22, the duplicate detector 16, and the other functional modules 18 may be implemented as separate physical and logical units or may be implemented in a single physical and logical unit. In addition, in some embodiments, the processor 22, the duplicate detector 16, and the other functional modules 18 may be implemented in hardware or as any suitable combination of hardware and software.

In some embodiments, the processor 22 is configured to control the apparatus 10 to discover nodes in the network. According to an embodiment, information identifying the discovered nodes may be stored, for example, in the database 110. The processor 22 may be configured to control the apparatus 10 to collect a list of IP addresses, MAC addresses, DNS names, and system names for each of the nodes discovered in the network.

According to one embodiment, the processor 22 may be configured to control the device 10 to execute an IP duplication detector configured to compare the IP address of each discovered node with the IP addresses of the current node and other discovered nodes, a MAC duplication detector configured to compare the MAC address of each discovered node with the MAC addresses of the current node and other discovered nodes, a DNS duplication detector configured to compare the DNS name of each discovered node with the DNS names of the current node and other discovered nodes, and a name duplication detector configured to compare the system name of each discovered node with the system names of the current node and other discovered nodes. The processor 22 may then be configured to determine whether there are duplicate nodes of other discovered nodes and/or the current node based on the comparisons performed by the IP duplication detector, the MAC duplication detector, the DNS duplication detector, and the name duplication detector.

In an embodiment, the processor 22 may be configured to control the apparatus 10 to discard duplicate nodes. According to one embodiment, the processor 22 may be configured to control the apparatus 10 to assign a priority to each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector to determine the order of operation. The apparatus 10 may be controlled to determine duplicate nodes, for example, by executing the following formula:

d ₁ .IsDuplicate()*d ₁ .Priority+…+d _n .IsDuplicate()*d _n .Priority.

In an embodiment, each of the discovered nodes may be assigned a node ID. The processor 22 may be configured to control the device 10 to assign a matching index to each node ID, wherein the matching index indicates the likelihood of a match between the discovered node and any of the current node and other discovered nodes. According to one embodiment, the processor 22 may be configured to control the device 10 to group duplicate nodes by node ID and sum the matching indices for the same node ID. In addition, a weight is assigned to each of the IP duplicate detector, MAC duplicate detector, DNS duplicate detector, and name duplicate detector. The weight indicates the reliability of the result provided by the corresponding duplicate detector.

Figure 6 illustrates an example flow chart of a method according to one embodiment. The method includes, at 600, discovering nodes in a network, for example, by a network monitoring device. The method may then include, at 610, collecting a list of Internet Protocol (IP) addresses, Media Access Control (MAC) addresses, Domain Name System (DNS) names, and system names for each of the nodes discovered in the network. At 620, the method includes comparing the IP address of each of the discovered nodes with the IP addresses of the current node and other discovered nodes. At 630, the method includes comparing the MAC address of each of the discovered nodes with the MAC addresses of the current node and other discovered nodes. At 640, the method includes comparing the DNS name of each of the discovered nodes with the DNS names of the current node and other discovered nodes. At 650, the method includes comparing the system name of each of the discovered nodes with the system names of the current node and other discovered nodes. The method may further include, at 660, determining duplicates of other discovered nodes and/or the current node based on the comparison of the IP addresses, MAC addresses, DNS names, and system names.

In some embodiments, the functions of any method described herein, such as those in Figures 3, 4, and 6, may be implemented by software and/or computer program code stored in a memory or other computer-readable or tangible medium and executed by a processor. In other embodiments, the functions may be implemented by hardware, such as by using an application-specific integrated circuit (ASIC), a programmable gate array (PGA), a field programmable gate array (FPGA), or any other combination of hardware and software.

Those skilled in the art will readily appreciate that the invention discussed above can be practiced by steps in a different order and/or using hardware components in a configuration different from that disclosed. Thus, while the invention has been described based on these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative constructions are apparent, while remaining within the spirit and scope of the invention. To determine the boundaries and limits of the invention, reference should be made to the appended claims.

Claims (11)

1. A method for deduplicating nodes in a network, comprising: Nodes in the network are detected by network monitoring devices; For each node discovered in the network, a list of Internet Protocol (IP) addresses, Media Access Control (MAC) addresses, Domain Name System (DNS) names, and system names is collected. The IP duplicate detector compares the IP address of each of the discovered nodes with the IP addresses of the current node and other discovered nodes. The MAC address of each of the discovered nodes is compared with the MAC addresses of the current node and the other discovered nodes by the MAC duplicate detector. The DNS name of each of the discovered nodes is compared by the DNS duplicate detector with the DNS names of the current node and the other discovered nodes; The system name of each of the discovered nodes is compared by the name duplication detector with the system names of the current node and the other discovered nodes; The comparison of the IP address, MAC address, DNS name, and system name is used to determine the duplicate nodes of the other discovered nodes and/or the current node; A priority is assigned to each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector, and the priority determines the execution order of each comparison step in the comparison process; and A weight is assigned to each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector, wherein the weight indicates the reliability of the results provided by each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector. 2. The method of claim 1, further comprising discarding the duplicate node. 3. The method of claim 1, wherein determining the duplicate nodes of the other discovered nodes and/or the current node comprises executing the following formula: d ₁ .IsDuplicate()*d ₁ .Priority+…+d _n .IsDuplicate()*d _n .Priority， Where _d1 refers to the first duplicate detector, ..., and _dn refers to the nth duplicate detector, _{dn.IsDuplicate} () is a function representing the conclusion of the nth duplicate detector regarding whether a node is a duplicate node, _dn.Priority indicates the priority of the nth duplicate detector, and the formula is executed if the IsAuthoritative flag for _d1 , ..., _dn is not set to true, and the IsAuthoritative flag being set to true indicates that the vote for the duplicate detector is considered final. 4. The method of claim 1, wherein each of the discovered nodes is assigned a node ID, and wherein the method further comprises assigning a matching index to each node ID, the matching index indicating the probability of a match between the discovered node and any of the current node and the other discovered nodes. 5. The method according to claim 4, further comprising grouping the duplicate nodes by node ID and summing the matching index for the same node ID. 6. An apparatus for deduplicating nodes in a network, comprising: At least one processor and at least one memory containing computer program code, The at least one memory and the computer program code are configured together with the at least one processor such that the device at least: Discover nodes in the network; For each node discovered in the network, a list of Internet Protocol (IP) addresses, Media Access Control (MAC) addresses, Domain Name System (DNS) names, and system names is collected. The at least one processor is further configured to perform: An IP duplicate detector is configured to compare the IP address of each of the discovered nodes with the IP addresses of the current node and other discovered nodes. A MAC duplicate detector is configured to compare the MAC address of each of the discovered nodes with the MAC addresses of the current node and the other discovered nodes; A DNS duplication detector is configured to compare the DNS name of each of the discovered nodes with the DNS names of the current node and the other discovered nodes; A name duplication detector is configured to compare the system name of each of the discovered nodes with the system names of the current node and the other discovered nodes; The at least one memory and the computer program code are further configured, together with the at least one processor, such that the device at least: The duplicate nodes that are duplicates of the other discovered nodes and/or the current node are determined based on the comparison results of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector. A priority is assigned to each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector, and the priority determines the execution order of each comparison step in the comparison process; and A weight is assigned to each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector, wherein the weight indicates the reliability of the results provided by each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector. 7. The apparatus of claim 6, wherein the at least one memory and the computer program code are further configured, together with the at least one processor, such that the apparatus at least discards the duplicate node. 8. The apparatus of claim 6, wherein the at least one memory and the computer program code are further configured, together with the at least one processor, such that the apparatus determines the repeating node by at least executing the following formula: d ₁ .IsDuplicate()*d ₁ .Priority+…+d _n .IsDuplicate()*d _n .Priority， Where _d1 refers to the first duplicate detector, ..., and _dn refers to the nth duplicate detector, _{dn.IsDuplicate} () is a function representing the conclusion of the nth duplicate detector regarding whether a node is a duplicate node, _dn.Priority indicates the priority of the nth duplicate detector, and the formula is executed if the IsAuthoritative flag for _d1 , ..., _dn is not set to true, and the IsAuthoritative flag being set to true indicates that the vote for the duplicate detector is considered final. 9. The apparatus of claim 6, wherein each of the discovered nodes is assigned a node ID, and wherein the at least one memory and the computer program code are further configured, together with the at least one processor, such that the apparatus assigns at least a matching index to each node ID, the matching index indicating the probability of a match between the discovered node and any of the current node and the other discovered nodes. 10. The apparatus of claim 9, wherein the at least one memory and the computer program code are further configured, together with the at least one processor, such that the apparatus groups the duplicate nodes by node ID and sums the matching index for the same node ID. 11. A computer-readable medium storing a computer program, wherein the computer program is configured to control a processor to perform a process, the process comprising: Nodes in the network are detected by network monitoring devices; For each node found in the network, a list of Internet Protocol (IP) addresses, Media Access Control (MAC) addresses, Domain Name System (DNS) names, and system names is collected. The IP duplicate detector compares the IP address of each of the discovered nodes with the IP addresses of the current node and other discovered nodes. The MAC address of each of the discovered nodes is compared with the MAC addresses of the current node and the other discovered nodes by the MAC duplicate detector. The DNS name of each of the discovered nodes is compared by the DNS duplicate detector with the DNS names of the current node and the other discovered nodes; The system name of each of the discovered nodes is compared by the name duplication detector with the system names of the current node and the other discovered nodes; The comparison of the IP address, MAC address, DNS name, and system name is used to determine the duplicate nodes of the other discovered nodes and/or the current node; A priority is assigned to each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector, and the priority determines the execution order of each comparison step in the comparison process; and A weight is assigned to each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector, wherein the weight indicates the reliability of the results provided by each of the IP duplicate detector, the MAC duplicate detector, the DNS duplicate detector, and the name duplicate detector.