CN119629252A - Request scheduling method, device, electronic device and storage medium in cluster system - Google Patents
Request scheduling method, device, electronic device and storage medium in cluster system Download PDFInfo
- Publication number
- CN119629252A CN119629252A CN202411855512.6A CN202411855512A CN119629252A CN 119629252 A CN119629252 A CN 119629252A CN 202411855512 A CN202411855512 A CN 202411855512A CN 119629252 A CN119629252 A CN 119629252A
- Authority
- CN
- China
- Prior art keywords
- node
- serviceable
- scheduling
- request
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/63—Routing a service request depending on the request content or context
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a request scheduling method, a device, electronic equipment and a computer readable storage medium in a cluster system, wherein the cluster system comprises a main service node and a plurality of storage nodes, each storage node is connected with the main service node, the method is applied to the main service node, the method comprises the steps of obtaining the request connection number of each storage node in a preset scheduling period, calculating the total number of the request connection of the cluster system, determining a scheduling threshold value in the preset scheduling period according to the total number of the request connection and the total number of the storage nodes in the cluster system, adding the storage nodes meeting the scheduling threshold value into a serviceable resource pool, adding the storage nodes not meeting the scheduling threshold value into an unserviceable resource pool, selecting a target storage node in the serviceable resource pool for a new connection request in the preset scheduling period, and scheduling the new connection request to the target storage node. The scheme can perform more balanced and reasonable scheduling on the client connection request so as to effectively ensure the stability of the cluster system.
Description
Technical Field
The present application relates to the field of cluster service technologies, and in particular, to a request scheduling method in a cluster system, and also to a request scheduling device, an electronic device, and a computer readable storage medium in the cluster system.
Background
The distributed storage system adopts a distributed and fully symmetrical architecture, integrates scattered storage resources, and provides a unified naming space for users, so that the users can access the system through any node and process client connection requests. In order to balance the connection requests of the distribution clients and ensure the service reliability, the storage system provides various load balancing strategies for users to select autonomously, such as a polling strategy, a connection number strategy and the like, and the system connects multiple clients to each storage node according to the selected strategies, so that the service and the hard disk pressure of different storage nodes are relatively balanced. However, in the actual use process, the load balancing effect may be undesirable due to various reasons, such as node downtime, load balancing defects, policy mismatch, and the like.
Therefore, how to perform more balanced and reasonable scheduling on the client connection request to effectively ensure the stability of the cluster system is a problem to be solved by those skilled in the art.
Disclosure of Invention
The application aims to provide a request scheduling method in a cluster system, which can perform more balanced and reasonable scheduling on client connection requests so as to effectively ensure the stability of the cluster system, and the application also aims to provide a request scheduling device, electronic equipment, a computer readable storage medium and a computer program product in the cluster system, which have the beneficial effects.
In a first aspect, the present application provides a request scheduling method in a cluster system, where the cluster system includes a main service node and a plurality of storage nodes, each storage node is connected to the main service node, and the method is applied to the main service node, and includes:
acquiring the request connection number of each storage node in a preset scheduling period, and calculating the total number of request connections of the cluster system;
determining a scheduling threshold value in the preset scheduling period according to the total number of the request connections and the total number of the nodes of the storage nodes in the cluster system;
Adding the storage nodes meeting the scheduling threshold into a serviceable resource pool, and adding the storage nodes not meeting the scheduling threshold into an unserviceable resource pool;
And selecting a target storage node from the serviceable resource pool for a new connection request in the preset scheduling period, and scheduling the new connection request to the target storage node.
Optionally, the cluster system further includes a domain name server, the main service node is connected to the domain name server, and the method further includes:
and uploading the node information and the request connection number of each serviceable node in the serviceable resource pool to the domain name server so that the domain name server responds to the domain name mounting request initiated by the client by referring to the node information and the connection request number of each serviceable node.
Optionally, the request scheduling method in the cluster system further includes:
performing fault detection on each storage node in the cluster system, and determining a fault storage node;
and selecting a target serviceable node from the serviceable resource pool, and scheduling all connection requests on the fault storage node to the target serviceable node.
Optionally, selecting a target serviceable node in the serviceable resource pool, and scheduling all connection requests on the failed storage node to the target serviceable node, including:
determining an actual number of requested connections on the failed storage node;
for each serviceable node in the serviceable resource pool, calculating a difference between the number of requested connections on the serviceable node and the scheduling threshold;
arranging the serviceable nodes in the serviceable resource pool according to the order of the difference from large to small to obtain a serviceable node sequence;
Selecting a previous one or more serviceable nodes in the sequence of serviceable nodes as the target serviceable node according to the actual request connection number;
Scheduling all of the connection requests on the failed storage node to the target serviceable node;
correspondingly, for a new connection request in the preset scheduling period, selecting a target storage node in the serviceable resource pool, and scheduling the new connection request to the target storage node, including:
Selecting a serviceable node with the minimum request connection number from the serviceable resource pool as the target storage node for the new connection request in the preset scheduling period;
And dispatching the new connection request to the target storage node.
Optionally, performing fault detection on each storage node in the cluster system, after determining a faulty storage node, further includes:
Recording node information of the fault storage node to a fault node list;
Performing fault detection on each fault storage node in the fault node list according to a preset time interval, and determining a fault recovery node;
And adding the fault recovery node into the serviceable resource pool.
Optionally, adding a storage node that meets the scheduling threshold to a serviceable resource pool, and adding a storage node that does not meet the scheduling threshold to an unserviceable resource pool, including:
Adding the storage nodes with the request connection numbers lower than the scheduling threshold to the serviceable resource pool;
And adding the storage nodes with the request connection numbers not lower than the scheduling threshold value into the non-serviceable resource pool.
Optionally, the request scheduling method in the cluster system further includes:
Counting the number of real-time connection requests of each non-serviceable node in the non-serviceable resource pool in the preset scheduling period;
and rejecting the non-serviceable nodes with the real-time connection request number lower than the scheduling threshold value from the non-serviceable resource pool, and adding the non-serviceable nodes into the serviceable resource pool.
In a second aspect, the present application also discloses a request scheduling device in a cluster system, where the cluster system includes a main service node and a plurality of storage nodes, each storage node is connected to the main service node, and the device is applied to the main service node, and includes:
the acquisition module is used for acquiring the request connection number of each storage node in a preset scheduling period and calculating the total number of the request connections of the cluster system;
The determining module is used for determining a scheduling threshold value in the preset scheduling period according to the total number of the request connections and the total number of the nodes of the storage nodes in the cluster system;
The setting module is used for adding the storage nodes meeting the scheduling threshold into a serviceable resource pool and adding the storage nodes not meeting the scheduling threshold into an unserviceable resource pool;
And the scheduling module is used for selecting a target storage node from the serviceable resource pool for the new connection request in the preset scheduling period, and scheduling the new connection request to the target storage node.
In a third aspect, the present application also discloses an electronic device, including:
A memory for storing a computer program;
a processor for implementing the steps of the request scheduling method in any of the clustered systems described above when executing the computer program.
In a fourth aspect, the present application also discloses a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of a request scheduling method in any cluster system as described above.
In a fifth aspect, the invention also discloses a computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of a request scheduling method in any of the clustered systems described above.
The application provides a request scheduling method in a cluster system, which comprises a main service node and a plurality of storage nodes, wherein each storage node is connected with the main service node, the method is applied to the main service node, and comprises the steps of obtaining the request connection number of each storage node in a preset scheduling period, calculating the total request connection number of the cluster system, determining a scheduling threshold value in the preset scheduling period according to the total request connection number and the total node number of the storage nodes in the cluster system, adding the storage nodes meeting the scheduling threshold value into a serviceable resource pool, adding the storage nodes not meeting the scheduling threshold value into an unserviceable resource pool, selecting a target storage node in the serviceable resource pool for a new connection request in the preset scheduling period, and scheduling the new connection request to the target storage node.
By using the technical scheme provided by the application, the main service node in the cluster system is utilized to detect the number of the request connection of each storage node in the cluster system according to the scheduling period, so that the total number of the request connection of the cluster system in the current scheduling period is counted, the scheduling threshold value in the current scheduling period can be determined by combining the total number of the storage nodes in the cluster system, and therefore, the client connection requests received in the current scheduling period can be scheduled by referring to the scheduling threshold value, namely, all the storage nodes in the cluster system are divided into serviceable nodes and non-serviceable nodes based on the scheduling threshold value in the current scheduling period, so that the serviceable nodes can be conveniently selected to provide services for new client connection requests. Therefore, the technical scheme can perform more balanced and reasonable scheduling on the client connection request so as to effectively ensure the stability of the cluster system.
In one embodiment of the application, the cluster system can further realize domain name mounting processing, the cluster system is further provided with a domain name server which is in communication connection with a main service node, the main service node can synchronously upload node information and request connection numbers of all the serviceable nodes in the current dispatching period to the domain name server, so that when the domain name server receives a domain name mounting request initiated by a client, the mountable node can be directly selected to respond to the domain name mounting request by referring to the node information and the request connection numbers of all the service nodes, and detection and screening of the mountable node in the cluster system are not needed again, thereby effectively improving the domain name mounting processing efficiency.
The request scheduling device, the electronic device, the computer readable storage medium and the computer program product in the cluster system provided by the application have the same technical effects as described above, and the application is not repeated here.
Drawings
In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the following will briefly describe the drawings that need to be used in the description of the prior art and the embodiments of the present application. Of course, the following drawings related to embodiments of the present application are only a part of embodiments of the present application, and it will be obvious to those skilled in the art that other drawings can be obtained from the provided drawings without any inventive effort, and the obtained other drawings also fall within the scope of the present application.
Fig. 1 is a schematic structural diagram of a cluster system according to the present application;
FIG. 2 is a flow chart of a request scheduling method in a cluster system according to the present application;
FIG. 3 is a schematic diagram of a request dispatcher in a cluster system according to the present application;
Fig. 4 is a schematic structural diagram of an electronic device according to the present application.
Detailed Description
The core of the application is to provide a request scheduling method in a cluster system, which can perform more balanced and reasonable scheduling on client connection requests so as to effectively ensure the stability of the cluster system, and the other core of the application is to provide a request scheduling device, electronic equipment, a computer readable storage medium and a computer program product in the cluster system, which all have the beneficial effects.
In order to more clearly and completely describe the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The embodiment of the application provides a request scheduling method in a cluster system.
First, referring to fig. 1, fig. 1 is a schematic structural diagram of a cluster system provided by the present application, where the cluster system includes a main service node and a plurality of storage nodes, and each storage node is connected to the main service node.
Further, referring to fig. 2, fig. 2 is a flow chart of a request scheduling method in a cluster system according to the present application, where the request scheduling method in the cluster system is applied to the primary service node shown in fig. 1, and may include the following steps S101 to S104.
S101, acquiring the request connection number of each storage node in a preset scheduling period, and calculating the total number of the request connections of the cluster system.
The step aims at realizing statistics of the number of the requested connections on each storage node in the cluster system and the total number of the requested connections in the cluster system. It should be noted that, the request scheduling method provided by the embodiment of the application can be executed according to a scheduling period, and S101-104 is executed in a circulating way every time when one scheduling period is started, so as to effectively ensure the uniformity of request scheduling, and of course, the specific duration of the scheduling period does not affect the implementation of the technical scheme, for example, the specific duration can be one day or one week. Specifically, at the beginning of a scheduling period, the number of requested connections on each storage node in the cluster system may be sequentially obtained, and the total number of requested connections in the cluster system may be obtained through addition and calculation.
S102, determining a scheduling threshold value in a preset scheduling period according to the total number of the requested connections and the total number of the nodes of the storage nodes in the cluster system.
This step aims at achieving the calculation of the scheduling threshold in the current scheduling period. In a specific implementation process, a quotient between the total number of requested connections and the total number of storage nodes in the storage system can be directly used as a scheduling threshold value in a current scheduling period. Of course, the determination manner of the scheduling threshold is not unique, for example, when the service capacities of the storage nodes are different, weight calculation based on the actual service capacity of the storage nodes can be added to obtain the scheduling threshold in the current scheduling period, so that the accuracy of the request scheduling result can be further improved.
And S103, adding the storage nodes meeting the scheduling threshold into the serviceable resource pool, and adding the storage nodes not meeting the scheduling threshold into the non-serviceable resource pool.
The method aims at realizing storage node division based on a scheduling threshold, namely adding the storage nodes meeting the scheduling threshold into a serviceable resource pool, and adding the storage nodes not meeting the scheduling threshold into an unserviceable resource pool. It can be understood that the storage nodes in the serviceable resource pool are all serviceable nodes in the scheduling period (i.e. can provide the connection request service), and the storage nodes in the non-serviceable resource pool are all non-serviceable nodes in the scheduling period (i.e. can not provide the connection request service).
In one embodiment of the present application, adding a storage node that meets the scheduling threshold to a pool of serviceable resources and adding a storage node that does not meet the scheduling threshold to a pool of non-serviceable resources may include:
Adding the storage nodes with the request connection numbers lower than the scheduling threshold value into a serviceable resource pool;
And adding the storage nodes with the request connection numbers not lower than the scheduling threshold value into the non-serviceable resource pool.
Therefore, the embodiment of the application provides a storage node dividing method based on a scheduling threshold, namely, a storage node with the number of the requested connections lower than the scheduling threshold is taken as a serviceable node to be added into a serviceable resource pool, and a storage node with the number of the requested connections not lower than the scheduling threshold is taken as an unserviceable node to be added into an unserviceable resource pool.
S104, selecting a target storage node in the serviceable resource pool for the new connection request in the preset scheduling period, and scheduling the new connection request to the target storage node.
This step aims at achieving a reasonable scheduling of new client connection requests. Specifically, when a new connection request initiated by any client is received in the present scheduling period, an optimal serviceable node can be directly selected from the serviceable resource pool as a target storage node to provide service for the new connection request, i.e. the new connection request is scheduled to the target storage node for processing. Still further, the new connection request may be handled directly with the serviceable node requesting the least number of connections as the target storage node.
In one embodiment of the present application, the request scheduling method in the clustered system may further include:
counting the number of real-time connection requests of each non-serviceable node in the non-serviceable resource pool in a preset scheduling period;
And eliminating the non-serviceable nodes with the real-time connection request number lower than the scheduling threshold value from the non-serviceable resource pool, and adding the non-serviceable nodes into the serviceable resource pool.
The request scheduling method in the cluster system provided by the embodiment of the application can further realize the dynamic adjustment of the storage nodes in the cluster system. Specifically, for each non-serviceable node in the non-serviceable resource pool, the number of connection requests can be counted in real time in the current scheduling period, and once the number of real-time connection requests is lower than the scheduling threshold, the corresponding non-serviceable node can be directly removed from the non-serviceable resource pool and added into the serviceable resource pool to continue to provide service for the client connection request.
It can be seen that, in the request scheduling method in the cluster system provided by the embodiment of the present application, by using the main service node in the cluster system, the number of request connections is detected for each storage node in the cluster system according to the scheduling period, so as to obtain the total number of request connections of the cluster system in the current scheduling period, thereby determining the scheduling threshold value in the current scheduling period in combination with the total number of storage nodes in the cluster system, and therefore, the client connection request received in the current scheduling period can be scheduled by referring to the scheduling threshold value, i.e. all the storage nodes in the cluster system are divided into serviceable nodes and non-serviceable nodes based on the scheduling threshold value in the current scheduling period, so that the serviceable nodes can be conveniently selected to provide services for new client connection requests. Therefore, the technical scheme can perform more balanced and reasonable scheduling on the client connection request so as to effectively ensure the stability of the cluster system.
Based on the above embodiments:
In one embodiment of the present application, the cluster system may further include a domain name server, the main service node is connected to the domain name server, and the request scheduling method in the cluster system may further include uploading node information and request connection numbers of the serviceable nodes in the serviceable resource pool to the domain name server, so that the domain name server responds to the domain name mounting request initiated by the client with reference to the node information and the connection request numbers of the serviceable nodes.
The request scheduling method in the cluster system provided by the embodiment of the application can further realize the domain name mounting processing function. Specifically, a domain name server (DNS server) may be deployed in the current cluster system, where the domain name server and the main service node may be in communication connection, so that in each scheduling period, the main service node may upload node information and a request connection number of each serviceable node in the serviceable resource pool acquired by the main service node to the domain name server. Therefore, when the domain name server receives the domain name mounting request initiated by the client in the current scheduling period, the domain name server can select a mountable node from all the serviceable nodes according to the information of the main service node to respond to the domain name mounting request, and domain name mounting processing is realized.
In the specific implementation process, the domain name server can directly select the serviceable node with the minimum request connection number as the mountable node, and feed back the node information of the mountable node to the client corresponding to the domain name mounting request, so that the client and the mountable node can perform information interaction, and the domain name mounting process is realized. In one possible implementation manner, the node information may be node IP information.
Therefore, in the embodiment of the application, the cluster system can further realize the domain name mounting process, and the main service node can synchronously upload the node information and the request connection number of each service node to the domain name server in the current dispatching period, so that when the domain name server receives the domain name mounting request initiated by the client, the domain name server can directly select the mountable node to respond to the domain name mounting request by referring to the node information and the request connection number of each service node, and the detection and the screening of the mountable node in the cluster system are not required to be carried out again, thereby effectively improving the domain name mounting process efficiency.
In one embodiment of the present application, the request scheduling method in the clustered system may further include:
performing fault detection on each storage node in the cluster system, and determining a fault storage node;
And selecting a target serviceable node in the serviceable resource pool, and scheduling all connection requests on the fault storage node to the target serviceable node.
The request scheduling method in the cluster system provided by the embodiment of the application can further realize the fault node of each storage node in the cluster system. The main service node can detect faults of each storage node in the cluster system periodically to determine whether the fault storage node occurs in the cluster system, and further, when the fault storage node occurs in the cluster system, the main service node can directly select a target serviceable node in the serviceable resource pool to accept all connection requests on the fault storage node. It will be appreciated that the number of target serviceable nodes may be one or more, and may be capable of accepting all connection requests on the failed storage node, which is not limited by the present application.
Wherein, fault detection is performed on each storage node in the cluster system, and determining the fault storage node may include:
sending a heartbeat signal to each storage node in the cluster system, and determining whether a heartbeat reaction signal fed back by the storage node is received within a preset time length;
when a heartbeat reaction signal fed back by the storage node is received within a preset time period, determining that the storage node is a normal storage node;
and when the heartbeat reaction signal fed back by the storage node is not received within the preset time, determining the storage node as a fault storage node.
The embodiment of the application provides a fault detection method of a storage node, which can be realized through a heartbeat monitoring mechanism. Specifically, the primary service node may periodically send a heartbeat signal to each storage node in the cluster system to determine whether a heartbeat reaction signal fed back by each storage node may be received within a preset duration. Obviously, if the heartbeat reaction signal fed back by the storage node can be received within the preset time, the storage node can be determined to be a normal storage node, otherwise, if the heartbeat reaction signal fed back by the storage node can not be received within the preset time, the storage node can be determined to be a fault storage node, so that the fault detection of the storage node is realized. It should be noted that, the specific value of the preset duration does not affect implementation of the technical solution, and the technical solution is set by a technician according to actual situations, which is not limited in the embodiment of the present application, and in one possible implementation manner, the preset duration may be 30 seconds.
Wherein selecting a target serviceable node in the serviceable resource pool and scheduling all connection requests on the failed storage node to the target serviceable node may include:
Determining an actual number of requested connections on the failed storage node;
For each serviceable node in the serviceable resource pool, calculating a difference value between the number of requested connections on the serviceable node and a scheduling threshold;
arranging the serviceable nodes in the serviceable resource pool according to the order of the difference from big to small to obtain a serviceable node sequence;
Selecting a previous one or more serviceable nodes in the serviceable node sequence as target serviceable nodes according to the actual request connection number;
Scheduling all connection requests on the failed storage node to the target serviceable node;
Accordingly, for a new connection request in a preset scheduling period, selecting a target storage node in the serviceable resource pool, and scheduling the new connection request to the target storage node, may include:
Selecting a serviceable node with the minimum request connection number from a serviceable resource pool as a target storage node for a new connection request in a preset scheduling period;
and scheduling the new connection request to the target storage node.
The embodiment of the application provides a method for realizing connection request scheduling on a fault storage node. Specifically, for each serviceable node in the serviceable resource pool, the difference between the number of requested connections on the serviceable node and the scheduling threshold value can be calculated, all serviceable nodes in the serviceable resource pool are arranged according to the order of the difference from large to small to obtain a serviceable node sequence, therefore, the screening of the target serviceable node in the serviceable node sequence can be carried out by referring to the actual number of requested connections on the fault storage node, and all connection requests on the fault storage node are scheduled to the target serviceable node for processing. More specifically, it is necessary to ensure that the sum of the differences corresponding to the selected target serviceable nodes is greater than or equal to the number of actual requested connections on the failed storage node, so as to effectively ensure that each connection request on the failed storage node can be scheduled to the serviceable node for processing.
Further, performing fault detection on each storage node in the cluster system, after determining the faulty storage node, may further include:
recording node information of the fault storage node to a fault node list;
performing fault detection on each fault storage node in the fault node list according to a preset time interval, and determining a fault recovery node;
And adding the fault recovery node into the serviceable resource pool.
The request scheduling method in the cluster system provided by the embodiment of the application can further realize node scheduling processing after the recovery of the fault node. Specifically, for the fault storage nodes in the cluster system, the node information of the fault storage nodes can be recorded in a list form, the main service node can periodically perform fault detection on each fault storage node in the fault node list to determine whether each fault storage node is recovered to be normal, and the recovered normal storage nodes (namely the fault recovery nodes) are re-added into the serviceable resource pool, so that dynamic adjustment of the serviceable nodes in the serviceable resource pool is realized. Furthermore, the connection request previously scheduled to other serviceable nodes can be relocated back to the fault recovery node to continue to provide service. Likewise, the specific value of the preset time interval does not affect implementation of the technical scheme, and the technical scheme is set by a technician according to actual conditions, which is not limited in the embodiment of the application, and in a possible implementation manner, the preset time interval can be 1 hour.
Based on the above embodiments, the embodiment of the present application takes a distributed storage system as an example, and provides another request scheduling method in a cluster system.
The implementation flow of the request scheduling method in the cluster system provided by the embodiment of the application is as follows:
1. on a main service node of the distributed storage system, setting and regularly collecting the request connection number of each storage node in the storage system, adding and calculating to obtain the total number N of request connection of all the storage nodes in the storage system, and collecting the total number M of storage nodes in the storage system.
2. To achieve the load balancing effect, the storage system determines a threshold value of the number of requested connections of each storage node as N/M (scheduling threshold value).
3. The master service node may check the number of requested connections K in real time, storage node by storage node matching. When K is more than or equal to N/M, deleting the IP address of the storage node from the serviceable resource pool, and forming an unserviceable resource pool by the deleted storage node.
4. The main service node inquires the IP address and the request connection number K of each storage node in the non-service resource pool one by one, when K is smaller than N/M, the main service node judges whether the IP address is in the service resource pool or not according to the IP address, if not, the IP address is added into the service resource pool again.
5. When the client service newly initiates a connection request, the main service node can select a serviceable storage node from the serviceable resource pool for the client to use according to the set balance strategy.
6. When a single node failure occurs in the distributed storage system due to an abnormal condition, the service on the failed storage node can be drifted to one or more storage nodes in the serviceable resource pool, and at the moment, the service pressure on the storage node can be doubled. Still further, the storage node may be added directly to the pool of non-serviceable resources.
7. When the fault is recovered, the service which drifts before can be re-migrated to the storage node which recovers the fault, and the fault recovery node and the storage node which receives the drifted service before can be added to the serviceable resource pool again.
8. The distributed storage system is also provided with a DNS server, a serviceable resource pool can be synchronously established on the DNS server, and the main service node can synchronously report the related information of the serviceable resource pool detected by the main service node to the DNS server.
9. When the service client initiates a domain name mounting request, the DNS server can directly acquire the IP address with the lowest request connection number from the serviceable resource pool on the DNS server, and answer to the service client for mounting.
10. When the DNS server finds that a large number of mounting requests are received in a short time, IP information response can be directly obtained from the serviceable resource pool in a polling mode and is sent to the service client.
It can be seen that, in the request scheduling method in the cluster system provided by the embodiment of the present application, by using the main service node in the cluster system, the number of request connections is detected for each storage node in the cluster system according to the scheduling period, so as to obtain the total number of request connections of the cluster system in the current scheduling period, thereby determining the scheduling threshold value in the current scheduling period in combination with the total number of storage nodes in the cluster system, and therefore, the client connection request received in the current scheduling period can be scheduled by referring to the scheduling threshold value, i.e. all the storage nodes in the cluster system are divided into serviceable nodes and non-serviceable nodes based on the scheduling threshold value in the current scheduling period, so that the serviceable nodes can be conveniently selected to provide services for new client connection requests. Therefore, the technical scheme can perform more balanced and reasonable scheduling on the client connection request so as to effectively ensure the stability of the cluster system.
The embodiment of the application provides a request scheduling device in a cluster system.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a request scheduling device in a cluster system, where the cluster system includes a main service node and a plurality of storage nodes, each storage node is connected to the main service node, and the device is applied to the main service node and may include:
The acquisition module 1 is used for acquiring the request connection number of each storage node in a preset scheduling period and calculating the total number of the request connections of the cluster system;
A determining module 2, configured to determine a scheduling threshold in a preset scheduling period according to the total number of requested connections and the total number of nodes of storage nodes in the cluster system;
a setting module 3, configured to add a storage node that meets a scheduling threshold to a serviceable resource pool, and add a storage node that does not meet the scheduling threshold to an unserviceable resource pool;
And the scheduling module 4 is used for selecting a target storage node from the serviceable resource pool for the new connection request in the preset scheduling period and scheduling the new connection request to the target storage node.
It can be seen that, according to the request scheduling device in the cluster system provided by the embodiment of the application, by using the main service node in the cluster system, the number of request connections is detected for each storage node in the cluster system according to the scheduling period, so as to count and obtain the total number of request connections of the cluster system in the current scheduling period, thereby determining the scheduling threshold value in the current scheduling period in combination with the total number of storage nodes in the cluster system, and therefore, the client connection requests received in the current scheduling period can be scheduled by referring to the scheduling threshold value, namely, all the storage nodes in the cluster system are divided into serviceable nodes and non-serviceable nodes based on the scheduling threshold value in the current scheduling period, so that the serviceable nodes can be conveniently selected to provide services for new client connection requests. Therefore, the technical scheme can perform more balanced and reasonable scheduling on the client connection request so as to effectively ensure the stability of the cluster system.
In one embodiment of the present application, the cluster system may further include a domain name server, the main service node is connected to the domain name server, and the request scheduling apparatus may further include:
And the domain name request processing module is used for uploading the node information and the request connection number of each serviceable node in the serviceable resource pool to the domain name server so that the domain name server responds to the domain name mounting request initiated by the client by referring to the node information and the connection request number of each serviceable node.
In one embodiment of the present application, the request scheduling apparatus in the cluster system may further include:
The detection module is used for carrying out fault detection on each storage node in the cluster system and determining a fault storage node;
and the migration module is used for selecting a target serviceable node from the serviceable resource pool and scheduling all connection requests on the fault storage node to the target serviceable node.
In one embodiment of the application, the migration module is specifically configured to determine an actual request connection number on a failed storage node, calculate, for each serviceable node in the serviceable resource pool, a difference between the request connection number on the serviceable node and a scheduling threshold, arrange the serviceable nodes in the serviceable resource pool in order of the difference from large to small to obtain a serviceable node sequence, select a previous one or more serviceable nodes in the serviceable node sequence as a target serviceable node according to the actual request connection number, and schedule all connection requests on the failed storage node to the target serviceable node;
Correspondingly, the scheduling module 4 may be specifically configured to select, for a new connection request in a preset scheduling period, a serviceable node with the smallest request connection number in the serviceable resource pool as a target storage node, and schedule the new connection request to the target storage node.
In one embodiment of the present application, the request scheduling apparatus in the cluster system may further include:
The recovery module is used for carrying out fault detection on each storage node in the cluster system, recording node information of the fault storage nodes to a fault node list after the fault storage nodes are determined, carrying out fault detection on each fault storage node in the fault node list according to a preset time interval, determining the fault recovery nodes, and adding the fault recovery nodes into the serviceable resource pool.
In one embodiment of the present application, the setting module 3 may be specifically configured to add a storage node with a number of requested connections below the scheduling threshold to the serviceable resource pool, and add a storage node with a number of requested connections not below the scheduling threshold to the non-serviceable resource pool.
In one embodiment of the present application, the request scheduling apparatus in the cluster system may further include:
And the monitoring module is used for counting the real-time connection request number of each non-serviceable node in the non-serviceable resource pool in a preset scheduling period, and removing the non-serviceable nodes with the real-time connection request number lower than the scheduling threshold value from the non-serviceable resource pool and adding the non-serviceable nodes into the serviceable resource pool.
For the description of the apparatus provided by the embodiment of the present application, refer to the above method embodiment, and the description of the present application is omitted here.
The embodiment of the application provides electronic equipment.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to the present application, where the electronic device may include:
a memory 11 for storing a computer program;
A processor 10, when executing a computer program, is configured to implement the steps of the request scheduling method in any of the cluster systems described above.
As shown in fig. 4, which is a schematic diagram of the composition structure of the electronic device, the electronic device may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all complete communication with each other through a communication bus 13.
In an embodiment of the present application, the processor 10 may be a central processing unit (Central Processing Unit, CPU), an asic, a dsp, a field programmable gate array, or other programmable logic device, etc.
The processor 10 may call a program stored in the memory 11, and in particular, the processor 10 may perform operations in an embodiment of a request scheduling method in a clustered system.
The memory 11 is used for storing one or more programs, and the programs may include program codes including computer operation instructions, and in the embodiment of the present application, at least the programs for implementing the following functions are stored in the memory 11:
Acquiring the request connection number of each storage node in a preset scheduling period, and calculating to obtain the total number of request connections of the cluster system;
determining a scheduling threshold value in a preset scheduling period according to the total number of the requested connections and the total number of the nodes of the storage nodes in the cluster system;
Adding a storage node meeting a scheduling threshold into a serviceable resource pool, and adding a storage node not meeting the scheduling threshold into an unserviceable resource pool;
and selecting a target storage node from the serviceable resource pool for a new connection request in a preset scheduling period, and scheduling the new connection request to the target storage node.
In one possible implementation, the memory 11 may include a storage program area that may store an operating system, and at least one application program required for functions, etc., and a storage data area that may store data created during use.
In addition, the memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid-state storage device.
The communication interface 12 may be an interface of a communication module for interfacing with other devices or systems.
Of course, it should be noted that the structure shown in fig. 4 is not limited to the electronic device in the embodiment of the present application, and the electronic device may include more or fewer components than those shown in fig. 4 or may combine some components in practical applications.
Embodiments of the present application provide a computer-readable storage medium.
The computer readable storage medium provided by the embodiment of the application stores a computer program, and when the computer program is executed by a processor, the steps of the request scheduling method in any cluster system can be realized.
The computer readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server, a data center, etc. that includes one or more available media, for example, a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape, etc.), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk), etc. each medium may store computer program code.
For the description of the computer-readable storage medium provided in the embodiment of the present application, refer to the above method embodiment, and the description of the present application is omitted here.
Embodiments of the present application provide a computer program product.
The computer program product provided by the embodiment of the application comprises a computer program/instruction, and the computer program/instruction can realize the steps of the request scheduling method in any cluster system when being executed by a processor.
Specifically, in the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.
Wherein a computer program product may comprise one or more computer programs/instructions which, when loaded and executed on a computer, may fully or partially produce a process or function in accordance with embodiments of the present application. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line, etc.) or wireless (e.g., infrared, wireless, microwave, etc.).
For an introduction to the computer program product provided by the embodiment of the present application, reference is made to the above method embodiment, and the description of the present application is omitted here.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The technical scheme provided by the application is described in detail. The principles and embodiments of the present application have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present application and its core ideas. It should be noted that it will be apparent to those skilled in the art that the present application may be modified and practiced without departing from the spirit of the present application.
Claims (10)
1. A request scheduling method in a cluster system, wherein the cluster system includes a main service node and a plurality of storage nodes, each of the storage nodes is connected to the main service node, the method is applied to the main service node, and comprises:
acquiring the request connection number of each storage node in a preset scheduling period, and calculating the total number of request connections of the cluster system;
determining a scheduling threshold value in the preset scheduling period according to the total number of the request connections and the total number of the nodes of the storage nodes in the cluster system;
Adding the storage nodes meeting the scheduling threshold into a serviceable resource pool, and adding the storage nodes not meeting the scheduling threshold into an unserviceable resource pool;
And selecting a target storage node from the serviceable resource pool for a new connection request in the preset scheduling period, and scheduling the new connection request to the target storage node.
2. The method for scheduling requests in a clustered system according to claim 1, wherein the clustered system further comprises a domain name server, the primary service node being connected to the domain name server, the method further comprising:
and uploading the node information and the request connection number of each serviceable node in the serviceable resource pool to the domain name server so that the domain name server responds to the domain name mounting request initiated by the client by referring to the node information and the connection request number of each serviceable node.
3. The method for scheduling requests in a clustered system of claim 1 further comprising:
performing fault detection on each storage node in the cluster system, and determining a fault storage node;
and selecting a target serviceable node from the serviceable resource pool, and scheduling all connection requests on the fault storage node to the target serviceable node.
4. A method of scheduling requests in a clustered system according to claim 3 wherein selecting a target serviceable node in the pool of serviceable resources and scheduling all connection requests on the failed storage node to the target serviceable node comprises:
determining an actual number of requested connections on the failed storage node;
for each serviceable node in the serviceable resource pool, calculating a difference between the number of requested connections on the serviceable node and the scheduling threshold;
arranging the serviceable nodes in the serviceable resource pool according to the order of the difference from large to small to obtain a serviceable node sequence;
Selecting a previous one or more serviceable nodes in the sequence of serviceable nodes as the target serviceable node according to the actual request connection number;
Scheduling all of the connection requests on the failed storage node to the target serviceable node;
correspondingly, for a new connection request in the preset scheduling period, selecting a target storage node in the serviceable resource pool, and scheduling the new connection request to the target storage node, including:
Selecting a serviceable node with the minimum request connection number from the serviceable resource pool as the target storage node for the new connection request in the preset scheduling period;
And dispatching the new connection request to the target storage node.
5. A method of scheduling requests in a clustered system according to claim 3, wherein performing fault detection on each storage node in the clustered system, after determining a faulty storage node, further comprises:
Recording node information of the fault storage node to a fault node list;
Performing fault detection on each fault storage node in the fault node list according to a preset time interval, and determining a fault recovery node;
And adding the fault recovery node into the serviceable resource pool.
6. The method of claim 1, wherein adding storage nodes that meet the scheduling threshold to a pool of serviceable resources and adding storage nodes that do not meet the scheduling threshold to a pool of non-serviceable resources comprises:
Adding the storage nodes with the request connection numbers lower than the scheduling threshold to the serviceable resource pool;
And adding the storage nodes with the request connection numbers not lower than the scheduling threshold value into the non-serviceable resource pool.
7. The method for scheduling requests in a clustered system of claim 6, further comprising:
Counting the number of real-time connection requests of each non-serviceable node in the non-serviceable resource pool in the preset scheduling period;
and rejecting the non-serviceable nodes with the real-time connection request number lower than the scheduling threshold value from the non-serviceable resource pool, and adding the non-serviceable nodes into the serviceable resource pool.
8. A request scheduling apparatus in a cluster system, the cluster system comprising a main service node and a plurality of storage nodes, each of the storage nodes being connected to the main service node, the apparatus being applied to the main service node, comprising:
the acquisition module is used for acquiring the request connection number of each storage node in a preset scheduling period and calculating the total number of the request connections of the cluster system;
The determining module is used for determining a scheduling threshold value in the preset scheduling period according to the total number of the request connections and the total number of the nodes of the storage nodes in the cluster system;
The setting module is used for adding the storage nodes meeting the scheduling threshold into a serviceable resource pool and adding the storage nodes not meeting the scheduling threshold into an unserviceable resource pool;
And the scheduling module is used for selecting a target storage node from the serviceable resource pool for the new connection request in the preset scheduling period, and scheduling the new connection request to the target storage node.
9. An electronic device, comprising:
A memory for storing a computer program;
processor for implementing the steps of the request scheduling method in a clustered system according to any of claims 1 to 7 when executing said computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the request scheduling method in a clustered system according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411855512.6A CN119629252B (en) | 2024-12-16 | Request scheduling method and device in cluster system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411855512.6A CN119629252B (en) | 2024-12-16 | Request scheduling method and device in cluster system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119629252A true CN119629252A (en) | 2025-03-14 |
| CN119629252B CN119629252B (en) | 2025-10-17 |
Family
ID=
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070184903A1 (en) * | 2006-02-08 | 2007-08-09 | Derek Liu | Network-based game system capable of serving massive number of game players |
| US20140304352A1 (en) * | 2013-04-06 | 2014-10-09 | Citrix Systems, Inc. | Systems and methods for cluster parameter limit |
| US20180219968A1 (en) * | 2017-01-31 | 2018-08-02 | Salesforce.Com, Inc. | Automatic surge protection of system resources |
| CN111225004A (en) * | 2018-11-23 | 2020-06-02 | 中移（杭州）信息技术有限公司 | Method and device for expanding server cluster and readable medium |
| US20210045007A1 (en) * | 2019-08-08 | 2021-02-11 | At&T Intellectual Property I, L.P. | Management of overload condition for 5g or other next generation wireless network |
| CN113504996A (en) * | 2021-07-28 | 2021-10-15 | 中国工商银行股份有限公司 | Load balance detection method, device, equipment and storage medium |
| CN117724849A (en) * | 2023-12-29 | 2024-03-19 | 金篆信科有限责任公司 | Connection pool management method, device, electronic equipment and computer readable storage medium |
| CN118069354A (en) * | 2024-02-04 | 2024-05-24 | 成都虚谷伟业科技有限公司 | Load balancing method, device, equipment and medium |
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070184903A1 (en) * | 2006-02-08 | 2007-08-09 | Derek Liu | Network-based game system capable of serving massive number of game players |
| US20140304352A1 (en) * | 2013-04-06 | 2014-10-09 | Citrix Systems, Inc. | Systems and methods for cluster parameter limit |
| US20180219968A1 (en) * | 2017-01-31 | 2018-08-02 | Salesforce.Com, Inc. | Automatic surge protection of system resources |
| CN111225004A (en) * | 2018-11-23 | 2020-06-02 | 中移（杭州）信息技术有限公司 | Method and device for expanding server cluster and readable medium |
| US20210045007A1 (en) * | 2019-08-08 | 2021-02-11 | At&T Intellectual Property I, L.P. | Management of overload condition for 5g or other next generation wireless network |
| CN113504996A (en) * | 2021-07-28 | 2021-10-15 | 中国工商银行股份有限公司 | Load balance detection method, device, equipment and storage medium |
| CN117724849A (en) * | 2023-12-29 | 2024-03-19 | 金篆信科有限责任公司 | Connection pool management method, device, electronic equipment and computer readable storage medium |
| CN118069354A (en) * | 2024-02-04 | 2024-05-24 | 成都虚谷伟业科技有限公司 | Load balancing method, device, equipment and medium |
Non-Patent Citations (1)
| Title |
|---|
| 赵丹;王伟;华中;孔哲;潘亚辰;: "Android平台下VOD系统动态负载均衡的应用研究", 电子设计工程, no. 08, 30 April 2016 (2016-04-30) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11729260B2 (en) | Internet-of-things resource access system and method | |
| CN113810304B (en) | A load balancing method, device, equipment and computer storage medium | |
| US11316792B2 (en) | Method and system of limiting traffic | |
| CN107872402B (en) | Global flow scheduling method and device and electronic equipment | |
| EP2515504B1 (en) | Content delivery method, system and schedule server | |
| CN109167674B (en) | Service node scoring method, Domain Name System (DNS) scheduling method and server | |
| CN103383689A (en) | Service process fault detection method, device and service node | |
| CN105516347A (en) | Method and device for load balance allocation of streaming media server | |
| CN106936618A (en) | A kind of collecting method and system | |
| CN109921925B (en) | Dial testing method and device | |
| CN110545197B (en) | Node state monitoring method and device | |
| US9736235B2 (en) | Computer system, computer, and load balancing method | |
| CN107105013B (en) | File processing method, server, terminal and system | |
| CN117155936A (en) | Method and device for realizing load balancing | |
| CN107203464B (en) | Method and device for positioning service problem | |
| CN112261133A (en) | CDN node control method, device, server and storage medium | |
| CN112751926A (en) | Method, system and related device for managing working nodes in cluster | |
| CN119629252B (en) | Request scheduling method and device in cluster system, electronic equipment and storage medium | |
| CN119629252A (en) | Request scheduling method, device, electronic device and storage medium in cluster system | |
| EP3756310A1 (en) | Method and first node for managing transmission of probe messages | |
| CN116055496B (en) | Monitoring data acquisition method and device, electronic equipment and storage medium | |
| CN116996517B (en) | Load balancing method, device, electronic equipment and medium | |
| US9967163B2 (en) | Message system for avoiding processing-performance decline | |
| CN117459536A (en) | Resource distribution method, cloud server, device, electronic equipment and storage medium | |
| CN115952003A (en) | Method, device, equipment and storage medium for cluster server load balancing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |