CN116166755B - Search engine database management method, device and search engine database - Google Patents

Abstract

The present application provides a search engine database management method, device and search engine database. The method includes: obtaining the current data storage capacity of each computer equipment node corresponding to the search engine database, the current storage usage rate of each computer equipment node, and the new fragmentation according to the current data storage capacity of each computer device node, the current storage usage rate and the number of new fragments, determine the target computer device node; assign the new fragments of the search engine database to the target computer device node . By comprehensively considering the current data storage capacity of each computer device node, the current storage usage rate, and the number of new fragments, the target computer device node corresponding to the new fragment is selected to ensure that after the new fragment is allocated to the corresponding target computer device node, each computer The data storage of device nodes can be balanced, which improves resource utilization and lays the foundation for improving the search efficiency of ES databases.

Description

Search engine database management method, device and search engine database

technical field

The present application relates to the technical field of databases, in particular to a search engine database management method and device and a search engine database.

Background technique

The Elasticsearch (ES) database is a widely used and powerful distributed full-text search engine today. Its index data can be distributed and stored on multiple computer device nodes. The data blocks stored here are called shards. The allocation mechanism of shards on multiple computer devices determines the balance of ES database storage. Therefore, when creating new shards, how to allocate them to corresponding computer device nodes has become a key research content.

In the prior art, the computer device node list is usually sorted in ascending order according to the number of existing shards, and the computer device node that currently has the fewest number of shards is found, and then the newly created shard is assigned to the node that currently has the fewest number of shards. Computer device node.

However, in the actual production environment, as the ES index is turned on and off, computer device nodes go online and offline, and some data is deleted alternately, if the newly created shards are allocated based on the existing technology, it will cause each computer to The data storage of device nodes is seriously unbalanced, which not only reduces the resource utilization rate, but also is not conducive to ensuring the search efficiency of the ES database.

Contents of the invention

This application provides a search engine database management method, device and search engine database to solve the situation that the data storage of each computer device node is seriously unbalanced in the prior art, which not only reduces the resource utilization rate, but also helps to ensure the search of the ES database Efficiency and other defects.

The first aspect of the present application provides a search engine database management method, including:

Obtain the current data storage capacity of each computer device node corresponding to the search engine database, the current storage usage rate of each computer device node, and the number of new fragments;

Determine the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments;

Allocating the new fragment of the search engine database to the target computer device node.

Optionally, the determining the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments includes:

According to the current storage usage rate of each of the computer equipment nodes, determine the current balance evaluation index between each of the computer equipment nodes;

Determine node selection constraints according to the current balance evaluation indicators among the computer equipment nodes;

According to the node selection constraints, the target computer device node is determined according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments.

Optionally, the determining the current balance evaluation index between each of the computer equipment nodes according to the current storage usage rate of each of the computer equipment nodes includes:

According to the current storage usage rate of each of the computer device nodes, determine the current storage usage rate standard deviation between each of the computer device nodes;

The standard deviation of the current storage usage rate is used as the current balance evaluation index among the computer device nodes.

Optionally, according to the node selection constraints, determining the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments includes:

According to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node and the number of new fragments, determine the computer device node to be selected;

Predicting the latest balance evaluation index between each computer device node after the new fragment of the search engine database is allocated to the computer device node to be selected;

When the latest balance evaluation index is smaller than the current balance evaluation index, the computer device node to be selected is determined as the target computing device node.

Optionally, the determining the computer device node to be selected according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments includes:

Predicting the data storage capacity of the new fragment according to the current data storage capacity of each of the computer device nodes;

According to the current storage usage rate of each of the computer equipment nodes, the number of new slices and the data storage capacity of the new slices, the computer equipment nodes to be selected are determined.

Optionally, the predicting the data storage capacity of the new fragment according to the current data storage capacity of each of the computer device nodes includes:

According to the current data storage capacity of each of the computer device nodes and the current number of fragments of each of the computer device nodes, the data storage capacity of the new fragment is predicted.

Optionally, the predicting the data storage capacity of the new fragment according to the current data storage capacity of each of the computer device nodes and the current fragmentation quantity of each of the computer device nodes includes:

According to the current data storage capacity of each described computer device node and the current fragmentation quantity of each described computer device node, determine the average data storage capacity of a single fragment of the search engine database;

The average data storage capacity of the single shard is used as the data storage capacity of the new shard.

Optionally, the predicting the data storage capacity of the new fragment according to the current data storage capacity of each of the computer device nodes and the current fragmentation quantity of each of the computer device nodes includes:

According to the current data storage capacity of each described computer device node and the current fragmentation quantity of each described computer device node, determine the geometric mean data storage capacity of a single fragment of the search engine database;

The geometric mean data storage capacity of the single shard is used as the data storage capacity of the new shard.

Optionally, the determining the computer device node to be selected according to the current storage usage rate of each computer device node, the number of new fragments and the data storage capacity of the new fragments includes:

According to the current storage utilization rate of each described computer device node, construct a storage utilization rate summary table;

Judging whether the number of the new fragments is greater than the total number of computer device nodes;

When the number of the new fragments is greater than the total number of computer equipment nodes, according to the storage usage summary table, screen candidate computer equipment nodes corresponding to the first number of new fragments;

According to the current data storage capacity of each computer device node to be selected and the data storage capacity of the new fragment, update the storage usage rate of each computer device node to be selected, and other than the computer device node to be selected The storage utilization rate of the computer device node remains unchanged, so as to iteratively update the storage utilization rate summary table, and use the updated storage utilization rate summary table as a new storage utilization rate summary table;

Updating the number of new shards to reduce the first amount, so as to iteratively update the number of new shards, and return to the step of judging whether the number of new shards is greater than the total number of computer device nodes .

Optionally, when the number of new shards is greater than the total number of computer device nodes, according to the storage usage summary table, screening candidate computer device nodes corresponding to the first number of new shards includes :

When the number of new fragments is greater than the total number of computer device nodes, sort each computing device node in ascending order according to the current storage usage rate of each computer device node represented by the storage usage summary table ;

According to the ascending sorting results of each computing device node, the first number of computer device nodes to be selected is screened from small to large.

Optionally, the method also includes:

The first quantity is determined according to the total number of computer equipment nodes and a preset first distribution ratio.

Optionally, the method also includes:

When the number of the new fragments is not greater than the total number of computer device nodes, it is judged whether the number of the new fragments is equal to 1;

When the number of new fragments is equal to 1, the computer device node with the lowest current storage utilization represented by the storage utilization summary table is determined as the candidate computer device node corresponding to the new fragment.

Optionally, the method also includes:

When the number of the new fragments is not greater than the total number of computer equipment nodes, and the number of the new fragments is greater than 1, according to the storage usage summary table, screen the second number of new fragments corresponding to the pending Select a computer device node;

According to the current data storage capacity of each computer device node to be selected and the data storage capacity of the new fragment, update the storage usage rate of each computer device node to be selected, and other than the computer device node to be selected The storage utilization rate of the computer device node remains unchanged, so as to iteratively update the storage utilization rate summary table, and use the updated storage utilization rate summary table as a new storage utilization rate summary table;

Updating the number of new fragments by a second quantity, so as to iteratively update the number of new fragments, and return to the step of judging whether the number of new fragments is equal to 1.

Optionally, when the number of new fragments is not greater than the total number of computer device nodes, and the number of new fragments is greater than 1, according to the storage usage summary table, the second number of The computer device nodes to be selected corresponding to the new shard, including:

When the number of new fragments is not greater than the total number of computer equipment nodes, and the number of new fragments is greater than 1, the current storage usage of each computer equipment node represented by the storage usage summary table rate, sorting each computing device node in ascending order;

According to the ascending sorting results of each of the computing device nodes, the second number of computer device nodes to be selected is screened from small to large.

Optionally, also include:

The second quantity is determined according to the total amount of computer equipment nodes and a preset second distribution ratio.

Optionally, the obtaining the current storage usage rate of each computer device node corresponding to the search engine database includes:

Acquiring the data storage space of each computer device node;

For any of the computer device nodes, the current storage usage rate of the computer device node is determined according to the current data storage capacity and data storage space of the computer device node.

The second aspect of the present application provides a search engine database management device, including:

An acquisition module, configured to acquire the current data storage capacity of each computer device node corresponding to the search engine database, the current storage usage rate of each computer device node, and the number of new fragments;

A determining module, configured to determine the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments;

The management module is used for distributing the new fragment of the search engine database to the target computer device node.

Optionally, the determination module is specifically used for:

According to the current storage usage rate of each of the computer equipment nodes, determine the current balance evaluation index between each of the computer equipment nodes;

Determine node selection constraints according to the current balance evaluation indicators among the computer equipment nodes;

According to the node selection constraints, the target computer device node is determined according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments.

Optionally, the determination module is specifically used for:

According to the current storage usage rate of each of the computer device nodes, determine the current storage usage rate standard deviation between each of the computer device nodes;

The standard deviation of the current storage usage rate is used as the current balance evaluation index among the computer device nodes.

Optionally, the determination module is specifically used for:

According to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node and the number of new fragments, determine the computer device node to be selected;

Predicting the latest balance evaluation index between each computer device node after the new fragment of the search engine database is allocated to the computer device node to be selected;

When the latest balance evaluation index is smaller than the current balance evaluation index, the computer device node to be selected is determined as the target computing device node.

Optionally, the determination module is specifically used for:

Predicting the data storage capacity of the new fragment according to the current data storage capacity of each of the computer device nodes;

According to the current storage usage rate of each of the computer equipment nodes, the number of new slices and the data storage capacity of the new slices, the computer equipment nodes to be selected are determined.

Optionally, the determination module is specifically used for:

According to the current data storage capacity of each of the computer device nodes and the current number of fragments of each of the computer device nodes, the data storage capacity of the new fragment is predicted.

Optionally, the determination module is specifically used for:

According to the current data storage capacity of each described computer device node and the current fragmentation quantity of each described computer device node, determine the average data storage capacity of a single fragment of the search engine database;

The average data storage capacity of the single shard is used as the data storage capacity of the new shard.

Optionally, the determination module is specifically used for:

According to the current data storage capacity of each described computer device node and the current fragmentation quantity of each described computer device node, determine the geometric mean data storage capacity of a single fragment of the search engine database;

The geometric mean data storage capacity of the single shard is used as the data storage capacity of the new shard.

Optionally, the determination module is specifically used for:

According to the current storage utilization rate of each described computer device node, construct a storage utilization rate summary table;

Judging whether the number of the new fragments is greater than the total number of computer device nodes;

When the number of the new fragments is greater than the total number of computer equipment nodes, according to the storage usage summary table, screen candidate computer equipment nodes corresponding to the first number of new fragments;

According to the current data storage capacity of each computer device node to be selected and the data storage capacity of the new fragment, update the storage usage rate of each computer device node to be selected, and other than the computer device node to be selected The storage utilization rate of the computer device node remains unchanged, so as to iteratively update the storage utilization rate summary table, and use the updated storage utilization rate summary table as a new storage utilization rate summary table;

Updating the number of new shards to reduce the first amount, so as to iteratively update the number of new shards, and return to the step of judging whether the number of new shards is greater than the total number of computer device nodes .

Optionally, the determination module is specifically used for:

When the number of new fragments is greater than the total number of computer device nodes, sort each computing device node in ascending order according to the current storage usage rate of each computer device node represented by the storage usage summary table ;

According to the ascending sorting results of each computing device node, the first number of computer device nodes to be selected is screened from small to large.

Optionally, the determination module is also used for:

The first quantity is determined according to the total number of computer equipment nodes and a preset first distribution ratio.

Optionally, the determination module is also used for:

When the number of the new fragments is not greater than the total number of computer device nodes, it is judged whether the number of the new fragments is equal to 1;

When the number of new fragments is equal to 1, the computer device node with the lowest current storage utilization represented by the storage utilization summary table is determined as the candidate computer device node corresponding to the new fragment.

Optionally, the determination module is also used for:

When the number of the new fragments is not greater than the total number of computer equipment nodes, and the number of the new fragments is greater than 1, according to the storage usage summary table, screen the second number of new fragments corresponding to the pending Select a computer device node;

According to the current data storage capacity of each computer device node to be selected and the data storage capacity of the new fragment, update the storage usage rate of each computer device node to be selected, and other than the computer device node to be selected The storage utilization rate of the computer device node remains unchanged, so as to iteratively update the storage utilization rate summary table, and use the updated storage utilization rate summary table as a new storage utilization rate summary table;

Updating the number of new fragments by a second quantity, so as to iteratively update the number of new fragments, and return to the step of judging whether the number of new fragments is equal to 1.

Optionally, the determination module is specifically used for:

When the number of new fragments is not greater than the total number of computer equipment nodes, and the number of new fragments is greater than 1, the current storage usage of each computer equipment node represented by the storage usage summary table rate, sorting each computing device node in ascending order;

According to the ascending sorting results of each of the computing device nodes, the second number of computer device nodes to be selected is screened from small to large.

Optionally, the determination module is also used for:

The second quantity is determined according to the total amount of computer equipment nodes and a preset second distribution ratio.

Optionally, the acquiring module is specifically used for:

Acquiring the data storage space of each computer device node;

For any of the computer device nodes, the current storage usage rate of the computer device node is determined according to the current data storage capacity and data storage space of the computer device node.

The third aspect of the present application provides a search engine database, including: several computer equipment nodes;

Any one of the computer device nodes executes the method described in the above first aspect and various possible designs of the first aspect.

A fourth aspect of the present application provides an electronic device, including: at least one processor and a memory;

the memory stores computer-executable instructions;

The at least one processor executes the computer-executed instructions stored in the memory, so that the at least one processor executes the method described in the above first aspect and various possible designs of the first aspect.

The fifth aspect of the present application provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the above first aspect and the first Aspects of various possible designs of the described method.

The technical solution of the present application has the following advantages:

The present application provides a search engine database management method, device, and search engine database, the method including: obtaining the current data storage capacity of each computer equipment node corresponding to the search engine database, the current storage usage rate of each computer equipment node, and the new fragmentation according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments, determine the target computer device node; distribute the new fragments of the search engine database to all Describe the target computer device node. The method provided by the above solution, by comprehensively considering the current data storage capacity of each computer device node, the current storage usage rate and the number of new fragments, selects the target computer device node corresponding to the new fragment to ensure that the new fragment is allocated to the corresponding target computer After the device node, the data storage of each computer device node can be balanced, which improves resource utilization and also lays the foundation for improving the search efficiency of the ES database.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present application, and those skilled in the art can also obtain other drawings based on these drawings.

Fig. 1 is the structural representation of the search engine database management system based on the embodiment of the present application;

FIG. 2 is a schematic flow diagram of a search engine database management method provided by an embodiment of the present application;

FIG. 3 is a schematic flow diagram of an exemplary search engine database management method provided by an embodiment of the present application;

FIG. 4 is a schematic flowchart of another exemplary search engine database management method provided by the embodiment of the present application;

FIG. 5 is a schematic structural diagram of a search engine database management device provided in an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a search engine database provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

By means of the above drawings, specific embodiments of the present application have been shown, which will be described in more detail hereinafter. These drawings and written description are not intended to limit the scope of the disclosed concept in any way, but to illustrate the concept of the application for those skilled in the art by referring to specific embodiments.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

First, the nouns involved in this application are explained:

Fragmentation: Fragmentation is the key to distributing data in the cluster of ES databases. It is the smallest unit of work and stores part of the index data. Shards can be primary shards or replica shards.

In addition, the terms "first", "second", etc. are used for descriptive purposes only, and should not be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. In the descriptions of the following embodiments, "plurality" means two or more, unless otherwise specifically defined.

In the prior art, the computer device node list is usually sorted in ascending order according to the number of existing shards, and the computer device node that currently has the fewest number of shards is found, and then the newly created shard is assigned to the node that currently has the fewest number of shards. Computer device node. However, in the actual production environment, as the ES index is turned on and off, computer device nodes go online and offline, and data deletion occurs alternately, if the newly created shards are allocated based on the existing technology, it will cause each computer to The serious imbalance of data storage in the device nodes not only reduces the resource utilization rate, the online service quality declines or the business stops, but it is also not conducive to ensuring the search efficiency of the ES database.

In view of the above problems, the embodiment of the present application provides a search engine database management method, device and search engine database, the method includes: obtaining the current data storage capacity of each computer equipment node corresponding to the search engine database, the current storage usage of each computer equipment node rate and the number of new fragments; according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node and the number of new fragments, determine the target computer device node; the new fragmentation of the search engine database, Assigned to the target computer device node. The method provided by the above solution, by comprehensively considering the current data storage capacity of each computer device node, the current storage usage rate and the number of new fragments, selects the target computer device node corresponding to the new fragment to ensure that the new fragment is allocated to the corresponding target computer After the device node, the data storage of each computer device node can be balanced, which improves resource utilization and also lays the foundation for improving the search efficiency of the ES database.

The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below in conjunction with the accompanying drawings.

First, the structure of the search engine database management system on which this application is based is explained:

The search engine database management method, device, and search engine database provided by the embodiments of the present application are suitable for distributing new fragments to target computer equipment nodes when new fragments are added to the search engine database, so that the fragments of the search engine database are balanced distribute. As shown in Figure 1, it is a schematic structural diagram of the search engine database management system based on the embodiment of the present application, which mainly includes a data acquisition device and a search engine database. The search engine database includes several computer equipment nodes, and each computer equipment node is equipped with a search engine database. The engine database management device, the data acquisition device is used to collect the current data storage capacity of each computer equipment node corresponding to the search engine database, the current storage utilization rate of each computer equipment node and the number of new fragments, and send the collected information to The search engine database management device in any computer device node determines the target computer device node of the new fragment based on the search engine database management device and the obtained information.

The embodiment of the present application provides a search engine database management method, which is used for allocating new fragments to target computer device nodes when new fragments are added to the search engine database, so as to balance the distribution of fragments of the search engine database. The execution subject of the embodiment of the present application is an electronic device, such as a server, a desktop computer, a notebook computer, a tablet computer and other electronic devices that can be used to manage the search engine database.

As shown in Figure 2, it is a schematic flow chart of the search engine database management method provided by the embodiment of the present application, the method includes:

Step 201, acquiring the current data storage capacity of each computer device node corresponding to the search engine database, the current storage usage rate of each computer device node, and the number of new fragments.

Wherein, the current data storage capacity represents the total amount of data currently stored by the computer device node, and the current storage usage rate represents the storage space occupancy rate of the computer device node.

Specifically, in one embodiment, the data storage space of each computer device node can be obtained; for any computer device node, according to the current data storage capacity and data storage space of the computer device node, determine the current storage space of the computer device node usage rate.

Specifically, the ratio between the current data storage amount and the data storage space may be used as the current storage usage rate of the computer device node.

Step 202: Determine the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments.

Specifically, the storage space usage of each computer device node can be analyzed according to the current storage usage rate of each computer device node, and then combined with the current data storage capacity of each computer device node and the number of new fragments to be allocated, determine each new The target computer device node corresponding to the fragment, so that after the new fragment is allocated to the corresponding target computer device node, each computer device node involved in the search engine database can maintain a balance of data storage.

Step 203, assigning new fragments of the search engine database to target computer device nodes.

Specifically, after determining the target computer device node corresponding to each new segment, each new segment is sequentially allocated to the corresponding target computer device node, and the specific allocation method is not limited in this embodiment of the application.

On the basis of the above embodiments, in order to further ensure the reliability of the selected target computer equipment node, as an implementable manner, in one embodiment, according to the current data storage capacity of each computer equipment node, each computer equipment node The current storage usage and the number of new shards determine the target computer device node, including:

Step 2021, according to the current storage usage rate of each computer equipment node, determine the current balance evaluation index between each computer equipment node;

Step 2022, according to the current balance evaluation index among the computer equipment nodes, determine node selection constraints;

Step 2023: Determine the target computer device node according to the node selection constraints, according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments.

It should be noted that the balance evaluation index represents the degree of difference in data storage among computer equipment nodes.

Specifically, node selection constraints may be determined with the goal of reducing the degree of difference in data storage among computer device nodes.

Specifically, in one embodiment, in order to further ensure the objectivity of the balance evaluation index, the standard deviation of the current storage usage rate between each computer device node can be determined according to the current storage usage rate of each computer device node; the current storage usage rate The standard deviation of the rate is used as the current balance evaluation index among the computer equipment nodes.

Wherein, the specific calculation method of the standard deviation of the current storage usage rate may refer to the standard deviation calculation formula, which is not limited in this embodiment of the present application.

Specifically, in one embodiment, computer device nodes to be selected can be determined according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments; After the new fragment is allocated to the computer equipment node to be selected, the latest balance evaluation index among the computer equipment nodes; when the latest balance evaluation index is smaller than the current balance evaluation index, the computer equipment node to be selected is determined as the target computing equipment node.

Specifically, when the data storage capacity of the new fragment is known, for any target computer device node, the target computer can be predicted according to the data storage capacity of the new fragment and the current data storage capacity of the target computer device node. The storage usage rate after the device node is assigned to the new fragment, and then recalculate the balance evaluation index between each computer device node, that is, predict that after the new fragment of the search engine database is allocated to the computer device node to be selected, each computer device node The latest balance evaluation index between. Since the embodiment of the present application adopts the standard deviation of the current storage usage rate as the current balance evaluation index among the computer equipment nodes, when the latest balance evaluation index is smaller than the current balance evaluation index, the representation reduces the data storage between the computer equipment nodes. Therefore, the computer device node to be selected is determined as the target computing device node.

Specifically, in one embodiment, when the data storage capacity of the new fragment is unknown, the data storage capacity of the new fragment can be predicted according to the current data storage capacity of each computer device node; according to the current data storage capacity of each computer device node The storage usage rate, the number of new shards and the data storage capacity of the new shards are used to determine the computer device nodes to be selected.

Specifically, in one embodiment, the data storage capacity of new shards may be predicted according to the current data storage capacity of each computer device node and the current number of shards of each computer device node.

Specifically, in the case where the data storage capacity of each fragment in each computer device node can be obtained directly, the data storage capacity and the total amount of fragments included in the search engine database can be obtained, and then according to the data storage capacity of each fragment The amount of data storage is to determine the total amount of data storage in the search engine database, and then predict the data storage amount of the new shard according to the total amount of data storage in the search engine database and the total amount of shards.

Specifically, according to the current data storage capacity of each computer device node and the current number of fragments of each computer device node, the data storage capacity of the new fragment can be predicted, and then combined with the current storage usage rate of each computer device node, the new fragmentation The number and the data storage capacity of the new fragment determine the computer device node to be selected.

Specifically, in one embodiment, the average data storage capacity of a single fragment of the search engine database can be determined according to the current data storage capacity of each computer device node and the current number of fragments of each computer device node; Data storage amount, as the data storage amount of the new shard.

Similarly, in one embodiment, the geometric mean data storage capacity of a single fragment of the search engine database can be determined according to the current data storage capacity of each computer device node and the current fragmentation quantity of each computer device node; The geometric mean data storage capacity, as the data storage capacity of the new shard.

Among them, using the geometric mean data storage capacity as the data storage capacity of the new shard can reduce the influence of extreme values and improve the accuracy of the prediction result of the data storage capacity of the new shard.

On the basis of the above embodiments, in order to avoid the extreme situation where all new fragments are accumulated on a single node, which will lead to a decrease in search efficiency and hotspot risks, as an implementable method, in an embodiment , according to the current storage usage rate of each computer device node, the number of new fragments and the data storage capacity of the new fragments, determine the computer device nodes to be selected, including:

Step 301, constructing a storage usage summary table according to the current storage usage of each computer device node;

Step 302, judging whether the number of new fragments is greater than the total number of computer device nodes;

Step 303, when the number of new fragments is greater than the total number of computer equipment nodes, according to the current storage usage rate of each computer equipment node, screen the candidate computer equipment nodes corresponding to the first number of new fragments;

Step 304, according to the current data storage capacity of each candidate computer device node and the data storage capacity of the new fragment, update the storage usage rate of each candidate computer device node, the storage of other computer device nodes except the candidate computer device node The utilization rate remains unchanged, so as to iteratively update the storage utilization summary table, and use the updated storage utilization summary table as the new storage utilization summary table;

In step 305, the number of updated new fragments is reduced by the first quantity, so as to iteratively update the number of new fragments, and return to the step of judging whether the number of new fragments is greater than the total number of computer device nodes.

Specifically, in one embodiment, when the number of new fragments is greater than the total number of computer device nodes, each computing device node can be sorted in ascending order according to the current storage usage rate of each computer device node represented by the storage usage summary table ; According to the ascending sorting results of each computing device node, filter the first number of computer device nodes to be selected from small to large.

Specifically, firstly, according to the current storage usage rate of each computer device node, statistics are made on the storage usage rate of physical nodes on the search engine database to obtain a storage usage rate summary table, which at least includes the unique identifier of each computer device node And its corresponding current storage usage rate. When the number of new shards is greater than the total number of computer device nodes, the new shards can be iteratively allocated in rounds, and the first number of new shards can be determined in each round according to the storage usage summary table Corresponding to the computer equipment node to be selected, and the computer equipment node with a smaller storage utilization rate is preferentially selected as the computer equipment node to be selected, and each time a round of selection of the computer equipment node to be selected is completed, the storage utilization rate summary table is updated once, The storage usage rate of each computer device node recorded in the storage usage summary table is its latest storage usage rate. Among them, in order to avoid the extreme situation where all the new fragments are accumulated on a single node, the embodiment of the present application adopts a decentralized allocation method, that is, the candidate computer equipment nodes determined in each round correspond to the new fragments one by one.

Specifically, in an embodiment, the first quantity may be determined according to the total number of computer equipment nodes and a preset first distribution ratio.

Specifically, the calculation result of the product of the total number of computer device nodes and the preset first distribution ratio may be used as the first quantity.

Exemplarily, when the total number of computer device nodes is 5 and the preset first distribution ratio is 2/3, the first number is determined to be 3.

Specifically, in one embodiment, when the number of new fragments is not greater than the total number of computer device nodes, it is judged whether the number of new fragments is equal to 1; when the number of new fragments is equal to 1, the storage usage summary table The computer device node with the lowest current storage usage rate is determined as the candidate computer device node corresponding to the new slice.

Specifically, when the number of new shards is equal to 1 in the first round of allocation, or after several rounds of iterative allocation, the number of new shards is reduced to 1, the computer device node with the lowest current storage usage can be directly determined as the The candidate computer device node corresponding to the new shard.

Specifically, in one embodiment, when the number of new fragments is not greater than the total number of computer device nodes, and the number of new fragments is greater than 1, according to the storage usage summary table, filter the second number of new fragments corresponding to Computer equipment nodes to be selected; according to the current data storage capacity of each computer equipment node to be selected and the data storage capacity of new fragments, update the storage utilization rate of each computer equipment node to be selected, and other computer equipment except the computer equipment node to be selected The storage usage rate of the node remains unchanged, so as to iteratively update the storage usage summary table, and use the updated storage usage summary table as a new storage usage summary table; the number of updated new fragments is reduced by a second amount, To iteratively update the number of new shards, and return to the step of judging whether the number of new shards is equal to 1.

Specifically, in one embodiment, when the number of new fragments is not greater than the total number of computer device nodes, and the number of new fragments is greater than 1, the current storage usage rate of each computer device node represented by the storage usage summary table , sorting each computing device node in ascending order; according to the ascending sorting result of each computing device node, screen the second number of computer device nodes to be selected from small to large.

Specifically, when the first round of allocation, the number of new fragments is not greater than the total number of computer equipment nodes, and the number of new fragments is greater than 1, or after several rounds of allocation, the number of new fragments is reduced to no more than the total number of computer equipment nodes amount, and the number of new fragments is greater than 1, then according to the current storage utilization rate of each computer device node represented by the current storage utilization rate summary, the candidate computer device nodes corresponding to the second number of new fragments are screened, and the priority is selected A computer device node with a smaller storage usage rate is used as a candidate computer device node. Wherein, the second quantity is smaller than the first quantity.

Specifically, in an embodiment, the second quantity may be determined according to the total number of computer equipment nodes and a preset second allocation ratio.

Specifically, the calculation result of the product of the total number of computer device nodes and the preset second distribution ratio may be used as the second quantity.

Wherein, the preset second distribution ratio is smaller than the preset first distribution ratio. When the total number of computer equipment nodes is 5 and the preset first distribution ratio is 1/3, the first number is determined to be 2.

Exemplarily, as shown in FIG. 3 , it is a schematic flowchart of an exemplary search engine database management method provided by the embodiment of the present application. In practical applications, when the number of updated new shards is a negative number, it can be directly set to zero, and when the number of new shards is equal to 0, the process ends.

Specifically, as shown in FIG. 4 , it is a schematic flowchart of another exemplary search engine database management method provided by the embodiment of the present application. The search engine database management method provided by the embodiment of the present application involves the expected standard deviation calculation module, node The operation of algorithms such as selector and iterative allocator, among which, the expected standard deviation is the balance evaluation index, the size of a single fragment is the data storage capacity of the new fragment, the storage rate is the storage utilization rate of the computer equipment node, and the node selection The device is used to select the target computer device node based on the method provided by the above-mentioned embodiment, and the iterative allocator is used to determine the target computer device node to be selected corresponding to each new fragment based on the method provided by the above-mentioned embodiment, and the update node range refers to the computer device to be selected The number of nodes (first or second).

In order to facilitate those skilled in the art to understand more clearly the search engine database management method provided by the embodiment of the present application, the example provided in the embodiment of the present application is as follows:

example one

Set up an ElasticSearch cluster containing 5 storage nodes (computer equipment nodes), and the storage usage rates of the 5 storage nodes (numbered D1, D2, D3, D4, D5) are 10%, 15%, 20%, and 17% respectively and 3%; the storage size (current data storage capacity) is 10GB, 15GB, 20GB, 17GB, and 3GB respectively; then the standard deviation of the current storage usage rate (the current balance evaluation index) is 5.20%.

If the calculated geometric mean of the shards (the data storage capacity of the new shard) is 1.85GB, and the number of new shards to be added is 3, the allocation is as follows: the number of shards 3 is less than the number of nodes 5, then choose to store For the 1/3 nodes with the lowest usage rate, 5 times 1/3 is approximately rounded to 2, namely D5 and D1, and these 2 nodes are selected for shard allocation.

Iterative allocation calculations. Calculate the expected standard deviation (the latest balance evaluation indicator) according to the following table:

First round allocation:

Wherein, the expected storage usage represents the updated storage usage.

As shown in the above table, the standard deviation of the expected storage usage rate after the first round of allocation of the three shards is 4.31%, which is less than the current 5.20%, and the balance of data storage becomes better, so this round of allocation can be accepted. At this time, there is one remaining Fragmentation.

To continue the second round of allocation, directly select the D5 node for allocation. The expected distribution outcome for the second round is calculated as follows:

So far, the three shards have been allocated, and the standard deviation of the final expected usage rate is 3.87%, which is lower than the original standard deviation of 5.20%. The allocation can be accepted. The final shard allocation is: D5 adds 2 shards, D1 Added 1 shard.

Example two

Assume an ElasticSearch cluster containing 5 storage nodes, and the storage usage rates of the 5 storage nodes (numbered D1, D2, D3, D4, D5) are 10%, 15%, 20%, 17% and 3% respectively; The sizes are 10GB, 15GB, 20GB, 17GB, and 3GB; then the standard deviation of the current storage usage is 5.20%;

To add 12 shards, the geometric average size of a single shard is 3.1GB.

First, the first round of iterative allocation is performed:

If the number of shards 12 is greater than the number of nodes 5, the range of 2/3 nodes with the lowest storage usage should be selected first, and 5 multiplied by 2/3 is approximately rounded to 3, that is, D5, D1, and D2. The expected allocation is calculated as follows:

The expected standard deviation is reduced to 4.06%, and there are still 9 shards remaining, which are greater than the number of nodes 5. Continue to choose, and at this time choose D5, D1 and D3 for the second round of allocation:

The expected standard deviation is reduced to 3.22%, and there are still 6 shards remaining, which are greater than the number of nodes 5. Continue to choose, and at this time choose D5, D1 and D2 for the third round of allocation:

The expected standard deviation is reduced to 2.51%, and there are still 3 shards left, which are less than the number of nodes 5, so 2 nodes should be selected. At this time, D5 and D1 are selected for the fourth round of allocation:

The expected standard deviation is reduced to 1.77%, and there is only one shard left. Directly select D5 with the lowest storage usage rate for the last allocation:

So far, all 12 shards have been allocated, and the standard deviation of the final expected storage usage rate is reduced to 1.09%, which is far lower than the initial value of 5.20%. The expected imbalance of each node is reduced by 79%.

The search engine database management method provided in the embodiment of the present application obtains the current data storage capacity of each computer device node corresponding to the search engine database, the current storage usage rate of each computer device node, and the number of new fragments; according to each computer device node The current data storage capacity, the current storage usage rate of each computer device node and the number of new fragments determine the target computer device node; the new fragments of the search engine database are allocated to the target computer device node. The method provided by the above solution, by comprehensively considering the current data storage capacity of each computer device node, the current storage usage rate and the number of new fragments, selects the target computer device node corresponding to the new fragment to ensure that the new fragment is allocated to the corresponding target computer After the device node, the data storage of each computer device node can be balanced, which improves resource utilization and also lays the foundation for improving the search efficiency of the ES database. In addition, taking into account the degree of node dispersion, the secondary selection is used for iterative calculation to realize the decentralized allocation of new shards, avoiding the extreme situation where all new shards are piled up on a single node, and further improving the data storage balance of computer equipment nodes.

An embodiment of the present application provides a search engine database management device, which is used to execute the search engine database management method provided in the foregoing embodiments.

As shown in FIG. 5 , it is a schematic structural diagram of a search engine database management device provided in the embodiment of the present application. The search engine database management device 50 includes: an acquisition module 501 , a determination module 502 and a management module 503 .

Wherein, the obtaining module is used to obtain the current data storage capacity of each computer device node corresponding to the search engine database, the current storage usage rate of each computer device node and the number of new fragments; The current data storage capacity, the current storage usage rate of each computer device node and the number of new fragments determine the target computer device node; the management module is used to allocate the new fragments of the search engine database to the target computer device node.

Specifically, in an embodiment, the determination module is specifically used to:

According to the current storage usage rate of each computer equipment node, determine the current balance evaluation index between each computer equipment node;

According to the current balance evaluation index between each computer equipment node, determine the node selection constraints;

According to the node selection constraints, the target computer device node is determined according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments.

Specifically, in an embodiment, the determination module is specifically used to:

According to the current storage usage rate of each computer device node, determine the current storage usage rate standard deviation between each computer device node;

The standard deviation of the current storage usage rate is used as the current balance evaluation index among the computer equipment nodes.

Specifically, in an embodiment, the determination module is specifically used to:

According to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node and the number of new fragments, determine the computer device node to be selected;

Predict the latest balance evaluation index between each computer device node after the new fragmentation of the search engine database is allocated to the computer device node to be selected;

When the latest balance evaluation index is smaller than the current balance evaluation index, the computer device node to be selected is determined as the target computing device node.

Specifically, in an embodiment, the determination module is specifically used to:

According to the current data storage capacity of each computer device node, predict the data storage capacity of the new fragment;

According to the current storage usage rate of each computer device node, the number of new fragments and the data storage capacity of the new fragments, the computer device node to be selected is determined.

Specifically, in an embodiment, the determination module is specifically used to:

According to the current data storage capacity of each computer device node and the current fragmentation quantity of each computer device node, the data storage capacity of the new fragment is predicted.

Specifically, in an embodiment, the determination module is specifically used to:

According to the current data storage capacity of each computer device node and the current fragmentation quantity of each computer device node, determine the average data storage capacity of a single fragment of the search engine database;

Use the average data storage capacity of a single shard as the data storage capacity of the new shard.

Specifically, in an embodiment, the determination module is specifically used to:

According to the current data storage capacity of each computer device node and the current fragmentation quantity of each computer device node, determine the geometric mean data storage capacity of a single fragment of the search engine database;

The geometric mean data storage capacity of a single shard is used as the data storage capacity of the new shard.

Specifically, in an embodiment, the determination module is specifically used to:

According to the current storage utilization rate of each computer equipment node, construct a storage utilization rate summary table;

Determine whether the number of new fragments is greater than the total number of computer equipment nodes;

When the number of new fragments is greater than the total amount of computer equipment nodes, according to the storage usage summary table, screen the candidate computer equipment nodes corresponding to the first number of new fragments;

According to the current data storage capacity of each computer device node to be selected and the data storage capacity of new fragments, update the storage usage rate of each computer device node to be selected, and the storage usage rate of other computer device nodes except the computer device node to be selected is not Change to iteratively update the storage usage summary table, and use the updated storage usage summary table as the new storage usage summary table;

Updating the number of new shards by reducing the first amount, so as to iteratively update the number of new shards, and return to the step of judging whether the number of new shards is greater than the total number of computer device nodes.

Specifically, in an embodiment, the determination module is specifically used to:

When the number of new fragments is greater than the total number of computer device nodes, sort each computing device node in ascending order according to the current storage usage rate of each computer device node represented by the storage usage summary table;

According to the ascending sorting results of each computing device node, the first number of computer device nodes to be selected is screened from small to large.

Specifically, in an embodiment, the determining module is also used for:

The first quantity is determined according to the total number of computer equipment nodes and a preset first distribution ratio.

Specifically, in an embodiment, the determining module is also used for:

When the number of new fragments is not greater than the total number of computer equipment nodes, determine whether the number of new fragments is equal to 1;

When the number of new fragments is equal to 1, the computer device node with the lowest current storage utilization represented by the storage utilization summary table is determined as the candidate computer device node corresponding to the new fragment.

Specifically, in an embodiment, the determining module is also used for:

When the number of new fragments is not greater than the total number of computer equipment nodes, and the number of new fragments is greater than 1, according to the storage usage summary table, screen the candidate computer equipment nodes corresponding to the second number of new fragments;

According to the current data storage capacity of each computer device node to be selected and the data storage capacity of new fragments, update the storage usage rate of each computer device node to be selected, and the storage usage rate of other computer device nodes except the computer device node to be selected is not Change to iteratively update the storage usage summary table, and use the updated storage usage summary table as the new storage usage summary table;

Updating the number of new shards by reducing the second number, so as to iteratively update the number of new shards, and return to the step of judging whether the number of new shards is equal to 1.

Specifically, in an embodiment, the determination module is specifically used to:

When the number of new fragments is not greater than the total number of computer equipment nodes, and the number of new fragments is greater than 1, sort each computing equipment node in ascending order according to the current storage utilization rate of each computer equipment node represented by the storage utilization summary table ;

According to the ascending sorting results of each computing device node, the second number of computer device nodes to be selected is screened from small to large.

Specifically, in an embodiment, the determining module is also used for:

The second quantity is determined according to the total number of computer equipment nodes and the preset second distribution ratio.

Specifically, in an embodiment, the acquisition module is specifically used to:

Obtain the data storage space of each computer device node;

For any computer device node, the current storage usage rate of the computer device node is determined according to the current data storage capacity and data storage space of the computer device node.

Regarding the search engine database management device in this embodiment, the specific manner in which each module executes operations has been described in detail in the embodiment of the method, and will not be described in detail here.

The search engine database management device provided in the embodiment of the present application is used to implement the search engine database management method provided in the above embodiment, and its implementation method is the same as the principle, so it will not be repeated here.

An embodiment of the present application provides a search engine database, which is used to implement the search engine database management method provided in the foregoing embodiments.

As shown in FIG. 6 , it is a schematic structural diagram of the search engine database provided by the embodiment of the present application. The search engine database 60 includes: several computer device nodes 601 .

Any computer device node executes the search engine database management method provided in the above embodiments.

A search engine database provided in the embodiment of the present application is used to implement the search engine database management method provided in the above embodiment, and its implementation method is the same as the principle, so it will not be repeated here.

An embodiment of the present application provides an electronic device configured to execute the search engine database management method provided in the foregoing embodiments.

As shown in FIG. 7 , it is a schematic structural diagram of an electronic device provided in the embodiment of the present application. The electronic device 70 includes: at least one processor 71 and a memory 72 .

The memory stores computer-executable instructions; at least one processor executes the computer-executable instructions stored in the memory, so that at least one processor executes the search engine database management method provided in the above embodiments.

An electronic device provided in an embodiment of the present application is used to execute the method for managing a search engine database provided in the above embodiment, and the implementation method is the same as the principle, so details are not repeated here.

An embodiment of the present application provides a computer-readable storage medium, and computer-executable instructions are stored in the computer-readable storage medium. When the processor executes the computer-executable instructions, the search engine database management method provided in any one of the above embodiments is implemented.

The storage medium containing computer-executable instructions in the embodiment of the present application can be used to store the computer-executable instructions of the search engine database management method provided in the foregoing embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separated, and a component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

The above-mentioned integrated units implemented in the form of software functional units may be stored in a computer-readable storage medium. The above-mentioned software functional units are stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or a processor (processor) to execute the methods described in the various embodiments of the present application. partial steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes. .

Those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above-mentioned functional modules is used as an example for illustration. The internal structure of the system is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not repeated here.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present application. scope.

Claims (16)

1. A search engine database management method is characterized in that, comprising: Obtain the current data storage capacity of each computer device node corresponding to the search engine database, the current storage usage rate of each computer device node, and the number of new fragments; Determine the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments; Allocating the new fragment of the search engine database to the target computer device node; According to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node and the number of new fragments, determining the target computer device node includes: According to the current storage usage rate of each of the computer equipment nodes, determine the current balance evaluation index between each of the computer equipment nodes; Determine node selection constraints according to the current balance evaluation indicators among the computer equipment nodes; According to the node selection constraints, determine the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments; According to the node selection constraints, determining the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments, including: According to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node and the number of new fragments, determine the computer device node to be selected; Predicting the latest balance evaluation index between each computer device node after the new fragment of the search engine database is allocated to the computer device node to be selected; When the latest balance evaluation index is smaller than the current balance evaluation index, determine the computer device node to be selected as the target computer device node; According to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node and the number of new fragments, determining the computer device node to be selected includes: Predicting the data storage capacity of the new fragment according to the current data storage capacity of each of the computer device nodes; Determine the computer equipment node to be selected according to the current storage usage rate of each of the computer equipment nodes, the number of new fragments and the data storage capacity of the new fragments; According to the current storage usage rate of each of the computer equipment nodes, the number of new fragments and the data storage capacity of the new fragments, determining the computer equipment node to be selected includes: According to the current storage utilization rate of each described computer device node, construct a storage utilization rate summary table; Judging whether the number of the new fragments is greater than the total number of computer device nodes; When the number of the new fragments is greater than the total number of computer equipment nodes, according to the storage usage summary table, screen candidate computer equipment nodes corresponding to the first number of new fragments; According to the current data storage capacity of each computer device node to be selected and the data storage capacity of the new fragment, update the storage usage rate of each computer device node to be selected, and other than the computer device node to be selected The storage utilization rate of the computer device node remains unchanged, so as to iteratively update the storage utilization rate summary table, and use the updated storage utilization rate summary table as a new storage utilization rate summary table; Updating the number of new shards to reduce the first amount, so as to iteratively update the number of new shards, and return to the step of judging whether the number of new shards is greater than the total number of computer device nodes . 2. The method according to claim 1, wherein said determining the current balance evaluation index between each said computer device node according to the current storage utilization rate of each said computer device node comprises: According to the current storage usage rate of each of the computer device nodes, determine the current storage usage rate standard deviation between each of the computer device nodes; The standard deviation of the current storage usage rate is used as the current balance evaluation index among the computer device nodes. 3. The method according to claim 1, wherein the predicting the data storage capacity of the new fragment according to the current data storage capacity of each of the computer device nodes comprises: According to the current data storage capacity of each of the computer device nodes and the current number of fragments of each of the computer device nodes, the data storage capacity of the new fragment is predicted. 4. The method according to claim 3, characterized in that, predicting the data of the new fragmentation according to the current data storage capacity of each of the computer equipment nodes and the current fragmentation quantity of each of the computer equipment nodes Storage capacity, including: According to the current data storage capacity of each described computer device node and the current fragmentation quantity of each described computer device node, determine the average data storage capacity of a single fragment of the search engine database; The average data storage capacity of the single shard is used as the data storage capacity of the new shard. 5. The method according to claim 3, characterized in that, predicting the data of the new fragmentation according to the current data storage capacity of each of the computer equipment nodes and the current fragmentation quantity of each of the computer equipment nodes Storage capacity, including: According to the current data storage capacity of each described computer device node and the current fragmentation quantity of each described computer device node, determine the geometric mean data storage capacity of a single fragment of the search engine database; The geometric mean data storage capacity of the single shard is used as the data storage capacity of the new shard. 6. The method according to claim 1, wherein when the number of the new fragments is greater than the total number of nodes of the computer device, according to the storage usage summary table, the first number of new fragments is screened. The computer equipment nodes to be selected corresponding to the fragments, including: When the number of new fragments is greater than the total number of computer device nodes, sort each computer device node in ascending order according to the current storage usage rate of each computer device node represented by the storage usage summary table ; According to the ascending sorting results of each of the computer device nodes, the first number of computer device nodes to be selected is screened from small to large. 7. The method according to claim 1, further comprising: The first quantity is determined according to the total number of computer equipment nodes and a preset first distribution ratio. 8. The method according to claim 1, further comprising: When the number of the new fragments is not greater than the total number of computer device nodes, it is judged whether the number of the new fragments is equal to 1; When the number of new fragments is equal to 1, the computer device node with the lowest current storage utilization represented by the storage utilization summary table is determined as the candidate computer device node corresponding to the new fragment. 9. The method of claim 8, further comprising: When the number of the new fragments is not greater than the total number of computer equipment nodes, and the number of the new fragments is greater than 1, according to the storage usage summary table, screen the second number of new fragments corresponding to the pending Select a computer device node; According to the current data storage capacity of each computer device node to be selected and the data storage capacity of the new fragment, update the storage usage rate of each computer device node to be selected, and other than the computer device node to be selected The storage utilization rate of the computer device node remains unchanged, so as to iteratively update the storage utilization rate summary table, and use the updated storage utilization rate summary table as a new storage utilization rate summary table; Updating the number of new fragments by a second quantity, so as to iteratively update the number of new fragments, and return to the step of judging whether the number of new fragments is equal to 1. 10. The method according to claim 9, wherein when the number of the new fragments is not greater than the total number of nodes of the computer equipment, and the number of the new fragments is greater than 1, according to the A summary table of storage utilization ratios, filtering candidate computer device nodes corresponding to the second number of new fragments, including: When the number of new fragments is not greater than the total number of computer equipment nodes, and the number of new fragments is greater than 1, the current storage usage of each computer equipment node represented by the storage usage summary table rate, sort each computer device node in ascending order; According to the ascending sorting results of each of the computer equipment nodes, the second number of computer equipment nodes to be selected is screened from small to large. 11. The method of claim 9, further comprising: The second quantity is determined according to the total amount of computer equipment nodes and a preset second distribution ratio. 12. The method according to claim 1, wherein the obtaining the current storage utilization rate of each computer device node corresponding to the search engine database comprises: Acquiring the data storage space of each computer device node; For any of the computer device nodes, the current storage usage rate of the computer device node is determined according to the current data storage capacity and data storage space of the computer device node. 13. A search engine database management device, characterized in that it comprises: An acquisition module, configured to acquire the current data storage capacity of each computer device node corresponding to the search engine database, the current storage usage rate of each computer device node, and the number of new fragments; A determining module, configured to determine the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments; A management module, configured to assign new fragments of the search engine database to the target computer device node; The determination module is specifically used for: According to the current storage usage rate of each of the computer equipment nodes, determine the current balance evaluation index between each of the computer equipment nodes; Determine node selection constraints according to the current balance evaluation indicators among the computer equipment nodes; According to the node selection constraints, determine the target computer device node according to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node, and the number of new fragments; The determination module is specifically used for: According to the current data storage capacity of each computer device node, the current storage usage rate of each computer device node and the number of new fragments, determine the computer device node to be selected; Predicting the latest balance evaluation index between each computer device node after the new fragment of the search engine database is allocated to the computer device node to be selected; When the latest balance evaluation index is smaller than the current balance evaluation index, determine the computer device node to be selected as the target computer device node; The determination module is specifically used for: Predicting the data storage capacity of the new fragment according to the current data storage capacity of each of the computer device nodes; Determine the computer equipment node to be selected according to the current storage usage rate of each of the computer equipment nodes, the number of new fragments and the data storage capacity of the new fragments; The determination module is specifically used for: According to the current storage utilization rate of each described computer device node, construct a storage utilization rate summary table; Judging whether the number of the new fragments is greater than the total number of computer device nodes; When the number of the new fragments is greater than the total number of computer equipment nodes, according to the storage usage summary table, screen candidate computer equipment nodes corresponding to the first number of new fragments; According to the current data storage capacity of each computer device node to be selected and the data storage capacity of the new fragment, update the storage usage rate of each computer device node to be selected, and other than the computer device node to be selected The storage utilization rate of the computer device node remains unchanged, so as to iteratively update the storage utilization rate summary table, and use the updated storage utilization rate summary table as a new storage utilization rate summary table; Updating the number of new shards to reduce the first amount, so as to iteratively update the number of new shards, and return to the step of judging whether the number of new shards is greater than the total number of computer device nodes . 14. A search engine database, characterized in that it comprises: several computer equipment nodes; Any one of the computer equipment nodes executes the method according to any one of claims 1 to 12. 15. An electronic device, comprising: at least one processor and a memory; the memory stores computer-executable instructions; The at least one processor executes the computer-implemented instructions stored in the memory, causing the at least one processor to perform the method according to any one of claims 1 to 12. 16. A computer-readable storage medium, wherein computer-readable instructions are stored in the computer-readable storage medium, and when the processor executes the computer-executable instructions, the computer-readable storage medium according to any one of claims 1 to 12 is implemented. described method.

Images