CN110636122A - Distributed storage method, server, system, electronic device and storage medium - Google Patents

Abstract

The embodiment of the invention relates to the field of computer technology, and discloses a distributed storage method, server, system, electronic equipment and storage medium. In the present invention, the write request of the file is received; the weight coefficient of each cluster in the storage cluster array is obtained; the weight coefficient is generated according to the current storage resources of the cluster; according to the weight coefficient of each cluster, the hash algorithm is used to determine the the target cluster for ; write the file to the target storage subcluster. In this way, when the storage capacity needs to be expanded, without affecting the efficiency of file upload and download, it can avoid the reduction of storage cluster reading and writing performance due to data redistribution, and improve the performance experience of users reading and writing cloud files. At the same time, the overall architecture is based on The cluster is the unit of capacity expansion, which can also improve the operation and maintenance efficiency of the cloud storage system for later operation and maintenance personnel.

Description

Distributed storage method, server, system, electronic device and storage medium

technical field

The embodiments of the present invention relate to the field of computer technology, and in particular to a distributed storage method, server, system, electronic equipment, and storage medium.

Background technique

Cloud storage is an extension of the concept of cloud computing and a new concept developed. It mainly provides data storage and services to the outside world through the collection of a large number of different types of storage devices in the network such as cluster applications, network technology or distributed file systems through application software. A system of access functions ensures data security and saves storage space. Currently common distributed storage systems include GFS Google file system, Luster parallel distributed file system, Ceph distributed file system and GlusterFS network file system, etc. Among them, the open-source Ceph is a reliable, scalable, unified, and distributed storage system solution, especially driven by the open-source cloud computing management platform of OpenStack, making Ceph sought after by various Internet companies as soon as it enters the industry.

The resource scheduling method based on the Ceph distributed cluster provided by the prior art is mainly based on a single Ceph distributed cluster, and resource scheduling is performed by exchanging space for time through HDD hard drive consolidation and adding SSD solid-state drive cache. For example, by adjusting the strong consistency scheme for reading and writing of a single Ceph distributed cluster, and adjusting the number of copies and the selection algorithm of distributed nodes to optimize the read and write performance of a single Ceph cluster.

However, in the process of realizing the present invention, the inventor found that: the existing technology can only focus on the performance experience of a single Ceph distributed cluster user when reading and writing, and does not take into account how to ensure that the storage resources of a single Ceph distributed cluster are low. Resource expansion is performed on the premise of user experience in reading and writing performance.

Contents of the invention

The purpose of the embodiments of the present invention is to provide a distributed storage method, server, system, electronic equipment, and storage medium, so that when the storage capacity needs to be expanded, without affecting the efficiency of file upload and download, avoid This leads to a reduction in the read and write performance of the storage cluster and improves the user's performance experience in reading and writing cloud files. At the same time, the overall architecture uses the cluster as the unit of capacity expansion, which can also improve the operation and maintenance efficiency of the cloud storage system for later operation and maintenance personnel.

In order to solve the above technical problems, the embodiment of the present invention provides a distributed storage method, including: receiving a file write request; obtaining the weight coefficient of each cluster in the storage cluster array; the weight coefficient is generated according to the current storage resources of the cluster ; Determine the target cluster for writing the file according to the weight coefficient of each cluster; write the file into the target cluster.

The embodiment of the present invention also provides a server, including: a request receiving module, used to receive a file read or write request; a calculation module, used to obtain the weight coefficient of each cluster in the storage cluster array, and according to the weight coefficient , using a hash algorithm to determine the target cluster for writing the file; the writing module is used for writing the file into the target cluster.

Compared with the prior art, the embodiment of the present invention uses multiple distributed storage clusters to form a storage cluster array, and at the same time sets corresponding weight coefficients for each cluster according to the current storage resources of each cluster, so that the files are assigned with the weight coefficient It is randomly distributed to each cluster, so that storage resources can be expanded or scheduled without affecting the user's read and write performance experience, and it can also improve the operation and maintenance efficiency of the entire distributed storage system for later operation and maintenance personnel.

In addition, the value of the weight coefficient includes a default weight coefficient; wherein, the default weight coefficient is generated by the database management system in real time according to the current remaining storage capacity of the cluster or the network speed of the computer room where the cluster is located; the default The weight coefficient is positively correlated with the remaining storage capacity or the network speed. Through this method, the weight coefficient can not only be produced according to the current remaining storage capacity of the cluster, but also can adjust the file writing strategy considering the influence of the network speed of the computer room on the cloud storage file reading and writing speed, so as to improve the user experience. better.

In addition, the weight coefficients also include user weight coefficients; wherein the user weight coefficients are set by the user according to actual conditions and stored in the database management system; when obtaining the weight coefficients of each cluster in the storage cluster array, priority Obtain the user weight coefficient. Through this method, when the operation and maintenance personnel can customize the distribution of the files that need to be stored according to the actual situation, it can make the use of storage space more flexible and can cope with more scenarios.

In addition, the storage cluster array includes a primary cluster array and a standby cluster array; each cluster in the primary cluster array and the standby cluster array corresponds to each other; the target cluster is a cluster in the aforementioned primary cluster array. When the target cluster fails, write the file to the cluster in the standby cluster array corresponding to the target cluster. Through such a means, the reliability of the entire storage system can be enhanced, and the system can run stably and reliably even when accidents such as a power outage in the computer room occur.

In addition, the index information composed of the identification of the file and the identification of the target cluster is generated; the index information is stored in the index cluster; wherein, the index cluster is constructed by a high-speed storage medium. Building an index cluster through high-speed media can greatly improve the response speed of the server to users when reading cloud storage files, and significantly improve user experience.

Description of drawings

One or more embodiments are exemplified by pictures in the accompanying drawings, and these exemplifications are not intended to limit the embodiments.

FIG. 1 is a flowchart of a distributed storage method according to a first embodiment of the present invention;

FIG. 2 is a flowchart of a distributed storage method according to a second embodiment of the present invention;

FIG. 3 is a flowchart of a distributed storage method according to a third embodiment of the present invention;

4 is a structural diagram of a storage cluster array according to a third embodiment of the present invention;

5 is a structural diagram of a server according to a fourth embodiment of the present invention;

6 is a structural diagram of a distributed storage system according to a fifth embodiment of the present invention;

FIG. 7 is a structural diagram of an electronic device according to a sixth embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, various implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings. However, those of ordinary skill in the art can understand that, in each implementation manner of the present invention, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following implementation modes, the technical solution claimed in this application can also be realized. The division of the following embodiments is for the convenience of description, and should not constitute any limitation to the specific implementation of the present invention, and the various embodiments can be combined and referred to each other on the premise of no contradiction.

The first embodiment of the present invention relates to a distributed storage method, which is applied on a server. In this embodiment, the write request of the file is received; the weight coefficient of each cluster in the storage cluster array is obtained; the weight coefficient is generated according to the current storage resources of the cluster; and the target for writing the file is determined according to the weight coefficient of each cluster cluster; writes the file into said target cluster. By combining multiple distributed storage clusters into a storage cluster array, and setting corresponding weight coefficients for each cluster according to the current storage resources of each cluster, so that the files are distributed to each cluster with a random probability of the weight coefficient. Storage resources can be expanded or scheduled without affecting the user's reading and writing performance experience, and at the same time, it can also improve the operation and maintenance efficiency of the entire distributed storage system for later operation and maintenance personnel.

The implementation details of the distributed storage method of this embodiment will be described in detail below, and the following content is only the implementation details provided for easy understanding, and is not necessary for implementing this solution.

The distributed storage method in this embodiment is shown in Figure 1, specifically includes:

Step 101, receiving a file write request.

Specifically, the user first needs to send a file upload request to the proxy server based on the hypertext transfer protocol HTTP through the client to establish a communication connection with the proxy server. The request received by the proxy server includes the ID of the file. The above clients are the native clients of Ceph in various programming languages.

Step 102, acquiring weight coefficients of each cluster in the storage cluster array.

In this embodiment, the aforementioned cluster is a cluster built based on the Ceph distributed file system, and the storage cluster array is a cluster array composed of multiple Ceph distributed file system clusters. When the storage space of the cluster array needs to be expanded, a cluster is used as a unit to expand.

Specifically, in order to reasonably write files into distributed storage clusters and make full and balanced use of cluster storage resources, each cluster has a weight coefficient generated according to the specific conditions of its own current storage resources. The above storage resources include: the current remaining storage space of the cluster and the current network speed of the server room where the cluster is located.

In a specific application, the current storage resource information of each cluster is stored in a small relational database management system MySQL, and the system will update the data dynamically and in real time. The system generates the current weight of each cluster based on the current storage resource information of each cluster coefficient. After the proxy server receives the file request, it will immediately request the weight coefficients of each cluster from MySQL and cache them in the server.

To illustrate with an example:

Cluster ID Current remaining storage capacity β Weight coefficient α 1 20% 0.4 2 20% 0.4 3 10% 0.2

As shown in the above table, a storage cluster array includes three Ceph clusters with equal total storage space, but the current remaining storage capacity of each cluster is not equal. When the space usage of each cluster reaches a uniform distribution, we can set a simple formula for MySQL to generate the current weight coefficient of each cluster:

Based on the above example, when the storage space of the cluster array needs to be expanded, a new cluster "cluster 4" is added, and the remaining storage capacity of this cluster is 100%, so its weight coefficient

In another example, when the total storage space of each cluster in a storage cluster array is not equal, the operation and maintenance personnel still expect the written files to be evenly distributed in each cluster, then the current remaining specific capacity of each cluster can be used to calculate, rather than the ratio of the remaining storage capacity to the total storage capacity in the above example.

In another example, multiple Ceph clusters in this embodiment are deployed in different computer rooms to enhance the reliability of the entire storage system and avoid sudden reductions in file read and write performance caused by physical factors. For example, if the computer room where a certain cluster is located is blocked in the network, that is, the data read and write performance of the cluster decreases, MySQL can also reduce the weight coefficient of the cluster according to other rules or formulas set by the user, so as to ensure Users can have a good performance experience in file reading and writing, and it also reduces the maintenance burden of operation and maintenance personnel.

Step 103, according to the weight coefficients of each cluster, a random distribution algorithm is used to determine the target cluster for writing the file.

Step 104, write the file into the target cluster.

In this embodiment, the aforementioned random distribution algorithm is a hash algorithm. First, a hash value table is preset, which includes two types of data, hash value and node, where each hash value corresponds to a node. Based on the foregoing examples, each cluster corresponds to a different number of nodes, where the number of nodes is proportional to the size of the cluster weight coefficient. Assuming a scaling factor of 100, cluster 1 corresponds to 40 nodes, cluster 2 corresponds to 40 nodes, and cluster 3 corresponds to 20 nodes. If there is a new cluster, directly add 100 nodes corresponding to the new cluster. The algorithm takes the file ID as an input parameter, obtains a hash value, and then obtains a cluster number through table lookup, and takes it as the output result. In this way based on the random distribution algorithm combined with the weighting algorithm, it can be guaranteed that the distribution of written files in the storage space can conform to the weight coefficient of each cluster as a whole.

Specifically, input the file ID in the file writing request to the above hash algorithm, and the algorithm returns an address, which is the number of the cluster used to write the file, and then establishes a data channel between the client and the Ceph cluster, Transfer and write the file to the corresponding cluster. Since the hash algorithm is a random algorithm, MySQL must monitor the storage capacity of each cluster in real time and dynamically generate weight coefficients for each cluster to ensure the performance of files when they are written to the storage medium, and at the same time ensure that the capacity of each cluster is sufficient Hold the file.

It should be noted that, the above-mentioned examples in this embodiment are illustrations for easy understanding, and do not limit the technical solution of the present invention.

Compared with the prior art, this embodiment forms a Ceph distributed storage cluster array with multiple Ceph distributed storage clusters as a whole, which can avoid the problem that storage resource expansion may not be possible when a single Ceph distributed storage cluster has insufficient storage resources At the same time, since there is no need to expand the capacity of a single Ceph distributed storage cluster, there is no need to perform data rebalancing operations within a single Ceph distributed storage cluster, which achieves the effect of storage resource expansion that is not perceived by users, and improves the user experience; And, since the weighted weight is set for each Ceph distributed storage cluster in advance, when writing the file, only need to perform hash calculation according to the file ID and the weighted weight of each Ceph distributed storage cluster to obtain the file The specific write address of the distributed storage system further improves the response speed of the distributed storage system and the balance of file distribution in the cluster.

The second embodiment of the present invention relates to a distributed storage method, the process is shown in Figure 2, including:

Step 201, receiving a file write request. This step is similar to step 101 in the first embodiment of the present invention, and will not be repeated here.

Step 202, read the size of the file.

Step 203, acquiring the user weight coefficients of each cluster in the storage cluster array.

In this embodiment, the operation and maintenance personnel can set the weight coefficients of each cluster according to the needs of the actual situation and save them in MySQL. When there are both the weight coefficient generated by MySQL itself and the user weight coefficient set by the user in MySQL, the proxy server obtains the user weight coefficient as the weight coefficient used by the algorithm.

In practical applications, the volume of files uploaded by users to the cloud varies widely, and the volume of large files may be hundreds of times that of small files. Since each cluster is built from a storage medium with a small capacity, the remaining storage capacity of a certain cluster may not be enough to store a large file. Therefore, it is necessary to set an appropriate strategy to ensure that the storage space can be used reasonably. At this time, the user can set the weight coefficients of multiple schemes to adapt to files of various sizes. For example, when the remaining space capacity of cluster 1 in clusters 1, 2, and 3 is low, when a write request for a large file is received, the weight coefficient scheme for large-volume files is adopted.

Specifically, first set the cluster weight coefficient scheme for large-volume files, that is: set the weight coefficients of clusters that cannot save large files to 0, and then reasonably assign the weight coefficients to the clusters with sufficient capacity to store the file according to conventional rules. each cluster. The weight coefficient scheme for small-volume files, namely: normally assign the weight coefficient to all clusters according to the storage resources of each cluster.

The above example only divides the size of files into two cases. In practical applications, files can be divided into N categories according to file size, where N is a natural number greater than 2, so as to make the use of storage capacity more reasonable. When the proxy server receives a file writing request, it first obtains the volume of the file, and obtains an appropriate weight coefficient scheme according to the volume.

In another example, the read/write rate of the storage medium will decrease due to the increase of space usage, so the operation and maintenance personnel can set a threshold of space usage for each storage cluster according to the actual situation. When a cluster space When the utilization rate reaches the threshold, the weight coefficient of the cluster is automatically set to 0 to ensure that the read and write rate of each cluster is kept above a user-acceptable read and write rate level.

In another example, when a new cluster is added to the storage cluster array due to capacity expansion, the operation and maintenance personnel can set all the weight coefficients of the old cluster to 0, and then distribute the weight coefficients evenly to each new cluster to This quickly achieves the purpose of evenly storing data in each cluster, thereby making the storage space in the storage cluster array more reasonable in utilization.

Step 204, according to the weight coefficients of each cluster, a random distribution algorithm is used to determine the target cluster for writing the file. This step is similar to step 103 in the first embodiment of the present invention, and will not be repeated here.

Step 205, write the file into the target cluster. This step is similar to step 104 in the first embodiment of the present invention, and will not be repeated here.

The step division of the above various methods is only for the sake of clarity of description. During implementation, it can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. ; Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.

Compared with the existing technology, in this embodiment, the operation and maintenance personnel can set the weight coefficients of different schemes according to the actual situation, and customize the distribution of the files that need to be stored currently, so that the utilization of storage space is more flexible. It can handle more scenarios and improve user performance experience.

The third embodiment of the present invention relates to a distributed storage method, the process is shown in Figure 3, including:

Step 301, receiving a file write request. This step is similar to step 101 in the first embodiment of the present invention, and will not be repeated here.

Step 302, acquiring weight coefficients of each cluster in the storage cluster array. This step is similar to step 102 in the first embodiment of the present invention, and will not be repeated here.

Step 303, according to the weight coefficients of each cluster, a random distribution algorithm is used to determine the target cluster for writing the file. This step is similar to step 103 in the first embodiment of the present invention, and will not be repeated here.

Step 304, sending a write request to the target cluster, and judging whether the target cluster is normal. If yes, go to step 305; if not, go to step 306.

Step 305, write the file into the target cluster; Step 306, write the file into the standby cluster corresponding to the target cluster in the standby cluster array.

Specifically, as shown in FIG. 4 , the entire storage cluster array includes a primary cluster array and a standby cluster array. Each cluster in the primary cluster array has a corresponding standby cluster in the standby cluster array. When the proxy server uses the algorithm to determine the cluster ID of the target cluster through the ID of the file to be written, it first sends a write request to the cluster in the main cluster array corresponding to the cluster ID, and then waits for the cluster status returned by the cluster array management server information, if the cluster is currently running normally, write the file uploaded by the user to the cluster. If the cluster is currently in an abnormal state, send a file write request to the standby cluster corresponding to the cluster, and then write the file uploaded by the user to the standby cluster corresponding to the cluster.

In practical applications, the standby cluster array can not only be used as an emergency record when files are written, but also can be used as a mirror image of the primary cluster array when users need to read cloud files.

Specifically, after the file is written to the target cluster, when the clusters are idle, that is, when the data throughput of each cluster is at a low level, the cluster array management server will copy a file and save it to the corresponding target cluster. in the standby cluster. In this way, the reliability of cloud files can be improved without affecting the user's reading and writing experience. When a cluster fails to read, users can read the file mirror stored in the standby cluster.

Step 307, generating index information composed of a file ID and a cluster identifier, and storing the index information in the index cluster.

Specifically, an index cluster in which all storage media are high-speed storage media is set in the storage system, and the high-speed storage media include: solid-state drive SSD, dynamic random access memory. When the proxy server receives a user's request to read a cloud file, it first searches the index cluster for the corresponding cluster ID according to the file ID, and then transmits the file data to the user's client. Since the storage media in the index cluster are all high-speed storage media, the query process can be controlled in a very short time, providing users with a better reading experience. At the same time, when the operation and maintenance personnel maintain the distributed storage system, they can also obtain the specific storage address of each file more quickly, thereby improving the operation and maintenance efficiency.

Compared with the existing technology, in this embodiment, by setting a backup cluster array and an index cluster in the distributed storage system, the reliability of long-term storage of files can be improved, and the performance experience of users when reading files can be improved. It can also improve the operation and maintenance efficiency of operation and maintenance personnel.

The fourth embodiment of the present invention relates to a server, the structure of which is shown in Figure 5, including:

A request receiving module 501, configured to receive a file read or write request;

The calculation module 502 is used to obtain and cache the weight coefficients of each cluster in the storage cluster array, and determine the target cluster for writing the file according to the weight coefficient combined with the hash algorithm;

In one embodiment, after determining the target cluster, the computing module first sends a write request to the target cluster, and then judges whether the target cluster is running normally according to the feedback from the management server of the storage cluster array. When the target cluster is abnormal, a write request is sent to the standby cluster corresponding to the target cluster in the standby cluster array.

The writing module 503 is used for writing the file uploaded by the user into the target cluster determined by the algorithm.

In one embodiment, after the writing module writes the user file into the target cluster, it generates index information of the file according to the file ID and the cluster number of the target cluster, and saves the index information in the index cluster.

Compared with the existing technology, the server in this embodiment determines the specific storage address of the file through the hash algorithm combined with the weight coefficient, so that the overall distribution of the file conforms to the distribution of the weight coefficient, thereby fully and reasonably utilizing the distributed storage system The space capacity avoids the degradation of storage cluster read and write performance due to data redistribution, and improves the performance experience of users reading and writing cloud files.

It is worth mentioning that all the modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, or a part of a physical unit, or multiple physical units. Combination of units. In addition, in order to highlight the innovative part of the present invention, units that are not closely related to solving the technical problems proposed by the present invention are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.

The fifth embodiment of the present invention relates to a distributed storage system, the process of which is shown in Figure 6, including:

The server 601 involved in the fourth embodiment of the present invention;

The storage cluster array 602 constructed by N storage clusters is used to store files uploaded by users, where N is a natural number greater than 1.

In a specific example, the storage cluster array may include a primary cluster array and a standby cluster array, wherein the number of clusters in the primary cluster array is equal to and corresponds to the number of clusters in the standby cluster. When the server uses an algorithm to determine the cluster ID of the target cluster according to the ID of the file to be written, it first sends a write request to the cluster in the main cluster array corresponding to the cluster ID, and then waits for the cluster status information returned by the cluster array management server , if the cluster is currently running normally, write the file uploaded by the user to the cluster. If the cluster is currently in an abnormal state, send a file write request to the standby cluster corresponding to the cluster, and then write the file uploaded by the user to the standby cluster corresponding to the cluster.

In a specific example, the standby cluster array can not only be used as an emergency record when writing files, but also can be used as a mirror image of the primary cluster array when users need to read cloud files. After the file is written to the target cluster, when each cluster is idle, that is, when the data throughput of each cluster is at a low level, the cluster array management server will copy a mirror image of the file and save it to the standby cluster corresponding to the above target cluster . In this way, the reliability of cloud files can be improved without affecting the user's reading and writing experience. When a cluster fails to read, users can read the file mirror stored in the standby cluster.

The index cluster 603 is used to store index information of files.

Specifically, the storage media of the index cluster are all high-speed storage media, and the high-speed storage media include: a solid-state drive SSD and a dynamic random access memory. When the proxy server receives a user's request to read a cloud file, it first searches the index cluster for the corresponding cluster ID according to the file ID, and then transmits the file data to the user's client. Since the storage media in the index cluster are all high-speed storage media, the query process can be controlled in a very short time, providing users with a better reading experience. At the same time, when the operation and maintenance personnel maintain the distributed storage system, they can also obtain the specific storage address of each file more quickly, thereby improving the operation and maintenance efficiency.

The database management system 604 is configured to store and manage capacity usage information of each cluster in the storage cluster array, and dynamically calculate and store a default weight coefficient according to the capacity usage information.

In one embodiment, the database management system is MySQL, and operation and maintenance personnel can set the weight coefficients of each cluster according to actual needs and save them in MySQL. When MySQL has both the weight coefficient generated by MySQL itself and the user weight coefficient set by the user, the server preferentially obtains the user weight coefficient as the weight coefficient used by the algorithm.

Compared with the prior art, this embodiment forms a Ceph distributed storage cluster array with multiple Ceph distributed storage clusters as a whole, which can avoid the problem that storage resource expansion may not be possible when a single Ceph distributed storage cluster has insufficient storage resources At the same time, since there is no need to expand the capacity of a single Ceph distributed storage cluster, there is no need to perform data rebalancing operations within a single Ceph distributed storage cluster, achieving the effect of storage resource expansion that is not perceived by users. At the same time, when the current storage space usage of the cluster is not balanced, setting different weight coefficients for each cluster can make the file data of each cluster reach a balanced distribution after the next file is written.

It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, the second embodiment and the third embodiment, and this embodiment can be mutually compatible with the first embodiment, the second embodiment and the third embodiment Cooperate with implementation. The relevant technical details mentioned in the first embodiment, the second embodiment, and the third embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Correspondingly, the relevant technical details mentioned in this implementation manner can also be applied in the first implementation manner, the second implementation manner and the third implementation manner.

It is worth mentioning that all the modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, or a part of a physical unit, or multiple physical units. Combination of units. In addition, in order to highlight the innovative part of the present invention, units that are not closely related to solving the technical problems proposed by the present invention are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.

The sixth embodiment of the present invention relates to an electronic device. As shown in FIG. 7 , it includes at least one processor 701; and a memory 702 communicatively connected to at least one processor 701; The instructions executed by the processor 701 are executed by at least one processor 701, so that the at least one processor 701 can execute the distributed storage method in the first or second or third implementation manner.

Wherein, the memory 702 and the processor 701 are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors 701 and various circuits of the memory 702 together. The bus may also connect together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein. The bus interface provides an interface between the bus and the transceivers. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing means for communicating with various other devices over a transmission medium. The data processed by the processor 701 is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the processor 701 . The processor 701 is responsible for managing the bus and general processing, and may also provide various functions including timing, peripheral interface, voltage regulation, power management and other control functions. And the memory 702 may be used to store data used by the processor 701 when performing operations.

The seventh embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The above method embodiments are implemented when the computer program is executed by the processor.

That is, those skilled in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, the program is stored in a storage medium, and includes several instructions to make a device ( It may be a single-chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes.

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific examples for realizing the present invention, and in practical applications, various changes can be made to it in form and details without departing from the spirit and spirit of the present invention. scope.

Claims (10)

1. A distributed storage method, characterized in that, an application server includes:

receiving a write request of a file;

acquiring a weight coefficient of each cluster in a storage cluster array; the weight coefficient is generated according to the current storage resource of the cluster;

determining a target cluster for writing the file according to the weight coefficient of each cluster;

and writing the file into the target cluster.

2. The distributed storage method of claim 1,

the weight coefficients comprise default weight coefficients;

the default weight coefficient is generated by a database management system in real time according to the current residual storage capacity of the cluster and the network speed of a machine room where the cluster is located;

the default weight factor is positively correlated with the remaining storage capacity or the network speed.

3. The distributed storage method of claim 2,

the weight coefficients further comprise user weight coefficients; the user weight coefficient is set by a user according to actual conditions and is stored in the database management system;

when the weight coefficient of each cluster in the storage cluster array is obtained, judging whether the user weight coefficient exists or not;

and if so, acquiring the user weight coefficient.

4. The distributed storage method of claim 1,

the storage cluster array comprises a main cluster array and a standby cluster array;

each cluster in the main cluster array corresponds to each cluster in the standby cluster array;

the target cluster is a cluster in the main cluster array;

and when the target cluster fails, writing the file into a cluster in the standby cluster array corresponding to the target cluster.

5. The distributed storage method according to any one of claims 1 to 4, comprising, after said writing the file in the target cluster:

generating index information consisting of the identification of the file and the identification of the target cluster;

storing the index information into an index cluster; wherein the index cluster is built by a high-speed storage medium.

6. A server, comprising:

the request receiving module is used for receiving a writing request of a file;

the computing module is used for acquiring a weight coefficient of each cluster in the storage cluster array and determining the target cluster for writing the file according to the weight coefficient;

and the writing module is used for writing the file into the target cluster.

7. A distributed storage system, comprising:

the storage cluster array comprises N clusters, wherein N is a natural number greater than 1;

the server of claim 6, configured to store the user's uploaded file in a cluster of the storage cluster array.

8. The distributed storage system according to claim 7, further comprising:

the index cluster is used for storing index information of the files in the storage cluster array;

and the database management system is used for storing and managing the current storage resource information of each cluster in the storage cluster array and dynamically calculating and storing the default weight coefficient according to the storage resource information.

9. An electronic device, comprising:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to perform the distributed storage method of any of claims 1 to 5.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the distributed storage method of any one of claims 1 to 5.

Images