CN111399754B - Method and device for releasing storage space and distributed system - Google Patents
Method and device for releasing storage space and distributed system Download PDFInfo
- Publication number
- CN111399754B CN111399754B CN201910826429.9A CN201910826429A CN111399754B CN 111399754 B CN111399754 B CN 111399754B CN 201910826429 A CN201910826429 A CN 201910826429A CN 111399754 B CN111399754 B CN 111399754B
- Authority
- CN
- China
- Prior art keywords
- time point
- reference time
- target reference
- storage
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The embodiment of the invention provides a method, a device and a distributed system for releasing storage space. Wherein, a distributed system includes: the management server is used for selecting a target reference time point serving as a deleting basis from the recorded multiple reference time points when the distributed system is detected to meet the preset storage space release condition; acquiring the current time point of an auxiliary clock as an auxiliary time point; transmitting the target reference time point and the auxiliary time point to each storage server in the distributed system; a storage server for receiving a target reference time point and an auxiliary time point; correcting the target reference time point by utilizing a difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected target reference time point; and deleting the sub-object of which the local storage time point is matched with the target reference time point after deviation correction from the stored sub-objects. The scheme can improve the release accuracy of the storage space.
Description
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a method and apparatus for releasing a storage space, and a distributed system.
Background
In order to facilitate storage and management of mass data, more and more distributed systems store objects of the data, so as to achieve the effects of resource centralized management and distributed sharing. When storing an Object, after uploading a complete data, such as a folder or a video, etc., from the user's perspective to the storage system, the managed server acts as an Object and splits the Object into a plurality of sub-objects, and the sub-objects obtained by splitting are distributed to each storage server for storage. Since the distribution positions of the storage servers of the distributed system are different, the system time of the different storage servers may be different, so in order to ensure that the management server can notify each storage service with a uniform storage time in the space release process, an auxiliary clock capable of unifying the system time of each storage server is generally set. The method specifically comprises the following steps: the management server determines a unified time point according to the system time of all the storage servers, takes the unified time point as an initial time point of the auxiliary clock, and continuously increases the time point of the auxiliary clock according to a preset clock increasing rule. Each time the object storage is completed, the management server records the current time point of the auxiliary clock as a reference time point of the stored object, wherein the reference time point is a unified storage time point of each sub-object of the stored object stored in the storage server.
Based on the auxiliary clock, in the related art, when the storage space is released, the management server sends the target reference time point with the attack as the basis of deletion to each storage server. Correspondingly, each storage server takes the received target reference time point as a retrieval basis to retrieve the locally stored sub-object, and deletes the retrieved sub-object.
However, because of the large difference between the system time of some storage servers and other storage servers in the distributed system, the difference between the reference time points recorded by the management server and the corresponding local storage time points of the certain servers may be large, so that when the certain servers use the target reference time points to release space, the stored child objects may be deleted erroneously.
Therefore, how to reduce the erroneous deletion of the stored child objects and improve the accuracy of the release of the storage space is a problem to be solved.
Disclosure of Invention
The embodiment of the invention aims to provide a method, a device and a distributed system for releasing storage space, so as to realize the effects of reducing the erroneous deletion of units of stored objects and improving the releasing accuracy of the storage space. The specific technical scheme is as follows:
In a first aspect, embodiments of the present invention provide a distributed system, the system comprising:
the management server is used for selecting a target reference time point serving as a deleting basis from the recorded multiple reference time points when the distributed system is detected to meet a preset storage space release condition; acquiring the current time point of an auxiliary clock as an auxiliary time point; transmitting the target reference point in time and the auxiliary point in time to each storage server in the distributed system;
the storage server is used for receiving the target reference time point and the auxiliary time point; correcting the target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected target reference time point; and deleting the sub-object of which the local storage time point is matched with the target reference time point after deviation correction from the stored sub-objects.
In a second aspect, an embodiment of the present invention provides a method for freeing a storage space, which is applied to a storage server of a distributed system, and includes:
receiving a target reference time point and an auxiliary time point which are sent by a management server of the distributed system; the target reference time point is a target reference time point which is selected from a plurality of recorded reference time points and is used as a deleting basis when the management server detects that the distributed system meets a preset storage space release condition; the auxiliary time point is the current time point of the acquired auxiliary clock after the management server determines the target reference time point;
Correcting the target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected target reference time point;
and deleting the sub-object of which the local storage time point is matched with the target reference time point after deviation correction from the stored sub-objects.
In a third aspect, an embodiment of the present invention provides a method for releasing a storage space, which is applied to a management server of a distributed system, where the method includes:
when the distributed system is detected to meet a preset storage space release condition, selecting a target reference time point as a deleting basis from the recorded multiple reference time points;
acquiring the current time point of an auxiliary clock as an auxiliary time point;
and correcting the target reference time point and the auxiliary time point by using the difference value between the current system time of the storage server and the auxiliary time point after the storage server receives the target reference time point and the auxiliary time point, obtaining a corrected target reference time point, and deleting a sub-object matched with the corrected target reference time point from the stored sub-objects by using the storage server.
In a fourth aspect, an embodiment of the present invention provides a storage space releasing device, which is applied to a storage server of a distributed system, and the device includes:
the receiving module is used for receiving a target reference time point and an auxiliary time point which are sent by a management server of the distributed system; the target reference time point is a target reference time point which is selected from a plurality of recorded reference time points and is used as a deleting basis when the management server detects that the distributed system meets a preset storage space release condition; the auxiliary time point is the current time point of the acquired auxiliary clock after the management server determines the target reference time point;
the deviation rectifying module is used for rectifying the target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a rectified target reference time point;
and the deleting module is used for deleting the sub-object with the local storage time point matched with the target reference time point after deviation correction from the stored sub-objects.
In a fifth aspect, an embodiment of the present invention provides a storage space releasing device, which is applied to a management server of a distributed system, and the device includes:
The time point determining module is used for selecting a target reference time point serving as a deleting basis from the recorded multiple reference time points when the distributed system is detected to meet a preset storage space release condition; acquiring the current time point of an auxiliary clock as an auxiliary time point;
and the sending module is used for sending the target reference time point and the auxiliary time point to each storage server in the distributed system, so that the storage server rectifies the target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point after receiving the target reference time point and the auxiliary time point, obtains the rectified target reference time point, and deletes the sub-object of which the local storage time point is matched with the rectified target reference time point from the stored sub-objects.
In a sixth aspect, an embodiment of the present invention provides a storage server, including:
the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are in communication with each other through the bus; a memory for storing a computer program; and a processor for executing the program stored in the memory to implement the step of the method for releasing the storage space provided in the second aspect.
In a seventh aspect, an embodiment of the present invention provides a management server, including:
the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are in communication with each other through the bus; a memory for storing a computer program; and a processor for executing a program stored in the memory, and implementing the step of the method for releasing the storage space provided in the third aspect.
In an eighth aspect, an embodiment of the present invention provides a computer readable storage medium, including a storage server, in which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the method for freeing a storage space provided in the second aspect.
In a ninth aspect, an embodiment of the present invention provides a computer readable storage medium, including a management server, in which a computer program is stored, the computer program implementing the steps of the method for freeing a storage space provided in the third aspect when executed by a processor.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a schematic diagram of a distributed system according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for freeing a memory space according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for freeing a memory space according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a releasing device for a storage space according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a releasing device for a storage space according to another embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating a storage server according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a management server according to an embodiment of the present invention.
Detailed Description
In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The large difference between the system time of some storage servers and other storage servers in the distributed system may cause the large difference between the reference time points recorded by the management server and the local storage time points corresponding to the certain servers, so that when the certain servers use the target reference time points to release space, the stored sub-objects may be deleted by mistake. For example, a sub-object SOb1 of a certain object Ob1 is larger than a reference time point RT1 of the object Ob1 recorded by the management server at a local storage time point of the storage server, and the reference time point of the object Ob1 is determined as the deletion basis target reference time point. When the storage space is released, since the local storage time point of the sub-object SOb1 is greater than the target reference time point RT1, the sub-object SOb1 is not deleted, resulting in missing deletion of the sub-object SOb1 that should be deleted. Alternatively, the local storage time point of the memory server of the Sob2 of another object Ob2 is not equal to the reference time point RT2 of the object Ob2 recorded by the management server, and is less than or equal to the target reference time point RT1. Therefore, when the storage space is released, the sub-object Sob2 is deleted, resulting in erroneous deletion of the sub-object Sob2 that should not be deleted.
In order to reduce erroneous deletion of units of stored objects and improve the accuracy of releasing storage space, the embodiment of the invention provides a method, a device and a distributed system for releasing storage space. The execution subject of the method for releasing the storage space may be a storage server of the distributed system or a management server of the distributed system. For ease of understanding, a distributed system according to an embodiment of the present invention will be described first.
In specific applications, the distributed system provided by the embodiments of the present invention may be various. By way of example, the distributed system may be a distributed storage system, a cloud storage system, a video system employing a distributed architecture for data storage, and so forth. Any distributed system is within the scope of embodiments of the present invention and is not limited in this regard.
As shown in fig. 1, the distributed system according to an embodiment of the present invention may include:
a management server 101, configured to select a target reference time point as a deletion basis from a plurality of recorded reference time points when it is detected that the distributed system satisfies a preset storage space release condition; acquiring the current time point of an auxiliary clock as an auxiliary time point; -sending a target reference point in time, and an auxiliary point in time to each storage server in the distributed system;
A storage server 102 for receiving a target reference time point and an auxiliary time point; correcting the target reference time point by utilizing a difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected target reference time point; and deleting the sub-object with the local storage time matched with the target reference time point after deviation correction from the stored sub-objects.
In a specific application, either one of the management server 101 and the storage server 102 may be an electronic device capable of data processing. For example, the device may include a desktop computer, a portable computer, an internet television, an intelligent mobile terminal, a wearable intelligent terminal, a server, etc., which are not limited herein, and any electronic device capable of implementing the embodiments of the present invention falls within the protection scope of the embodiments of the present invention.
The number of storage servers 102 in the distributed system may be plural, and the number of management servers 101 may be plural or 1. If the number of the management servers 101 is plural, a management server cluster may be formed to improve management efficiency and reduce system anomaly risk caused by management server anomaly.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
In order to achieve reasonable layout and easy understanding, a method for freeing storage space according to an embodiment of the present invention is described below from the perspective of interaction between a management server and a storage server.
As shown in fig. 2, the method for releasing storage space according to an embodiment of the present invention is applied to the distributed system according to the embodiment of fig. 1, and the method may include:
s201, when the management server detects that the distributed system satisfies a preset storage space release condition, selecting a target reference time point as a deletion basis from the recorded multiple reference time points.
In a specific application, the detection manner of whether the distributed system meets the preset storage space release condition by the management server may be various. For example, the detection manner may be that when the management server detects that the storage space of the distributed system reaches a preset threshold, it is determined that the distributed system meets a preset storage space release condition. The detection time when the storage space reaches the preset threshold may be according to a preset period, when the object storage is performed, or when the object storage is completed. Alternatively, the detection manner may be that the management server detects whether an instruction about releasing the storage space input by the user is received, and if so, determines that the distributed system is detected to satisfy a preset storage space release condition.
Any detection method capable of determining whether the distributed system meets the preset storage space release condition can be used in the present invention, which is not limited in this embodiment.
When the distributed system stores the object, the storage server completes the storage of the sub-object of the object according to the storage task issued by the management server, and then reports the information about the completion of the storage to the management server. For example, the storage server may report the index information of the sub-object of the object that completes the storage to the management server, or the storage server may report the notification information that completes the storage to the management server. After receiving the information reported by the storage server, the management server acquires the current time point of the auxiliary clock and records the current time point as the reference time point of the stored object, wherein the reference time point is the unified storage time point of each sub-object of the stored object stored in the storage server. Therefore, the management server has recorded a plurality of reference time points when detecting that the distributed system satisfies a preset storage space release condition.
In order to perform the release of the storage space with the target reference time point in the subsequent steps S204 to S205, it is necessary to determine the target reference time point as a basis of deletion in step S201. The selection of the target reference point in time may be varied. For example, the smallest time point may be selected as the target reference time point from among the recorded plurality of reference time points; the smallest point in time corresponds to the earliest storage time. Alternatively, for example, a preset number of time points may be selected as target reference time points in order from small to large from among the recorded plurality of reference time points. Alternatively, for example, a specified time point may be selected as the target reference time point from among the recorded plurality of reference time points. For example, a specified time point entered by the user, or a preset specified time point, or the like.
In addition, in two exemplary descriptions selected according to the size of the reference time point, the management server can orderly record the reference time point so as to directly select from recorded reference time points when determining the target reference time point, thereby saving steps of comparing the sizes and sorting and improving the release efficiency of the storage space.
S202, the management server acquires the current time point of the auxiliary clock as an auxiliary time point.
The system time between the individual storage servers storing the sub-objects of the object may be different, while during the storage space release process the management server determines from the object's perspective whether to delete. Therefore, the management server cannot directly record the local storage time point of the sub-object in the storage server as a deletion basis, but needs to determine the uniform storage time of the sub-object of a certain object; so that the management server can inform each storage server to delete the corresponding sub-object using the unified storage time point. For this purpose, an auxiliary clock is usually provided which is capable of unifying the system times of the individual storage servers. The method specifically comprises the following steps: the management server determines a unified time point according to the system time of all the storage servers, takes the unified time point as an initial time point of the auxiliary clock, and continuously increases the time point of the auxiliary clock according to a preset clock increasing rule. For example, the unified time point determined by the management server is 20180910143056, i.e. 14 points of 2018, 09, 10, 30 minutes and 56 seconds, and the preset clock increment rule may be 1 increment for each second of the auxiliary clock.
The specific manner in which the management server determines a unified point in time according to the system time of all the storage servers may be various. For example, the management server may take the average of the system times of all the storage servers as a unified point in time. For example, the maximum and minimum system times among the system times of all storage servers are removed, and the remaining system times are averaged. Alternatively, the management server may take the system time with the largest number of system times among the system times of all the storage servers as a unified point in time, for example. Any manner in which the management server determines a uniform point in time according to the system time of all the storage servers may be used in the present invention, which is not limited in this embodiment.
Every time the object storage is completed, the management server records the current time point of the auxiliary clock as the reference time point of the stored object, the initial time point of the auxiliary clock is determined according to the system time of all the storage servers, and the difference between the system time of the storage servers and the reference time point is equivalent to the difference between the system time of the storage servers and the time point of the auxiliary clock. Therefore, for the case where the reference time of the object and the local storage time of the sub-object of the object are greatly different due to the large difference in system time between the respective storage servers, the current time point of the auxiliary clock can be acquired as the auxiliary time point. Thereby, the target reference time point can be rectified in the subsequent step S203 by using the difference value between the auxiliary time point and the current system time of the storage server.
S203, the management server transmits the target reference time point and the auxiliary time point to each storage server in the distributed system.
S204, the storage server rectifies the received target reference time point by utilizing the difference value between the current system time and the received auxiliary time point, and obtains the rectified target reference time point.
In step S203, the management server sends the target reference time point and the auxiliary time point to each storage server in the distributed system, and the storage server performs sub-object deletion by using the received time points, so that it is ensured that the management server only needs to issue the corresponding time points at one time no matter how many sub-objects are to be deleted. Compared with the method that the management server determines the sub-objects to be deleted and then sends each sub-object to be deleted as a task to the storage server in the distributed system, the method can reduce the interaction times of the management server and the storage server and improve the release efficiency of the storage space.
In step S204, the storage server may use the difference value between the current system time and the received auxiliary time point to correct the received target reference time point. For example, the storage server may calculate a difference between the current system time of the storage server itself and the received auxiliary time point, and adjust the target reference time point according to the obtained difference, to obtain the target reference time point after deviation correction. Alternatively, the storage server may calculate a difference between the current system time of the storage server itself and the received auxiliary time point, and adjust the target reference time point according to the obtained difference and a preset fluctuation threshold value, to obtain the target reference time point after deviation correction. For ease of understanding and rational layout, step S204 is described in detail in the following in an alternative embodiment.
S205, the storage server deletes the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects.
The local storage time point of the child object may be: when the storage server completes the storage of the child object, the system time of the storage server. For example, the storage server may record the system time of the storage server itself as the local storage time point of the child object when the storage of the child object is completed.
In particular applications, the matching of the local storage time point to the corrected target reference time point may be varied. For example, a local storage time point less than or equal to the target reference time point may be determined to match the target reference time point. At this time, the deleted sub-object is deleted according to the match smaller than or equal to, so that it can be ensured that the deleted sub-object is the earliest storage time among the stored sub-objects. The child object whose storage time is relatively earliest is more in line with the user's intention to delete than the child object whose storage time is relatively later. Alternatively, the storage server may search for a time period corresponding to the received local storage time point from a preset correspondence between the target reference time point and the time period, and determine the local storage time point belonging to the searched time period as matching the target reference time point. For example, the time period found may be: the time period 20180910143500 to 20180910150500 corresponding to the target reference time point 20180910143500 corresponds to a child object stored in 30 minutes from the target reference time point, which is deleted by the storage server. At this time, more sub-objects can be deleted in the primary deletion process, more storage space is released, the step of releasing the storage space again caused by insufficient release of the storage space is saved, and the release efficiency of the storage space is improved.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
Optionally, step S204 is as follows: correcting the target reference time point by using a difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected target reference time point, wherein the method specifically comprises the following steps of:
inputting the current system time, the auxiliary time point and the target reference time point of the storage server into a preset correction formula to obtain a corrected target reference time point; the preset correction formula comprises the following steps:
Ser_time=del_time+(cur_time-loop_time+protect_time);
the Ser_time is a target reference time point after deviation correction, del_time is a target reference time point, cur_time is the current system time of the storage server, loop_time is an auxiliary time point, and protection_time is a preset fluctuation threshold. Specifically, the management server determines a unified time point according to the system time of all the storage servers, takes the unified time point as an initial time point of the auxiliary clock, and maintains the time point of the auxiliary clock to continuously increment according to a preset clock increment rule. Each time the object storage is completed, the management server records the current time point of the auxiliary clock as a reference time point of the stored object, wherein the reference time point is a unified storage time point of each sub-object of the stored object stored in the storage server. Based on the auxiliary clock, when the storage space is released, the management server sends the target reference time point del_time with the attack as the basis of deletion to each storage server, and the time point of the auxiliary clock is taken as an auxiliary time point loop_time.
In a specific application, the preset fluctuation threshold may be set according to historical experience. For example, before the storage space release is performed on the 3 rd month 16 th 2018, the history data of the storage space release is analyzed, and there is a case that the child object should be deleted, because the local storage time is 1 larger than the target reference time point, which is not deleted. At this time, the preset fluctuation threshold value may be set to 1. Of course, the preset fluctuation threshold may be 0. The specific setting of the preset fluctuation threshold may be performed according to specific requirements, which is not limited in this embodiment. Compared with the condition that the preset fluctuation threshold is not set, the method and the device can adjust the size of the target reference time point after correction according to the condition that whether the specific storage space is released accurately or not, and improve the release accuracy of the storage space.
For example, the current system time of the storage server SS1 itself is 2018, 10, 15, 18, and 30 minutes at the auxiliary time point 2018, 10, and 15, 30 minutes at the target reference time point 2018, 9, 10, and 10, inputs a preset correction formula ser_time=del_time+ (cur_time-loop_time+protect_time) =2018, 10, 15, 30 minutes+ (2018, 10, 15, 2018, 10, 18, 0) =2018, 9, 10, 12, 30 minutes. That is, for the storage server SS1, the target reference time point after correction is 30 minutes at 12 points on 9 months 10 days 2018. Of course, for a storage server where there is no difference between the system time and the reference time, the target reference time point after the correction is equal to the target reference time point.
Optionally, in step S205 described above: after deleting the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects, the storage server in the storage space release method provided by the embodiment of the invention further comprises the following steps:
the storage server detects whether the storage server meets the preset storage space available condition or not; if yes, ending the release of the storage space, otherwise, adjusting the target reference time point according to the appointed increment to obtain the current target reference time point; correcting the current target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected current target reference time point; deleting the sub-object of which the local storage time point is matched with the current target reference time point after deviation correction from the stored sub-objects; and returning to the step of executing the storage server to detect whether the storage server meets the preset storage space available condition.
In a specific application, there may be a case where after the release of the storage space, the storage space of the storage server still cannot meet the storage requirement and needs to be released continuously. In order to ensure that the storage space can be released until the storage space meets the size required for storage, and to make sure that the deleted sub-objects are as few as possible, the storage server may detect whether the storage server itself meets the preset storage space availability condition after step S205. If so, indicating that the storage space has been freed to the extent that the storage requirement is met, there is no need to continue to free the storage space, and thus, the release of the storage space can be ended. If not, indicating that the storage space of the storage server still cannot meet the storage requirement, the release needs to continue, and thus the target reference point in time may be adjusted to select a child object to be deleted from among the undeleted child objects. Because the deleted sub-object is the sub-object whose storage time point is relatively earliest, when the storage space is continuously released, the target reference time point can be adjusted according to the designated increment, so that when the storage space is released by using the current target reference time point, the sub-object whose storage time point is relatively earliest in the un-deleted sub-object is deleted.
The preset storage space available condition may be various. For example, it may be that the storage space of the storage server is greater than or equal to a preset storage space threshold. Alternatively, it may be an example that an instruction about ending the release of the memory space sent by the management server is received. In addition, the instruction about ending the release of the storage space sent by the management server may specifically be an instruction input by the user. Any predetermined condition of available storage space may be used in the present invention, and the present embodiment is not limited thereto.
The storage server adjusts the target reference time point according to the specified increment to obtain the current target reference time point, which specifically includes: the storage server inputs the target reference time point and the designated increment into a preset increment formula to obtain the current target reference time point. The preset increasing formula is as follows: current target reference time point = target reference time point + specified delta.
For example, the storage server SS1 detects that the preset storage space availability condition is not satisfied, and adjusts the target reference time point 2018, 9, 10, 15, 30 minutes according to the specified increment of 1 hour, to obtain the current target reference time point 2018, 9, 10, 16, 30 minutes. Using the difference value between the current system time of the storage server itself 10 minutes at 10 months 10 days 10 and 15 minutes at 10 days 10 months 10 days 18 minutes at 2018 as the auxiliary time point for-3 hours, correcting the 30 minutes of the current target reference time point 2018, 9, 10, 16 days, and 30 minutes of the current target reference time point 2018, 9, 10, 13 days; and deleting the sub-object of which the local storage time point is matched with the current target reference time point after deviation correction from the stored sub-objects.
As shown in fig. 3, a flow of a method for freeing a storage space according to another embodiment of the present invention is applied to the distributed system according to the embodiment of fig. 3, and the method may include:
s301, when the management server detects that the distributed system meets a preset storage space release condition, selecting a target reference time point serving as a deleting basis from a plurality of recorded reference time points.
S302, the management server acquires the current time point of the auxiliary clock as an auxiliary time point.
S303, the management server transmits the target reference time point, and the auxiliary time point to each storage server in the distributed system.
S301 to S303 are the same steps as S201 to S203 in the embodiment of fig. 2 of the present invention, and are not described herein again, and detailed descriptions of the embodiment of fig. 2 of the present invention are omitted.
S304, the management server determines target index information serving as a deleting basis from the plurality of index information. Wherein the index information includes: an identification of the virtual resource pool in which the child object is stored, and/or an object identification.
In a specific application, there may be a problem that the local storage time of the sub-object of the object that does not need to be deleted matches the target reference time point after the correction at the storage server, resulting in that the sub-object of the object that does not need to be deleted is deleted; and the sub-object of the object to be deleted is not matched with the target reference time point after deviation correction at the local storage time of the storage server, so that the problem that the sub-object of the object to be deleted is not deleted is caused. In addition, when the storage server deletes the sub-object whose local storage time point matches the target reference time point, deletion is required from all the stored sub-objects, and the search range at the time of deletion is too large, which may cause a problem of decreasing the deletion efficiency.
In this regard, the management server may determine the target index information as a deletion basis from the plurality of index information, so that the subsequent storage server may reduce the search range in the deletion using the virtual resource pool identifier in the target index information in step S307, and ensure accurate deletion of the child object using the object identifier in the target index information. The determining manner of the target index information specifically may include: searching an object corresponding to the target reference time point from the pre-stored corresponding relation between the reference time point and the object, and determining the index information of the searched object as target index information.
In addition, in a specific application, in order to ensure that the granularity of the stored data is small, the data is relatively more accurate, and fine management of the data is realized, for example, video content can be stored as a plurality of sub-objects with the duration of seconds, so that management can be performed on the second level. To this end, the storage hierarchy of the child objects may specifically include: objects > strips- > units, an object may be divided into a plurality of strips, N (e.g. 5/10, etc.) units form a strip, all objects include a plurality of units, and sub-objects stored in the storage server are units. Also, in order to facilitate different preset storage space release conditions to be performed for different objects, multiple objects of the same type may be stored in the same virtual resource pool (socket). For example, the data stored in the virtual resource pool B1 may not be deleted, and the virtual resource pool identifier in the target index information may not be the identifier of the virtual resource pool B1; the object in the virtual resource pool B2 satisfies a preset storage space release condition, and the identifier of the virtual resource pool B2 may be included in the target index information.
Accordingly, according to the storage hierarchy, the index information of the sub-objects may specifically include: an identification of a virtual resource pool, an object identification, and a stripe identification. Based on this, when the storage space is released, the management server may determine the target index information as a deletion basis from the plurality of index information through the above step S304, so that the sub-object is deleted according to the target index information and the target reference time point in the subsequent step S307, thereby accurately deleting all the sub-objects under the target index information. For example, object1 in a certain virtual resource pool B1 needs to be deleted, and the stripes that constitute the object include stripe st1 and stripe st2; when the deletion of the sub-object is performed according to the target index information and the target reference time point, the units in the strip st1 having the identification of the object1 and the units in the strip st2 having the identification of the object1 may be all deleted for the virtual resource pool B1. At this time, compared with the case that no target index information is used as the deleting basis, when the local storage time of the sub-object of the object to be deleted is not matched with the corrected target reference time point in the storage server, the sub-object is deleted according to the target index information and the target reference time point, and the complete deletion of the object can be realized. Moreover, compared with the deletion only according to the virtual resource pool identifier and the target reference time point, the deletion of the child objects of the object which does not need to be deleted can be avoided, namely the deleted unit can be ensured to improve the release accuracy of the storage space.
In addition, the execution order of step S304 may be various. Illustratively, step S304 may be performed concurrently with step S302, may be performed prior to step S302, or may be performed after step S302.
S305, the management server sends the target index information to each storage server of the distributed system.
In a specific application, the execution sequence of step S305 may be various. Illustratively, step S305 may be performed concurrently with step S303, may be performed prior to step S303, or may be performed after step S303. When step S305 and step S303 are performed simultaneously, the information required for space release is issued uniformly, so that delay caused by multiple issues and occupation of processing resources of the management server by multiple interactions of the management server and the storage server can be reduced.
S306, the storage server rectifies the received target reference time point by utilizing the difference value between the current system time and the received auxiliary time point, and the rectified target reference time point is obtained.
S306 and S204 of the embodiment of FIG. 2 are the same steps, and are not repeated here, and detailed descriptions of the embodiment of FIG. 2 are omitted.
S307, the storage server deletes the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects by utilizing the target index information.
In a specific application, the storage server uses the target index information to delete the sub-object whose local storage time matches the target reference time point after deviation correction from the stored sub-objects. For example, the storage server may search for a sub-object satisfying the two parallel search conditions as a parallel search condition with the target index information and the local storage time matched with the rectified target reference time point, and delete the searched sub-object. For example, object1 in a certain virtual resource pool B1 needs to be deleted, and the strips that compose the object include strip st1 and strip st2; when the sub-object is deleted according to the target index information and the target reference time point, the unit in the strip st1 and the unit in the strip st2 which are matched with the target reference time point after deviation correction and belong to the virtual resource pool B1 and have the object1 mark can be searched for and deleted.
Or, for example, when the object identifier is included in the target index information, the storage server may first search the sub-object whose local storage time point matches with the corrected target reference time point, then screen the sub-object with the target index information in the search result, delete the screened sub-object, so as to ensure that the object to be deleted is completely deleted, the object not to be deleted can be completely stored, and the sub-object is not deleted by mistake. For example, object1 in a certain virtual resource pool B1 needs to be deleted, and the stripes that constitute the object include stripe st1 and stripe st2. The unit with the local storage time matched with the target reference time point after deviation correction can be searched, the sub-object in the strip st1 and the sub-object in the strip st2 which belong to the virtual resource pool B1 and are provided with the object1 identification are screened from the searched unit, and deletion is carried out. Of course, in the above two exemplary descriptions, if a certain class of object is not stored as a virtual resource pool, it may be directly searched according to the search condition in the exemplary description, and no search needs to be performed for the virtual resource pool.
Or, for example, when the target index information includes the identifier of the virtual resource pool, the storage server may first narrow the search range to a virtual resource pool having the identifier of the virtual resource pool, search for a sub-object whose local storage time point matches the target reference time point after the deviation in the virtual resource pool having the identifier of the virtual resource pool, and delete the searched sub-object. For example, an object1 needs to be deleted, and the strips that make up the object include strip st1 and strip st2. For the virtual resource pool B1, units whose local storage time matches the target reference time point after the correction can be retrieved, and sub-objects in the strip st1 and sub-objects in the strip st2 with the identification of the object1 are screened from the retrieved units, and deleted.
In the embodiment of fig. 3, the searching range in deleting is narrowed by the virtual resource pool identifier in the target index information, and the object to be deleted is ensured to be completely deleted and the sub-object of the object not to be deleted is not deleted by mistake by the object identifier in the target index information, so that the integrity of the object not to be deleted is ensured.
In addition, after determining the target index information, the management server may delete the target index information displayed to the user, or may indicate that the object corresponding to the target index information has been deleted when displaying the target index information to the user, so as to ensure that the index information is consistent with the object stored in the storage server. For example, the object index information is object1 and strip1, and object1 and strip1 may be deleted from the index information, or in the index information presented to the user, it is noted that object1 and strip1 have been deleted. At this time, the problem that the object corresponding to the index cannot be found due to the fact that the object is deleted and the index is not deleted can be avoided, and consistency of index information and storage conditions is guaranteed.
As shown in fig. 4, a storage space releasing device according to an embodiment of the present invention is applied to a storage server of a distributed system, and the device may include:
a receiving module 401, configured to receive a target reference time point and an auxiliary time point sent by a management server of the distributed system; the target reference time point is a target reference time point which is selected from a plurality of recorded reference time points and is used as a deleting basis when the management server detects that the distributed system meets a preset storage space release condition; the auxiliary time point is the current time point of the acquired auxiliary clock after the management server determines the target reference time point;
the deviation rectifying module 402 is configured to rectify the target reference time point by using a difference value between the current system time of the storage server and the auxiliary time point, so as to obtain a rectified target reference time point;
and the deleting module 403 is configured to delete, from the stored sub-objects, a sub-object whose local storage time point matches the target reference time point after the correction.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
Optionally, the deviation rectifying module 402 is specifically configured to:
inputting the current system time of the storage server, the auxiliary time point and the target reference time point into a preset correction formula to obtain a corrected target reference time point; the preset correction formula comprises the following steps:
Ser_time=del_time+(cur_time-loop_time+protect_time);
the ser_time is a target reference time point after deviation correction, the del_time is a target reference time point, the cur_time is the current system time of the storage server, the loop_time is the auxiliary time point, and the protection_time is a preset fluctuation threshold.
Optionally, the receiving module 401 is further configured to:
receiving target index information sent by the management server; the target index information is index information which is determined by the management server from a plurality of index information and is used as a deleting basis; the index information includes: the identification of the virtual resource pool of the child object is stored, and/or the object identification is stored;
the deletion module 403 is specifically configured to:
and deleting the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects by utilizing the target index information.
Optionally, the device further includes: a detection module;
the detection module is used for detecting whether the storage server meets a preset storage space available amount condition or not after deleting the sub-object with the local storage time point matched with the target reference time point after deviation correction from the stored sub-objects; if yes, ending the release of the storage space, otherwise, triggering the deviation rectifying module 402 to adjust the target reference time point according to the specified increment to obtain the current target reference time point;
the deviation rectifying module 402 is further configured to rectify the current target reference time point by using a difference value between the current system time of the storage server and the auxiliary time point, to obtain a rectified current target reference time point;
the deleting module 403 is further configured to delete, from the stored sub-objects, a sub-object whose local storage time point matches the corrected current target reference time point, and trigger the detecting module to execute the detection of whether the storage server itself meets a preset storage space availability condition.
As shown in fig. 5, a storage space releasing device according to an embodiment of the present invention is applied to a management server of a distributed system, and the device may include:
A time point determining module 501, configured to select, when it is detected that the distributed system meets a preset storage space release condition, a target reference time point as a deletion basis from a plurality of recorded reference time points; acquiring the current time point of an auxiliary clock as an auxiliary time point;
and the sending module 502 is configured to rectify the target reference time point and the auxiliary time point by using a difference value between the current system time of the storage server and the auxiliary time point after the storage server receives the target reference time point and the auxiliary time point, obtain a rectified target reference time point, and delete a sub-object that matches the local storage time point with the rectified target reference time point from the stored sub-objects.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
Optionally, the apparatus further includes: an index information determining module;
the index information determining module is used for determining target index information serving as a deleting basis from a plurality of index information, and sending the target index information to each storage server in the distributed system, so that the storage server deletes the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-object by utilizing the target index information after receiving the target index information;
wherein the index information includes: an identification of the virtual resource pool in which the child object is stored, and/or an object identification.
Corresponding to the above embodiment, the embodiment of the present invention further provides a storage server, as shown in fig. 6, where the storage server may include:
processor 601, communication interface 602, memory 603 and communication bus 604, wherein processor 601, communication interface 602, memory 603 complete the communication each other through communication bus 604;
a memory 603 for storing a computer program;
the processor 601 is configured to implement the steps of the method for releasing the storage space applied to the storage server in any of the embodiments described above when executing the computer program stored in the memory 603.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
The embodiment of the invention also provides a management server, as shown in fig. 7, the management server may include:
a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702 and the memory 703 communicate with each other through the communication bus 704;
a memory 703 for storing a computer program;
the processor 701 is configured to implement the steps of the method for releasing the storage space applied to the management server in any of the above embodiments when executing the computer program stored in the memory 703.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
The Memory may include RAM (Random Access Memory ) or NVM (Non-Volatile Memory), such as at least one magnetic disk Memory. Optionally, the memory may be at least one memory device located remotely from the processor.
The processor may be a general-purpose processor, including a CPU (Central Processing Unit ), NP (Network Processor, network processor), etc.; but also DSP (Digital Signal Processor ), ASIC (Application Specific Integrated Circuit, application specific integrated circuit), FPGA (Field-Programmable Gate Array, field programmable gate array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
An embodiment of the present invention provides a computer readable storage medium, including a storage server, where a computer program is stored in the computer readable storage medium, and when the computer program is executed by a processor, the steps of the method for releasing a storage space applied to the storage server in any of the embodiments described above are implemented.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
An embodiment of the present invention provides a computer readable storage medium, including a management server, where the computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for releasing a storage space applied to the management server in any of the embodiments described above are implemented.
In the scheme provided by the embodiment of the invention, the initial time point of the auxiliary clock is obtained according to the system time of all the storage servers, and the reference time point serving as the deleting basis is the time point of the auxiliary clock; and, the local storage time point of the child object in the storage server is the system time of the storage server when the storage of the child object is completed. Therefore, the difference value between the current system time of the storage server and the current time point of the auxiliary clock serving as the auxiliary time point can reflect the difference value between the target reference time point and the local storage time point of the sub-object and can be used for correcting the deviation of the target reference time point; therefore, compared with the target reference time point, the target reference time point after deviation correction is ensured, and the difference between the system time of the storage server and the target reference time point is corrected. Therefore, when the storage space is released, the stored sub-object is deleted by using the target reference time point after correction, so that the difference value between the target reference time point after correction and the local storage time point of the sub-object can be ensured to be smaller than the difference value between the target reference time point and the local storage time point of the sub-object, thereby reducing the deletion error of the sub-object caused by the larger difference between the target reference time point and the local storage time point of the sub-object in the storage server, and improving the release accuracy of the storage space.
In yet another embodiment of the present invention, a computer program product containing instructions that, when executed on a storage server, cause the storage server to perform the method of releasing storage space applied to the storage server in any of the embodiments described above is also provided.
In a further embodiment of the present invention, a computer program product comprising instructions that, when executed on a management server, cause the management server to perform the method of any of the embodiments described above, applied to the storage space of the management server, is also provided.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, fiber optic, DSL (Digital Subscriber Line), or wireless (e.g., infrared, radio, microwave, etc.) means, the computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, data center, etc., that contains an integration of one or more available media, the available media may be magnetic media (e.g., floppy Disk, hard Disk, magnetic tape), optical media (e.g., DVD (Digital Versatile Disc, digital versatile Disk)), or semiconductor media (e.g., SSD State, solid State Disk), etc.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the device, storage server, management server and system embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, with reference to the partial description of the method embodiments being relevant.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (11)
1. A distributed system, the system comprising:
the management server is used for selecting a target reference time point serving as a deleting basis from the recorded multiple reference time points when the distributed system is detected to meet a preset storage space release condition; acquiring the current time point of an auxiliary clock as an auxiliary time point; transmitting the target reference point in time and the auxiliary point in time to each storage server in the distributed system; the setting mode of the time point of the auxiliary clock comprises the following steps: determining a unified time point according to the system time of each storage server, taking the unified time point as an initial time point of an auxiliary clock, and maintaining the time point of the auxiliary clock to continuously increment according to a preset clock increment rule; the reference time point is determined in the following manner: every time the object storage is completed, the management server records the current time point of the auxiliary clock as the reference time point of the stored object; the reference time point is a unified storage time point of each sub-object of the stored object stored in the storage server;
The storage server is used for receiving the target reference time point and the auxiliary time point; correcting the target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected target reference time point; deleting the sub-object of which the local storage time point is matched with the target reference time point after deviation correction from the stored sub-objects;
the management server is also adapted to provide,
determining target index information serving as a deleting basis from a plurality of index information, and sending the target index information to each storage server of the distributed system; wherein the index information includes: the identification of the virtual resource pool with the sub-objects stored therein, and/or the object identification; the determining mode of the target index information comprises the following steps: searching an object corresponding to the target reference time point from the pre-stored corresponding relation between the reference time point and the object, and determining index information of the searched object as target index information;
the storage server is further used for receiving the target index information;
the storage server deletes the sub-object with the local storage time point matched with the target reference time point after deviation correction from the stored sub-objects, and the method comprises the following steps:
And deleting the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects by the storage server by utilizing the target index information.
2. The system of claim 1, wherein the storage server rectifies the target reference time point using a difference value between a current system time of the storage server itself and the auxiliary time point, to obtain a rectified target reference time point, comprising:
inputting the current system time of the storage server, the auxiliary time point and the target reference time point into a preset correction formula to obtain a corrected target reference time point; the preset correction formula comprises the following steps:
Ser_time=del_time+(cur_time-loop_time+protect_time);
the ser_time is a target reference time point after deviation correction, the del_time is a target reference time point, the cur_time is the current system time of the storage server, the loop_time is the auxiliary time point, and the protection_time is a preset fluctuation threshold.
3. The system of any of claims 1 to 2, wherein the storage server is further configured to:
After deleting the sub-object with the local storage time point matched with the target reference time point after deviation correction from the stored sub-objects, detecting whether the storage server meets a preset storage space available condition or not;
if yes, ending the release of the storage space, otherwise, adjusting the target reference time point according to the appointed increment to obtain the current target reference time point; correcting the current target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected current target reference time point; deleting the sub-object of which the local storage time point is matched with the current target reference time point after deviation correction from the stored sub-objects; and returning to execute the detection on whether the storage server meets the preset storage space available condition.
4. A method of freeing storage space for a storage server of a distributed system, the method comprising:
receiving a target reference time point and an auxiliary time point sent by a management server of the distributed system, and receiving target index information sent by the management server; the target reference time point is a target reference time point which is selected from a plurality of recorded reference time points and is used as a deleting basis when the management server detects that the distributed system meets a preset storage space release condition; the auxiliary time point is the current time point of the acquired auxiliary clock after the management server determines the target reference time point; the target index information is index information which is determined by the management server from a plurality of index information and is used as a deleting basis; the index information includes: the identification of the virtual resource pool of the child object is stored, and/or the object identification is stored; the determining mode of the target index information comprises the following steps: searching an object corresponding to the target reference time point from the pre-stored corresponding relation between the reference time point and the object, and determining index information of the searched object as target index information; the setting mode of the time point of the auxiliary clock comprises the following steps: determining a unified time point according to the system time of each storage server, taking the unified time point as an initial time point of an auxiliary clock, and maintaining the time point of the auxiliary clock to continuously increment according to a preset clock increment rule; the reference time point is determined in the following manner: every time the object storage is completed, the management server records the current time point of the auxiliary clock as the reference time point of the stored object; the reference time point is a unified storage time point of each sub-object of the stored object stored in the storage server;
Correcting the target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected target reference time point;
deleting the sub-object of which the local storage time point is matched with the target reference time point after deviation correction from the stored sub-objects;
the deleting the sub-object with the local storage time matched with the target reference time point after deviation correction from the stored sub-objects comprises the following steps:
and deleting the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects by utilizing the target index information.
5. The method according to claim 4, wherein the step of correcting the target reference time point by using a difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected target reference time point includes:
inputting the current system time of the storage server, the auxiliary time point and the target reference time point into a preset correction formula to obtain a corrected target reference time point; the preset correction formula comprises the following steps:
Ser_time=del_time+(cur_time-loop_time+protect_time);
The ser_time is a target reference time point after deviation correction, the del_time is a target reference time point, the cur_time is the current system time of the storage server, the loop_time is the auxiliary time point, and the protection_time is a preset fluctuation threshold.
6. The method of any of claims 4 to 5, wherein after deleting the sub-object whose local storage time point matches the rectified target reference time point from the stored sub-objects, the method further comprises:
detecting whether the storage server meets a preset storage space available condition or not;
if yes, ending the release of the storage space, otherwise, adjusting the target reference time point according to the appointed increment to obtain the current target reference time point;
correcting the current target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a corrected current target reference time point;
deleting the sub-object with the local storage time point matched with the corrected current target reference time point from the stored sub-objects, and returning to execute the detection of whether the storage server meets the preset storage space available condition.
7. A method of freeing storage space, characterized by being applied to a management server of a distributed system, the method comprising:
when the distributed system is detected to meet a preset storage space release condition, selecting a target reference time point as a deleting basis from the recorded multiple reference time points; the reference time point is determined in the following manner: every time the object storage is completed, the management server records the current time point of the auxiliary clock as the reference time point of the stored object; the reference time point is a unified storage time point of each sub-object of the stored object stored in the storage server;
acquiring the current time point of an auxiliary clock as an auxiliary time point; the setting mode of the time point of the auxiliary clock comprises the following steps: determining a unified time point according to the system time of each storage server, taking the unified time point as an initial time point of an auxiliary clock, and maintaining the time point of the auxiliary clock to continuously increment according to a preset clock increment rule;
determining target index information serving as a deleting basis from a plurality of index information, and sending the target index information to each storage server in the distributed system, so that the storage server corrects the target reference time point by utilizing a difference value between the current system time of the storage server and the auxiliary time point after receiving the target reference time point and the auxiliary time point, and deleting a sub-object of which the local storage time point is matched with the corrected target reference time point from stored sub-objects; the deleting the sub-object with the local storage time point matched with the target reference time point after deviation correction from the stored sub-objects comprises the following steps: after receiving the target index information, the storage server deletes the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects by utilizing the target index information;
Wherein the index information includes: the identification of the virtual resource pool with the sub-objects stored therein, and/or the object identification; the determining mode of the target index information comprises the following steps: searching an object corresponding to the target reference time point from the pre-stored corresponding relation between the reference time point and the object, and determining the index information of the searched object as target index information.
8. A storage space release device for a storage server of a distributed system, the device comprising:
the receiving module is used for receiving a target reference time point and an auxiliary time point which are sent by a management server of the distributed system, and receiving target index information sent by the management server; the target reference time point is a target reference time point which is selected from a plurality of recorded reference time points and is used as a deleting basis when the management server detects that the distributed system meets a preset storage space release condition; the auxiliary time point is the current time point of the acquired auxiliary clock after the management server determines the target reference time point; the target index information is index information which is determined by the management server from a plurality of index information and is used as a deleting basis; the index information includes: the identification of the virtual resource pool of the child object is stored, and/or the object identification is stored; the determining mode of the target index information comprises the following steps: searching an object corresponding to the target reference time point from the pre-stored corresponding relation between the reference time point and the object, and determining index information of the searched object as target index information; the setting mode of the time point of the auxiliary clock comprises the following steps: determining a unified time point according to the system time of each storage server, taking the unified time point as an initial time point of an auxiliary clock, and maintaining the time point of the auxiliary clock to continuously increment according to a preset clock increment rule; the reference time point is determined in the following manner: every time the object storage is completed, the management server records the current time point of the auxiliary clock as the reference time point of the stored object; the reference time point is a unified storage time point of each sub-object of the stored object stored in the storage server;
The deviation rectifying module is used for rectifying the target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point to obtain a rectified target reference time point;
the deleting module is used for deleting the sub-object with the local storage time point matched with the target reference time point after deviation correction from the stored sub-objects;
the deleting module is specifically configured to:
and deleting the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects by utilizing the target index information.
9. The device according to claim 8, wherein the deviation rectifying module is specifically configured to:
inputting the current system time of the storage server, the auxiliary time point and the target reference time point into a preset correction formula to obtain a corrected target reference time point; the preset correction formula comprises the following steps:
Ser_time=del_time+(cur_time-loop_time+protect_time);
the ser_time is a target reference time point after deviation correction, the del_time is a target reference time point, the cur_time is the current system time of the storage server, the loop_time is the auxiliary time point, and the protection_time is a preset fluctuation threshold.
10. The apparatus according to any one of claims 8 to 9, further comprising: a detection module;
the detection module is used for detecting whether the storage server meets a preset storage space available amount condition or not after deleting the sub-object with the local storage time point matched with the target reference time point after deviation correction from the stored sub-objects; if yes, ending the release of the storage space, otherwise, triggering the deviation rectifying module to adjust the target reference time point according to the appointed increment to obtain the current target reference time point;
the deviation rectifying module is further configured to rectify the current target reference time point by using a difference value between the current system time of the storage server and the auxiliary time point, so as to obtain a rectified current target reference time point;
the deleting module is further configured to delete, from the stored sub-objects, a sub-object whose local storage time point matches the corrected current target reference time point, and trigger the detecting module to execute the detection of whether the storage server itself meets a preset storage space availability condition.
11. A storage space release device for a management server of a distributed system, the device comprising:
the time point determining module is used for selecting a target reference time point serving as a deleting basis from the recorded multiple reference time points when the distributed system is detected to meet a preset storage space release condition; acquiring the current time point of an auxiliary clock as an auxiliary time point; the setting mode of the time point of the auxiliary clock comprises the following steps: determining a unified time point according to the system time of each storage server, taking the unified time point as an initial time point of an auxiliary clock, and maintaining the time point of the auxiliary clock to continuously increment according to a preset clock increment rule; the reference time point is determined in the following manner: every time the object storage is completed, the management server records the current time point of the auxiliary clock as the reference time point of the stored object; the reference time point is a unified storage time point of each sub-object of the stored object stored in the storage server;
the sending module is used for sending the target reference time point and the auxiliary time point to each storage server in the distributed system, determining target index information serving as a deleting basis from a plurality of index information, sending the target index information to each storage server in the distributed system, enabling the storage server to correct the target reference time point by utilizing the difference value between the current system time of the storage server and the auxiliary time point after receiving the target reference time point and the auxiliary time point, obtaining a corrected target reference time point, and deleting a sub-object of which the local storage time point is matched with the corrected target reference time point from stored sub-objects; the deleting the sub-object with the local storage time point matched with the target reference time point after deviation correction from the stored sub-objects comprises the following steps: after receiving the target index information, the storage server deletes the sub-object with the local storage time point matched with the rectified target reference time point from the stored sub-objects by utilizing the target index information;
Wherein the index information includes: the identification of the virtual resource pool with the sub-objects stored therein, and/or the object identification; the determining mode of the target index information comprises the following steps: searching an object corresponding to the target reference time point from the pre-stored corresponding relation between the reference time point and the object, and determining the index information of the searched object as target index information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910826429.9A CN111399754B (en) | 2019-09-03 | 2019-09-03 | Method and device for releasing storage space and distributed system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910826429.9A CN111399754B (en) | 2019-09-03 | 2019-09-03 | Method and device for releasing storage space and distributed system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111399754A CN111399754A (en) | 2020-07-10 |
| CN111399754B true CN111399754B (en) | 2023-11-03 |
Family
ID=71434023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910826429.9A Active CN111399754B (en) | 2019-09-03 | 2019-09-03 | Method and device for releasing storage space and distributed system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111399754B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112433809B (en) * | 2020-11-05 | 2023-12-22 | 北京浪潮数据技术有限公司 | JVM memory management method, device, equipment and readable storage medium |
| CN119376651B (en) * | 2024-12-27 | 2025-03-21 | 苏州元脑智能科技有限公司 | Storage space management methods, products, equipment and media |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201726424U (en) * | 2009-08-18 | 2011-01-26 | 升东网络科技发展(上海)有限公司 | Distributed storage system |
| JP2015109053A (en) * | 2013-12-05 | 2015-06-11 | ブラザー工業株式会社 | Written data processing device |
| US9152578B1 (en) * | 2013-03-12 | 2015-10-06 | Emc Corporation | Securing data replication, backup and mobility in cloud storage |
| CN107734008A (en) * | 2017-09-27 | 2018-02-23 | 柏科数据技术(深圳)股份有限公司 | Method, apparatus, node device and the storage medium of a kind of troubleshooting in data-storage system |
| CN109558065A (en) * | 2017-09-25 | 2019-04-02 | 杭州海康威视系统技术有限公司 | Data-erasure method and distributed memory system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8650365B2 (en) * | 2011-09-02 | 2014-02-11 | Compuverde Ab | Method and device for maintaining data in a data storage system comprising a plurality of data storage nodes |
| US9378337B2 (en) * | 2013-06-19 | 2016-06-28 | Sap Se | Data item deletion in a database system |
| US9658773B2 (en) * | 2015-02-09 | 2017-05-23 | International Business Machines Corporation | Management of extents for space efficient storage volumes by reusing previously allocated extents |
| JP6507882B2 (en) * | 2015-06-26 | 2019-05-08 | セイコーエプソン株式会社 | Network system and control method of network system |
-
2019
- 2019-09-03 CN CN201910826429.9A patent/CN111399754B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201726424U (en) * | 2009-08-18 | 2011-01-26 | 升东网络科技发展(上海)有限公司 | Distributed storage system |
| US9152578B1 (en) * | 2013-03-12 | 2015-10-06 | Emc Corporation | Securing data replication, backup and mobility in cloud storage |
| JP2015109053A (en) * | 2013-12-05 | 2015-06-11 | ブラザー工業株式会社 | Written data processing device |
| CN109558065A (en) * | 2017-09-25 | 2019-04-02 | 杭州海康威视系统技术有限公司 | Data-erasure method and distributed memory system |
| CN107734008A (en) * | 2017-09-27 | 2018-02-23 | 柏科数据技术(深圳)股份有限公司 | Method, apparatus, node device and the storage medium of a kind of troubleshooting in data-storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111399754A (en) | 2020-07-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108089893B (en) | Method and device for determining redundant resources, terminal equipment and storage medium | |
| CN107656807B (en) | Method and device for automatic elastic scaling of virtual resources | |
| CN108446407A (en) | Database audit method based on block chain and device | |
| CN108255620B (en) | Service logic processing method, device, service server and system | |
| CN109241084B (en) | Data query method, terminal equipment and medium | |
| WO2016197814A1 (en) | Junk file identification and management method, identification device, management device and terminal | |
| CN109672608B (en) | Method for transmitting messages according to time | |
| CN111488377A (en) | Data query method and device, electronic equipment and storage medium | |
| CN111399754B (en) | Method and device for releasing storage space and distributed system | |
| CN111562884A (en) | Data storage method and device and electronic equipment | |
| CN109976828B (en) | Method and device for configuring file | |
| CN112685474A (en) | Application management method, device, equipment and storage medium | |
| US20240370289A1 (en) | Method, apparatus and system for managing container clusters and components, and storage medium | |
| CN111966867B (en) | Object deletion method, data processing method and device | |
| CN114253969A (en) | Real-time data offline storage method and device and electronic equipment | |
| CN111291127B (en) | Data synchronization method, device, server and storage medium | |
| CN113760854A (en) | Method for identifying data in HDFS memory and related equipment | |
| CN113609168A (en) | Data export method, device, terminal and readable storage medium | |
| CN118656381A (en) | Query statement processing method, device, equipment and computer medium | |
| CN112035498A (en) | Data block scheduling method and device, scheduling layer node and storage layer node | |
| CN111651443A (en) | Data management method and device, electronic equipment and storage medium | |
| US20230297486A1 (en) | Arrangement plan search device, computer system, and arrangement plan search method | |
| JP6574310B2 (en) | Evaluation information matching method, apparatus and server | |
| CN111045983A (en) | Nuclear power station electronic file management method and device, terminal equipment and medium | |
| CN112583761A (en) | Management method and device of security entity, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |