CN107357535A - A kind of method and device for determining the cold and hot grade of data - Google Patents

Abstract

The invention discloses a method and device for determining the hot and cold grades of data. The method includes: calculating the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data; marking the hot data mark of the data The weighted calculation is performed on the value and the cold data mark value, and the hot and cold level of the data is determined according to the corresponding relationship between the weighted calculation result and the hot and cold level. The basis of the present invention to analyze and determine data attributes is the historical processing method of data, such as data written by host, data written by garbage collection, etc., rather than time, even if power failure occurs, the basis of the present invention will not be affected by it , but remains accurate, so the data does not need to go through multiple determination processes to determine the hot and cold grades of the data, so that the present invention can accurately and comprehensively determine the hot and cold attributes of the data.

Description

A method and device for determining data cold and hot levels

technical field

The present invention relates to the technical field of data storage, in particular to a method and device for determining the hot and cold levels of data.

Background technique

With the development and wide application of technologies such as the Internet, cloud computing, and the Internet of Things, massive data needs to be processed and stored all the time, which puts forward higher requirements for the performance of the storage system. It is widely used due to its fast writing speed and low power consumption. In order to prolong the service life of solid-state drives and improve the performance of solid-state drives, it is necessary to process them through garbage collection and wear leveling algorithms, and effective analysis and determination of hot and cold attributes of data is necessary to improve the effects of garbage collection and wear leveling algorithms way.

The existing method of analyzing and determining the hot and cold attributes of data is based on the time of data access, that is, recording and analyzing the time of data access. The data with a longer access time interval is cold data, and the data with a shorter access time interval is hot data. However, there is usually no crystal oscillator in the solid-state hard disk to ensure the clock during the power-off process. If the solid-state hard disk has a power-off phenomenon, the time will start from zero after the solid-state hard disk is powered on again, but the time stamp stored in the data as a basis However, it will not change accordingly, which leads to inaccurate time intervals of data access, thus making it impossible to accurately and comprehensively determine the hot and cold attributes of the data.

Contents of the invention

The object of the present invention is to provide a method and device for determining the hot and cold grades of data, so as to solve the problem that the hot and cold attributes of data cannot be determined accurately and comprehensively.

In order to solve the above technical problems, the present invention provides a method for determining the hot and cold grades of data, including:

calculating a hot data mark value and a cold data mark value of the data according to a historical processing manner of the data;

Carry out weighted calculation on the hot data mark value and cold data mark value of the data, and determine the cold or hot level of the data according to the corresponding relationship between the weighted calculation result and the cold or hot level.

Optionally, the calculating the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data includes:

The historical processing method of the data is to increase the hot data mark value when the host writes;

The historical processing method of the data is to increase the cold data mark value when garbage collection is written;

The historical processing method of the data is to increase the cold data mark value when error handling is recovered and written.

Optionally, the calculating the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data includes:

The historical processing method of the data is to increase the hot data mark value when the host writes;

The historical processing method of the data is to increase the cold data mark value when garbage collection is written;

The historical processing method of the data is that the cold data mark value and the hot data mark value do not change when error processing is recovered and written.

Optionally, after determining the hot and cold grades of the data according to the correspondence between the weighted calculation results and the hot and cold grades, the method further includes:

Store the hot and cold levels of the data in the FTL table or in the flash memory chip.

Optionally, after determining the hot and cold grades of the data according to the correspondence between the weighted calculation results and the hot and cold grades, the method further includes:

According to the hot and cold level of the data and the global erasure situation, the target block for data writing is selected, and the data is written.

The present invention also provides a device for determining the hot and cold grades of data, including: a calculation module and a weighted judgment module;

Wherein, the calculation module is used to calculate the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data;

The weighted determination module is configured to perform weighted calculation on the hot data mark value and the cold data mark value of the data, and determine the cold or hot level of the data according to the corresponding relationship between the weighted calculation result and the cold or hot level.

Optionally, the calculation module includes: a thermal data calculation unit and a first cold data calculation unit;

Wherein, the hot data calculation unit is used to increase the hot data mark value when the historical processing mode of the data is host writing;

The first cold data calculation unit is used to increase the cold data mark value when the historical processing mode of the data is garbage collection and writing; increase the cold data mark when the historical processing mode of the data is error processing recovery writing value.

Optionally, the calculation module includes: a thermal data calculation unit and a second cold data calculation unit;

Wherein, the hot data calculation unit is used to increase the hot data mark value when the historical processing mode of the data is host writing;

The second cold data calculation unit is configured to increase the cold data mark value when the historical processing mode of the data is garbage collection and writing.

Optionally, the device further includes: a storage module, configured to store the hot and cold levels of the data in an FTL table or in a flash memory chip.

Optionally, the device further includes: a writing module, configured to select a target block for data writing according to the hot and cold levels of the data and the global erasure situation, and write the data.

The method and device for determining the hot and cold grades of data provided by the present invention firstly calculate the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data, and then mark the hot data mark value and the cold data mark value of the data The value is weighted and calculated, and the hot and cold level of the data is determined according to the corresponding relationship between the weighted calculation result and the hot and cold level. The basis of the present invention to analyze and determine data attributes is the historical processing method of data, such as data written by host, data written by garbage collection, etc., rather than time, even if power failure occurs, the basis of the present invention will not be affected by it , but remains accurate, so the data can determine the hot and cold grades of the data without performing multiple determination processes, so that the present invention can accurately and comprehensively determine the hot and cold attributes of the data.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only For some embodiments of the present invention, those skilled in the art can also obtain other drawings based on these drawings without any creative work.

FIG. 1 is a flow chart of a method for determining the hot and cold levels of data provided by an embodiment of the present invention;

Fig. 2 is a flow chart of the first embodiment of the method for determining the hot and cold level of data provided by the embodiment of the present invention;

Fig. 3 is a flow chart of the second specific implementation of the method for determining the hot and cold level of data provided by the embodiment of the present invention;

Fig. 4 is a structural block diagram of an apparatus for determining data coldness and hotness levels provided by an embodiment of the present invention.

detailed description

The core of the present invention is to provide a method and device for determining the hot and cold grades of data, so as to solve the problem that the hot and cold attributes of data cannot be accurately and comprehensively determined.

In order to enable those skilled in the art to better understand the solution of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. Apparently, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Please refer to Fig. 1, Fig. 1 is a flowchart of a method for determining the hot and cold levels of data provided by an embodiment of the present invention, including the following steps:

Step 101: Calculate the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data.

There are many ways to process data in solid-state drives, one is host writing, one is garbage collection writing, and the other is error recovery processing writing. The data written by the host can be considered to be biased towards hot data, the data written by garbage collection can be considered to be biased towards cold data, and the data recovered by error processing maintains its original hot and cold attributes, but is more biased towards cold data. The data in the solid-state disk may be processed many times. For example, after the data is written by the host, it may also be garbage collected. When the data is written by the host, it is biased towards hot data, but after being garbage collected, other It will gradually become colder, so when determining the hot and cold level of the data, it should be combined with its previous processing situation. Different processing methods will affect different cold and hot tag values. Every time the data is processed, it must be based on the processing Calculate the cold and hot tag values of the data once.

Step 102: Carry out weighted calculations on the hot and cold data flag values of the data, and determine the hot and cold grades of the data according to the correspondence between the weighted calculation results and the hot and cold grades.

When performing weighted calculations, weights can be assigned to hot data flag values and cold data flag values for calculation according to actual conditions. The results obtained by the weighted calculation can directly correspond to the hot and cold grades according to the specific calculations in practice, or there may be a certain correspondence between the hot and cold grades. For example, the difference between the results obtained by the weighted calculation and the hot and cold grades is 10 times, etc. The specific corresponding relationship needs to be set according to actual requirements. The number of hot and cold levels can also be set according to the actual situation. For example, set the hot and cold levels to 1-10. The lower the level, the colder the data, and the higher the level, the hotter the data.

In the embodiment of the present invention, the basis for analyzing and determining the data attributes is the historical processing method of data, such as data written by host, data written by garbage collection, etc., rather than time, even if there is a power failure phenomenon, the basis of the present invention will not Affected by it, it remains accurate, so the data does not need to go through multiple determination processes to determine the hot and cold grades of the data, so that the present invention can accurately and comprehensively determine the hot and cold attributes of the data.

In order to facilitate a further understanding of the present invention, the present invention will be described in detail below in conjunction with specific embodiments of the present invention.

Please refer to FIG. 2. FIG. 2 is a flow chart of the first specific implementation of the method for determining the hot and cold levels of data provided by the embodiment of the present invention, including the following steps:

Step 201: Increase the hot data mark value when the historical data processing method is host write; increase the cold data mark value when the data historical processing method is garbage collection write; when the data historical processing method is error processing recycling write Increase the cold data flag value when .

Because the data in the solid-state disk may have been processed many times, each time the data is processed, the cold and hot tag values of the data must be calculated according to the processing method. For example, when the historical processing method of the data When there are two host writes, two writes in the historical processing mode, and two error processing recovery writes, the hot data mark value will be increased twice, and the cold data mark value will be increased four times.

Step 202: Carry out weighted calculations on the hot and cold data flag values of the data, and determine the hot and cold grades of the data according to the correspondence between the weighted calculation results and the hot and cold grades.

Please refer to FIG. 3. FIG. 3 is a flow chart of the second specific implementation of the method for determining the hot and cold levels of data provided by the embodiment of the present invention, including the following steps:

Step 301: When the historical data processing method is host writing, increase the hot data mark value; when the data historical processing method is garbage collection and writing, increase the cold data mark value; when the data historical processing method is error processing recovery and writing Neither the cold data tag value nor the hot data tag value is changed.

Step 302: Carry out weighted calculations on the hot and cold data flag values of the data, and determine the hot and cold grades of the data according to the correspondence between the weighted calculation results and the hot and cold grades.

In the second specific implementation mode, most of them are the same as the first specific implementation mode, and the same parts will not be described again, and the different parts will be described below.

In this specific embodiment, when the historical processing mode of data is error handling, recycling and writing, neither the cold data tag value nor the hot data tag value changes. In the first specific embodiment, when the historical processing mode of data is error processing, recycling and writing When increasing the cold data mark value, which one to use depends on the actual test.

The methods in the above two specific embodiments can be further improved, and the improved methods are as follows:

Store the hot and cold levels of data in FTL tables or in flash memory chips.

In this way, when the data is processed, the hot and cold grades of the data can be analyzed and determined accordingly, and when the data is accessed, it can be known whether the hot and cold grades of the data have changed, and what kind of changes have occurred.

It is also possible to select the target block for data writing according to the hot and cold level of the data and the global erasure situation, and write the data.

In the solid-state hard disk, the number of erasing times for each block is different, some blocks have more times of erasing, and some blocks have less times of erasing. Select the destination block with the most erasure times for the data with the lowest cold and hot grade, and write it into the destination block; In the target block, the content of the cold data will not change for a long time, so that the erasure frequency of the block with a large number of erasures can be reduced; for the data with the highest level of cold and hot, select the target block with the least number of erasures, and Write it into the target block; select the target block with less erasure times for data with high cold and hot grades, and write it into the target block. The frequency of hot data being rewritten is high, so that the The erase frequency of blocks with a small number of erasures is increased to achieve wear leveling.

The following is an introduction to the device for determining the data coldness level provided by the embodiment of the present invention. The device for determining the data coldness level described below and the method for determining the data coldness level described above can be referred to for each other.

Please refer to FIG. 4. FIG. 4 is a structural block diagram of an apparatus for determining data coldness and heat levels provided by an embodiment of the present invention. The apparatus for determining data coldness and heat levels includes: a calculation module 401 and a weighted determination module 402;

Wherein, the calculation module 401 is used to calculate the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data;

The weighting determination module 402 is configured to perform weighted calculation on the hot data mark value and the cold data mark value of the data, and determine the cold or hot level of the data according to the correspondence between the weighted calculation result and the cold or hot level.

The calculation module 401 includes a hot data calculation unit 403 , a first cold data calculation unit 404 and a second cold data calculation unit 405 .

Wherein, the hot data calculation unit 403 is used to increase the hot data mark value when the historical processing mode of the data is host writing;

The first cold data calculation unit 404 is configured to increase the cold data mark value when the historical processing mode of the data is garbage collection write; increase the cold data mark value when the historical processing mode of the data is error processing recovery write ;

The second cold data calculation unit 405 is configured to increase the cold data mark value when the historical processing mode of the data is garbage collection and writing.

In order to make a corresponding analysis and determination of the hot and cold grades of the data when the data is processed, and to know whether the hot and cold grades of the data have changed and what kind of changes have occurred when accessing the data, the embodiments of the present invention provide The means for determining the hot and cold grades of data may further include:

The storage module is used to store the cold and hot levels of the data in the FTL table or in the flash memory chip.

In order to achieve wear leveling, the device for determining the hot and cold levels of data provided by the embodiments of the present invention may further include:

The writing module is used for selecting the target block for data writing according to the hot and cold grades of the data and the global erasing situation, and writing the data.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for the related information, please refer to the description of the method part.

Professionals can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the possible For interchangeability, in the above description, the composition and steps of each example have been generally described according to their functions. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

The method and device for determining the hot and cold levels of data provided by the present invention have been introduced in detail above. In this paper, specific examples are used to illustrate the principle and implementation of the present invention, and the descriptions of the above embodiments are only used to help understand the method and core idea of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, some improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims (10)

1. A method for determining the hot and cold grades of data, comprising: calculating a hot data mark value and a cold data mark value of the data according to a historical processing manner of the data; Carry out weighted calculation on the hot data mark value and cold data mark value of the data, and determine the cold or hot level of the data according to the corresponding relationship between the weighted calculation result and the cold or hot level. 2. The method according to claim 1, wherein the calculating the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data comprises: When the historical processing mode of the data is written by host, increase the hot data mark value; When the historical processing mode of the data is garbage collection and writing, increase the cold data mark value; When the historical processing mode of the data is error handling recovery write, increase the cold data mark value. 3. The method according to claim 1, wherein the calculating the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data comprises: When the historical processing mode of the data is written by host, increase the hot data mark value; When the historical processing mode of the data is garbage collection and writing, increase the cold data mark value; When the history processing mode of the data is error processing recovery writing, the cold data mark value and the hot data mark value do not change. 4. The method according to any one of claims 1-3, characterized in that, after determining the hot and cold grades of the data according to the correspondence between the weighted calculation results and the hot and cold grades, further comprising: Store the hot and cold levels of the data in the FTL table or in the flash memory chip. 5. The method according to any one of claims 1-3, characterized in that, after determining the hot and cold grades of the data according to the correspondence between the weighted calculation results and the hot and cold grades, further comprising: According to the hot and cold level of the data and the global erasure situation, the target block for data writing is selected, and the data is written. 6. A device for determining the hot and cold grades of data, comprising: a calculation module and a weighted judgment module; Wherein, the calculation module is used to calculate the hot data mark value and the cold data mark value of the data according to the historical processing mode of the data; The weighted determination module is configured to perform weighted calculation on the hot data mark value and the cold data mark value of the data, and determine the cold or hot level of the data according to the corresponding relationship between the weighted calculation result and the cold or hot level. 7. The device according to claim 6, wherein the calculation module comprises: a thermal data calculation unit and a first cold data calculation unit; Wherein, the hot data calculation unit is used to increase the hot data mark value when the historical processing mode of the data is host write; The first cold data calculation unit is used to increase the cold data mark value when the historical processing method of the data is garbage collection and writing; when the historical processing method of the data is error processing recovery writing, increase the cold Data tag value. 8. The device according to claim 6, wherein the calculation module comprises: a thermal data calculation unit and a second cold data calculation unit; Wherein, the hot data calculation unit is used to increase the hot data mark value when the historical processing mode of the data is host writing; The second cold data calculation unit is configured to increase the cold data mark value when the historical processing mode of the data is garbage collection and writing. 9. The device according to any one of claims 6-8, further comprising: The storage module is used to store the cold and hot levels of the data in the FTL table or in the flash memory chip. 10. The device according to any one of claims 6-8, further comprising: The writing module is used for selecting the target block for data writing according to the hot and cold grades of the data and the global erasing situation, and writing the data.