WO2024130889A1 - Data compression method, system and device, and computer readable storage medium - Google Patents

Abstract

A data compression method, system and device, and a computer-readable storage medium. The method comprises: acquiring target data to be compressed; counting the total number of bytes in said target data and the occurrence probability of each type of characters; determining a target linear function of a function x*log₂(x) under a preset approximate condition; on the basis of the target linear function, determining an information entropy operation formula without a logarithm operation; calculating a target information entropy of said target data by means of the information entropy operation formula and on the basis of the total number of bytes and the occurrence probability; and determining whether the target information entropy is smaller than a preset value, if the target information entropy is smaller than the preset value, compressing said target data, and if the target information entropy is greater than or equal to the preset value, not compressing said target data. In the present application, the function x*log₂(x) needs to be converted into a target linear function without a logarithm operation. In this way, the information entropy operation formula without a logarithm operation can be determined on the basis of the target linear function, so that the calculation efficiency of the target information entropy is improved, thereby improving the data compression efficiency.

Description

A data compression method, system, device and computer readable storage medium

This application claims priority to the Chinese patent application filed with the China Patent Office on December 22, 2022, with application number 202211658252.4 and invention name “A data compression method, system, device and computer-readable storage medium”, the entire contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the technical field of data compression, and in particular to a data compression method, system, device and computer-readable storage medium.

Background technique

Data compression can potentially reduce the storage space of data, increase the logical capacity of storage devices, and thus reduce the storage and transmission costs of data, so it has a high technical appeal. Data compression is a computationally intensive operation, and the actual data reduction effect it achieves also depends on the compressibility of the data to be compressed. If compression is performed on incompressible data, the size of the compressed data will not become smaller, which wastes the high computational overhead. Therefore, if the compressibility of data can be accurately predicted, thereby avoiding compressing incompressible data, the high compression computational overhead can be avoided.

In the process of data compression, information entropy is a key indicator for measuring data compressibility and predicting data compression rate. The calculation of information entropy requires logarithmic operations, which has a certain computational complexity and complex hardware implementation logic, which makes the calculation efficiency of information entropy low, thus affecting the efficiency of data compression.

In summary, how to improve the efficiency of data compression is a problem that needs to be solved urgently by those skilled in the art.

Summary of the invention

In view of this, the purpose of this application is to provide a data compression method, which can solve the technical problem of how to improve data compression efficiency to a certain extent. This application also provides a data compression system, device and computer-readable storage medium. The specific scheme is as follows:

A data compression method, comprising:

Obtain target data to be compressed;

Counting the total number of bytes in the target data and the occurrence probability of each type of character;

Determine the target linear function of the function x*log ₂ (x) under the preset approximate conditions;

Determine an information entropy calculation formula without logarithmic calculation based on the target linear function;

Calculate the target information entropy of the target data based on the total number of bytes and the occurrence probability by using the information entropy calculation formula;

It is determined whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed; if the target information entropy is greater than or equal to the preset value, the target data is not compressed.

Preferably, the determining of the target linear function of the function x*log ₂ (x) under a preset approximate condition comprises:

Determine the target linear function of function x*log ₂ (x) under the preset approximate condition;

The target linear function includes:
y＝x*k(x)+2(x-2 ^k(x) );

Here, k(x) represents a function for determining the value of K that makes the value of x fall within the range of [2 ^K ,2 ^K+1 ).

Preferably, the information entropy calculation formula without logarithmic operation is determined based on the target linear function, including:

Determine the information entropy calculation formula without logarithmic operation based on the target linear function;

The information entropy calculation formula includes:

Among them, H represents the target information entropy; N represents the total number of bytes; M represents the total number of character types; f(i) represents the occurrence probability of the i-th type of character, where the value range of i is [0,M].

Preferably, the determining of the target linear function of the function x*log ₂ (x) under a preset approximate condition comprises:

Determine the target linear function of function x*log ₂ (x) under the preset approximate condition;

The target linear function includes:
y＝x*k(x)+((2-a)x-2 ^k(x)+1 );

Wherein, k(x) represents a function for determining the K value that makes the value of x between [2 ^K ,2 ^K+1 ); a represents an adjustment parameter.

Preferably, it also includes:

The value of the adjustment parameter is determined based on the value of 1.5*2 ^K.

Preferably, the information entropy calculation formula without logarithmic operation is determined based on the target linear function, including:

Determine the information entropy calculation formula without logarithmic operation based on the target linear function;

The information entropy calculation formula includes:

Among them, H represents the target information entropy; N represents the total number of bytes; M represents the total number of character types; f(i) represents the occurrence probability of the i-th type of character, where the value range of i is [0,M].

Preferably, the calculation process of k(x) includes:

Convert the value of x to a 32-bit binary integer;

Count the number of prefix 0s in the 32-bit integer;

The difference between 31 and the numerical value is taken as the K value corresponding to k(x).

Preferably, the process of calculating the target information entropy of the target data based on the total number of bytes and the occurrence probability by using the information entropy calculation formula includes:

Determine the operation result of x*k(x) based on the shift-add operation;

Take the lower (k(x)-1) bits of x and get the result of (x-2 ^k(x) ).

A data compression system, comprising:

A first acquisition module, used for acquiring target data to be compressed;

A first statistical module, used for counting the total number of bytes in the target data and the occurrence probability of each type of character;

A first determination module is used to determine a target linear function of the function x*log ₂ (x) under a preset approximate condition;

A second determination module is used to determine an information entropy calculation formula without logarithmic operation based on the target linear function;

A first calculation module, configured to calculate the target information entropy of the target data based on the total number of bytes and the occurrence probability by using the information entropy calculation formula;

The first judgment module is used to judge whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed; if the target information entropy is greater than or equal to the preset value, the target data is not compressed.

A data compression device, comprising:

Memory for storing computer programs;

A processor is used to implement the steps of any of the above data compression methods when executing the computer program.

A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned data compression methods.

The present application provides a data compression method, which obtains target data to be compressed; counts the total number of bytes in the target data and the probability of occurrence of each type of character; determines the target linear function of the function x*log ₂ (x) under preset approximate conditions; determines the information entropy calculation formula without logarithmic operation based on the target linear function; calculates the target information entropy of the target data based on the total number of bytes and the probability of occurrence through the information entropy calculation formula; determines whether the target information entropy is less than a preset value, if the target information entropy is less than the preset value, the target data is compressed, if the target information entropy is greater than or equal to the preset value, the target data is not compressed. In the present application, after counting the total number of bytes in the target data and the probability of occurrence of each type of character, it is necessary to determine the target linear function of the function x*log ₂ (x) under preset approximate conditions, that is, to transform the function x*log ₂ (x) into a target linear function without logarithmic operation and with simpler logic, so that the information entropy calculation formula without logarithmic operation can be determined based on the target linear function, and then the target information entropy of the target data can be calculated without complex logarithmic operation, which improves the calculation efficiency of the target information entropy, and further improves the efficiency of data compression based on the target information entropy. The data compression system, device and computer-readable storage medium provided in this application also solve the corresponding technical problems.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

FIG1 is a flow chart of a data compression method provided in an embodiment of the present application;

Figure 2 is a simulation graph of x*log ₂ (x);

FIG3 is a simulation comparison result diagram of x*log ₂ (x) and x*k(x)+(2*x-2 ^k(x)+1 );

FIG4 is a schematic diagram of the structure of a data compression system provided in an embodiment of the present application;

FIG5 is a schematic diagram of the structure of a data compression device provided in an embodiment of the present application;

FIG. 6 is another schematic diagram of the structure of a data compression device provided in an embodiment of the present application.

Detailed ways

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

Please refer to FIG. 1 , which is a flow chart of a data compression method provided in an embodiment of the present application.

A data compression method provided in an embodiment of the present application may include the following steps:

Step S101: Acquire target data to be compressed.

In practical applications, the target data to be compressed may be obtained first, and the type and size of the target data may be determined according to the specific application scenario, and this application does not make any specific limitation here.

Step S102: Count the total number of bytes in the target data and the occurrence probability of each type of character.

In practical applications, after obtaining the target data to be compressed, the total number of bytes in the target data and the probability of occurrence of each type of character can be counted. Taking characters as an example, computers often use one byte (also known as 1Byte), that is, 8-bit characters, to represent them. Then there are 2^8=256 types of characters, that is, the mathematical value range of characters is [0,255]. Here, the probability of occurrence of each type of character is the probability of occurrence of each unique character, that is, each value in the range of 0 to 255. Prepare for the subsequent calculation of the information entropy of the target data.

Step S103: Determine a target linear function of the function x*log ₂ (x) under a preset approximate condition.

Step S104: Determine an information entropy calculation formula that does not contain logarithmic operations based on the target linear function.

In practical applications, on the one hand, the existing information entropy calculation process involves logarithmic calculations, which are time-consuming and will result in low information entropy calculation efficiency; on the other hand, the use of information entropy to measure data compressibility or predict data compression rate does not require an accurate information entropy value, and a high-precision information entropy approximation has little effect on the accuracy of compression rate prediction; therefore, the present application herein determines a target linear function of the function x*log ₂ (x) required in the information entropy calculation process under preset approximate conditions, and determines an information entropy calculation formula that does not contain logarithmic calculations based on the target linear function, so as to eliminate the logarithmic calculation in the information entropy calculation process, and obtain an information entropy calculation formula that does not affect the compression rate prediction.

Step S105: Calculate the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula.

In practical applications, after determining the information entropy calculation formula without logarithmic operation based on the target linear function, the target information entropy of the target data can be calculated based on the total number of bytes and the probability of occurrence through the information entropy calculation formula.

Step S106: Determine whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, execute step S107: compress the target data; if the target information entropy is greater than or equal to the preset value, execute step S108: do not compress the target data.

In practical applications, after calculating the target information entropy of the target data based on the total number of bytes and the probability of occurrence through the information entropy calculation formula, it is possible to determine whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed. If the target information entropy is greater than or equal to the preset value, the target data is not compressed. It should be noted that the specific value of the preset value that determines whether to compress the target data can be determined according to the application scenario, and this application does not make specific limitations here.

The present application provides a data compression method, which comprises the following steps: obtaining target data to be compressed; counting the total number of bytes in the target data and the probability of occurrence of each type of character; determining a target linear function of the function x*log ₂ (x) under a preset approximate condition; determining an information entropy calculation formula without logarithmic calculation based on the target linear function; and calculating the target data based on the total number of bytes and the probability of occurrence through the information entropy calculation formula. ; determine whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed. If the target information entropy is greater than or equal to the preset value, the target data is not compressed. In the present application, after counting the total number of bytes in the target data and the probability of occurrence of each type of character, it is necessary to determine the target linear function of the function x*log ₂ (x) under preset approximate conditions, that is, to transform the function x*log ₂ (x) into a target linear function that does not contain logarithmic operations and has simpler logic. In this way, an information entropy calculation formula that does not contain logarithmic operations can be determined based on the target linear function, and then the target information entropy of the target data can be calculated without performing complex logarithmic operations, thereby improving the calculation efficiency of the target information entropy, and further improving the efficiency of data compression based on the target information entropy.

In an exemplary application scenario, in a data compression method provided by an embodiment of the present application, considering that there are multiple events S = {E ₁ , ..., _En } in an existing system S, and the probability distribution of each event P = {p ₁ , ..., p _n }, the information of each event itself is:
I _e = -log ₂ p _i ;

The logarithm is based on 2 and the unit is bits;

Information entropy is the average amount of information contained in each message received, that is:

In the context of data compression, information entropy refers to byte entropy. The basic unit of data storage is a byte (also known as 1Byte), which includes 8 data bits, that is, it consists of 8 bits. In the above formula, the value range of n is [0,255], with a total of 256 values. Suppose the total number of bytes of the input message is N, and the probability of occurrence of each character i (which can be understood as the number of occurrences) is f(i) (where i∈[0,255]), then

The probability of occurrence of character x is p(i) = f(i)/N. Then

The information entropy calculation formula is:

Among them, H represents the target information entropy, that is, the information entropy of the target data; N represents the total number of bytes, that is, the total number of bytes of the target data; M represents the total number of character types, which is the total number of unique characters in the target data, that is, the total number of characters with different values; f(i) represents the probability of occurrence of the i-th type of character, that is, the probability of the i-th type of character appearing in the target data, where the value range of i is [0, M];

Multiplying both sides of the above information entropy calculation formula by N yields:

At this time, the forms of N·log ₂ (N) and f(i)log ₂ (f(i)) can be uniformly expressed as x*log ₂ (x), that is, it is only necessary to determine the target linear function of the function x*log ₂ (x) under the preset approximate condition, without the need to perform the complex logarithmic operation in the information entropy calculation formula; the simulation graph of x*log ₂ (x) is shown in FIG2 , and it can be seen from FIG2 that x*log ₂ (x) is close to the linear distribution y=kx;

The preset approximate condition can be understood as calculating the target linear function by dividing the interval to improve the accuracy. The interval [2 ^K ,2 ^K+1 ) to which the value of x belongs can be set, and then x*log ₂ (x) is converted into an approximate target linear function based on the interval;

Specifically, the value of x*log ₂ (x) at point 2 ^K is K·2 ^K , and the value at point 2 ^K+1 is (K+1)·2 ^K+1 . Apply the slope formula to determine the target linear function, and the slope R is:

According to the linear distribution formula, the slope R = (y ₂ -y ₁ )/(x ₂ -x ₁ ) is obtained. At this time, the linear equation can be expressed as yy ₁ = R(xx ₁ ), and the target linear function that approximates the function x*log ₂ (x) can be expressed as:
yK· ^2K ＝(K+2)(x- ^2K );

Assuming that k(x) represents a function that determines the K value that makes the value of x between [2 ^K ,2 ^K+1 ), that is, k(x) represents a function that finds the K value that is an integer and less than or equal to the logarithmic base of x, the target linear function can be expressed as:
y＝k(x)*2k ^(x) +(x-2k ^(x) )*(2k ^(x) +2)＝x*k(x)+2(x-2k ^(x) );

At this time, the information entropy calculation formula can be expressed as:

Furthermore, considering:

x*log ₂ (x)≈k(x)*2 ^k(x) +(x-2 ^k(x) )*(k(x)+2)＝x*k(x)+(2*x-2 ^k(x)+1 ). By comparing x*log ₂ (x) and x*k(x)+(2*x-2 ^k(x)+1 ), we can see that (2*x-2 ^k(x)+1 ) is an estimate of the decimal part of log ₂ (x). For ease of understanding, assuming that the value of x is 5, then k(x)＝2, log ₂ (x)＝2.32. Then, x*k(x) corresponds to the integer part of x*log ₂ (x), that is, the operation of 5*2, and (2*x-2 ^k(x)+1 ) corresponds to the operation of the decimal part of x*log ₂ (x), that is, the operation of 0.32*5. As mentioned above, x*log ₂ (x) is close to linear distribution, while x*k(x) is a linear distribution. (2*x-2 ^k(x)+1 ) is a compensatory approximation of the true x*log ₂ (x) based on linear distribution. Therefore, in order to further improve the similarity accuracy between the target linear function and x*log ₂ (x), the adjustment parameter a can be introduced in (2*x-2 ^k(x)+1 ) to further adjust the target linear function to:
y＝x*k(x)+((2-a)x-2 ^k(x)+1 );

At this time, the information entropy calculation formula can be:

It should be noted that, in a specific application scenario, a fixed value can be preset for the adjustment parameter a according to the changes of x and log ₂ (x), for example, different a values can be given in the intervals of [2,4], [4,8], [8,16], [16,32], etc. In addition, after comparing the simulation graphs of x*log ₂ (x) and x*k(x)+(2*x-2 ^k(x)+1 ), the comparison result is shown in FIG3 , wherein the upper line represents x*k(x)+(2*x-2 ^k(x)+1 ). It can be seen from the simulation graph that the maximum difference point between x*log ₂ (x) and x*k(x)+(2*x-2 ^k(x)+1 ) in the interval [2 ^K ,2 ^K+1 ] is 1.5*2 ^K . Therefore, in order to further improve the accuracy of the adjustment parameter a, the value of the adjustment parameter can also be determined based on the value of 1.5*2 ^K.

In a specific application scenario, in order to find the integer K, the function k(x) is defined for finding the integer K for the value x. The value of x is between the interval [ ^2K , 2K ⁺¹ ). The operation process of k(x) can be as follows: convert the value of x into a binary 32-bit integer; count the number of values with prefix 0 in the 32-bit integer; and take the difference between 31 and the number as the K value corresponding to k(x).

For ease of understanding, assume that the value of x is 12, which is converted to binary as 12=1100b. The highest bit is 1 at the 3rd bit, and the values of bits 4 to 31 are all 0, so K=32-(31-4+1)=4.

Of course, the following formula can also be used to calculate k(x):

In a specific application scenario, in order to improve the operation efficiency, in the process of calculating the target information entropy of the target data based on the total number of bytes and the probability of occurrence through the information entropy calculation formula, k(x) is the logarithmic base of x, which is a very small number relative to x, then x*k(x) is a binary multiplication that can be converted into a simple shift-add operation, that is, the operation result of x*k(x) is determined based on the shift-add operation; the operation of taking the lower (k(x)-1) bits of x is obtained to obtain the operation result of (x-2 ^k(x) ); in this way, the calculation complexity of the entire information entropy is reduced, and it is convenient to realize the calculation of information entropy in hardware. In addition, after experiments, it can be known that the information entropy calculation method of the present application is about 15% faster than the standard information entropy calculation method, and the error is within 2%, so that the data compression efficiency is fast and the error is small.

Please refer to FIG. 4 , which is a schematic diagram of the structure of a data compression system provided in an embodiment of the present application.

A data compression system provided in an embodiment of the present application may include:

A first acquisition module 101 is used to acquire target data to be compressed;

The first statistical module 102 is used to count the total number of bytes in the target data and the occurrence probability of each type of character;

A first determination module 103 is used to determine a target linear function of the function x*log ₂ (x) under a preset approximate condition;

A second determination module 104 is used to determine an information entropy calculation formula without logarithmic calculation based on the target linear function;

The first calculation module 105 is used to calculate the target information entropy of the target data based on the total number of bytes and the probability of occurrence through an information entropy calculation formula;

The first judgment module 106 is used to judge whether the target information entropy is less than a preset value. If the information entropy is less than the preset value, the target data is compressed. If the target information entropy is greater than or equal to the preset value, the target data is not compressed.

In a data compression system provided by an embodiment of the present application, the first determination module may include:

A first determining unit, used to determine a target linear function of the function x*log ₂ (x) under a preset approximate condition;

The target linear functions include:
y＝x*k(x)+2(x-2 ^k(x) );

Here, k(x) represents a function for determining the value of K that makes the value of x fall within the range of [2 ^K ,2 ^K+1 ).

In a data compression system provided by an embodiment of the present application, the second determination module may include:

A second determining unit is used to determine an information entropy calculation formula without logarithmic calculation based on the target linear function;

The information entropy calculation formula includes:

Among them, H represents the target information entropy, that is, the information entropy of the target data; N represents the total number of bytes, that is, the total number of bytes of the target data; M represents the total number of character types, which is the total number of unique characters in the target data, that is, the total number of characters with different values; f(i) represents the probability of occurrence of the i-th type of character, that is, the probability of the i-th type of character appearing in the target data, where the value range of i is [0,M].

In a data compression system provided by an embodiment of the present application, the first determination module may include:

A third determining unit is used to determine a target linear function of the function x*log ₂ (x) under a preset approximate condition;

The target linear functions include:
y＝x*k(x)+((2-a)x-2 ^k(x)+1 );

Wherein, k(x) represents a function for determining the K value that makes the value of x between [2 ^K ,2 ^K+1 ); a represents an adjustment parameter.

A data compression system provided in an embodiment of the present application may further include:

The third determination module is used to determine the value of the adjustment parameter based on the value of 1.5*2 ^K.

In a data compression system provided by an embodiment of the present application, the second determination module may include:

The fourth determining unit is used to determine the information entropy operation without logarithmic operation based on the target linear function. Calculation formula;

The information entropy calculation formula includes:

Among them, H represents the target information entropy, that is, the information entropy of the target data; N represents the total number of bytes, that is, the total number of bytes of the target data; M represents the total number of character types, which is the total number of unique characters in the target data, that is, the total number of characters with different values; f(i) represents the probability of occurrence of the i-th type of character, that is, the probability of the i-th type of character appearing in the target data, where the value range of i is [0,M].

In a data compression system provided by an embodiment of the present application, the calculation process of k(x) includes:

Convert the value of x to a 32-bit binary integer;

Count the number of leading 0s in 32-bit integers;

The difference between 31 and the individual value is taken as the K value corresponding to k(x).

In a data compression system provided by an embodiment of the present application, a first calculation module calculates the target information entropy of target data based on the total number of bytes and the probability of occurrence through an information entropy calculation formula, and is used to: determine the operation result of x*k(x) based on a shift-add operation; and obtain the operation result of (x-2 ^k(x) ) by taking the lower (k(x)-1) bits of x.

The present application also provides a data compression device and a computer-readable storage medium, both of which have the corresponding effects of a data compression method provided in an embodiment of the present application. Please refer to Figure 5, which is a schematic diagram of the structure of a data compression device provided in an embodiment of the present application.

A data compression device provided in an embodiment of the present application includes a memory 201 and a processor 202. The memory 201 stores a computer program. When the processor 202 executes the computer program, the steps of the data compression method described in any of the above embodiments are implemented.

Please refer to FIG6 . Another data compression device provided in the embodiment of the present application may also include: an input port 203 connected to the processor 202, used to transmit commands input from the outside to the processor 202; a display unit 204 connected to the processor 202, used to display the processing results of the processor 202 to the outside; a communication module 205 connected to the processor 202, used to realize the communication between the data compression device and the outside. The display unit 204 can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module 205 includes but is not limited to mobile high-definition link technology (HML), universal serial bus, etc. (USB), High-Definition Multimedia Interface (HDMI), wireless connection: Wireless Fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s.

An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the data compression method described in any of the above embodiments are implemented.

The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

For the description of the relevant parts of the data compression system, device and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the data compression method provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

The data compression scheme provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims (11)

A data compression method, comprising: Obtain target data to be compressed; Counting the total number of bytes in the target data and the occurrence probability of each type of character; Determine the target linear function of the function x*log ₂ (x) under the preset approximate conditions; Determine an information entropy calculation formula without logarithmic calculation based on the target linear function; Calculate the target information entropy of the target data based on the total number of bytes and the occurrence probability by using the information entropy calculation formula; It is determined whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed; if the target information entropy is greater than or equal to the preset value, the target data is not compressed. The method according to claim 1, characterized in that the step of determining a target linear function of the function x*log ₂ (x) under a preset approximate condition comprises: Determine the target linear function of function x*log ₂ (x) under the preset approximate condition; The target linear function includes:
y＝x*k(x)+2(x-2 ^k(x) ); Here, k(x) represents a function for determining the value of K that makes the value of x fall within the range of [2 ^K ,2 ^K+1 ). The method according to claim 2, characterized in that the information entropy calculation formula without logarithmic operation is determined based on the target linear function, comprising: Determine the information entropy calculation formula without logarithmic operation based on the target linear function; The information entropy calculation formula includes:
Among them, H represents the target information entropy; N represents the total number of bytes; M represents the total number of character types; f(i) represents the occurrence probability of the i-th type of character, where the value range of i is [0,M]. The method according to claim 1, characterized in that the step of determining a target linear function of the function x*log ₂ (x) under a preset approximate condition comprises: Determine the target linear function of function x*log ₂ (x) under the preset approximate condition; The target linear function includes:
y＝x*k(x)+((2-a)x-2 ^k(x)+1 ); Wherein, k(x) represents a function for determining the K value that makes the value of x between [2 ^K ,2 ^K+1 ); a represents an adjustment parameter. The method according to claim 4, further comprising: The value of the adjustment parameter is determined based on the value of 1.5*2 ^K. The method according to claim 4, characterized in that the information entropy calculation formula without logarithmic operation is determined based on the target linear function, comprising: Determine the information entropy calculation formula without logarithmic operation based on the target linear function; The information entropy calculation formula includes:
Among them, H represents the target information entropy; N represents the total number of bytes; M represents the total number of character types; f(i) represents the occurrence probability of the i-th type of character, where the value range of i is [0,M]. The method according to claim 3 or 6, characterized in that the calculation process of k(x) comprises: Convert the value of x to a 32-bit binary integer; Count the number of prefix 0s in the 32-bit integer; The difference between 31 and the numerical value is taken as the K value corresponding to k(x). The method according to claim 3 or 6 is characterized in that the process of calculating the target information entropy of the target data based on the total number of bytes and the occurrence probability through the information entropy calculation formula includes: Determine the operation result of x*k(x) based on the shift-add operation; Take the lower (k(x)-1) bits of x and get the result of (x-2 ^k(x) ). A data compression system, comprising: A first acquisition module, used for acquiring target data to be compressed; The first statistical module is used to count the total number of bytes in the target data and the number of occurrences of each type of character. Present probability; A first determination module is used to determine a target linear function of the function x*log ₂ (x) under a preset approximate condition; A second determination module is used to determine an information entropy calculation formula without logarithmic operation based on the target linear function; A first calculation module, configured to calculate the target information entropy of the target data based on the total number of bytes and the occurrence probability by using the information entropy calculation formula; The first judgment module is used to judge whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed; if the target information entropy is greater than or equal to the preset value, the target data is not compressed. A data compression device, comprising: Memory for storing computer programs; A processor, configured to implement the steps of the data compression method according to any one of claims 1 to 8 when executing the computer program. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the data compression method according to any one of claims 1 to 8 are implemented.