CN119892106A

CN119892106A - Service data processing method, device, computer equipment and readable storage medium

Info

Publication number: CN119892106A
Application number: CN202411881436.6A
Authority: CN
Inventors: 叶新发; 董文辉; 吴桂禄; 田欣; 张美玲
Original assignee: Kingdee Software China Co Ltd
Current assignee: Kingdee Software China Co Ltd
Priority date: 2024-12-18
Filing date: 2024-12-18
Publication date: 2025-04-25

Abstract

The present application relates to a business data processing method, apparatus, computer device, computer readable storage medium and computer program product. The method comprises the steps of obtaining first service data, wherein target data belonging to target dimensions in the first service data are range data, the range data comprise at least two values, determining a data coding system based on the value range of the target data in the first service data, coding the target data in the first service data according to the data coding system to obtain a system coding result, wherein the system coding result comprises a data coding system and system coding range information corresponding to the target data, and performing data compression on the first service data based on the system coding result to obtain first compressed service data corresponding to the first service data. The method can improve the processing efficiency of the service data.

Description

Service data processing method, device, computer equipment and readable storage medium

Technical Field

The present application relates to the field of computer technology, and in particular, to a service data processing method, apparatus, computer device, computer readable storage medium, and computer program product.

Background

With the development of computer technology, the traffic data volume of various traffic systems has increased dramatically, and data compression technology has become an important means for improving the traffic data storage and transmission efficiency. The data compression technology can effectively reduce the storage space requirement of the service data by reducing the redundant information of the service data, and is beneficial to improving the transmission efficiency of the service data.

However, when compression and decompression processing is performed on service data, various complicated and cumbersome operations are often required, resulting in low processing efficiency for service data.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a service data processing method, apparatus, computer device, computer readable storage medium, and computer program product that can improve service data processing efficiency.

In a first aspect, the present application provides a service data processing method, including:

acquiring first service data, wherein target data belonging to a target dimension in the first service data is range data, and the range data comprises at least two valued data;

Determining a data code system based on a value range of target data in the first service data;

Coding target data in the first service data according to the data coding system to obtain a system coding result, wherein the system coding result comprises a data coding system and system coding range information corresponding to the target data;

and carrying out data compression on the first service data based on the binary coding result to obtain first compressed service data corresponding to the first service data.

In a second aspect, the present application further provides a service data processing apparatus, including:

the first business data acquisition module is used for acquiring first business data, wherein target data belonging to a target dimension in the first business data is range data, and the range data comprises at least two valued data;

the code system determining module is used for determining a data code system based on the value range of target data in the first service data;

the coding processing module is used for coding the target data in the first service data according to the data coding system to obtain a binary coding result, wherein the binary coding result comprises a data coding system and binary coding range information corresponding to the target data;

And the compression processing module is used for carrying out data compression on the first service data based on the binary coding result to obtain first compressed service data corresponding to the first service data.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the above business data processing method when executing the computer program.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the above business data processing method.

In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the above business data processing method.

The business data processing method, the business data processing device, the computer equipment, the computer readable storage medium and the computer program product are characterized in that for first business data with target data belonging to target dimensions being range data, a data coding system is determined based on the value range of the target data in the first business data, the target data in the first business data is coded according to the data coding system, a system coding result comprising the data coding system and the corresponding system coding range information of the target data is obtained, and data compression is carried out on the first business data based on the system coding result, so that first compressed business data corresponding to the first business data is obtained. For the first service data of which the target data belonging to the target dimension is the range data, the data coding system determined according to the value range of the target data is coded to obtain a binary coding result comprising the data coding system and the binary coding range information corresponding to the target data, so that the compression processing of the first service data is realized, the compression processing of the service data is simplified, and the compression processing efficiency of the service data is improved.

In a sixth aspect, the present application provides a service data processing method, including:

acquiring a service data processing request, and acquiring first compressed service data according to the service data processing request;

acquiring a binary coding result from the first compressed service data, wherein the binary coding result comprises a data coding system and binary coding range information corresponding to data belonging to a target dimension in the first compressed service data;

Decoding the binary coding range information according to the data coding system to obtain target data belonging to a target dimension in the first compressed service data, wherein the target data is range data comprising at least two values;

And based on target data belonging to a target dimension in the first compressed service data, performing data decompression on the first compressed service data to obtain the first service data.

In a seventh aspect, the present application further provides a service data processing apparatus, including:

the processing request acquisition module is used for acquiring a service data processing request and acquiring first compressed service data according to the service data processing request;

the coding result acquisition module is used for acquiring a binary coding result from the first compressed service data, wherein the binary coding result comprises a data coding system and binary coding range information corresponding to data belonging to a target dimension in the first compressed service data;

the decoding processing module is used for decoding the binary coding range information according to the data coding system to obtain target data belonging to a target dimension in the first compressed service data, wherein the target data is range data comprising at least two values;

the decompression processing module is used for carrying out data decompression on the first compressed service data based on the target data belonging to the target dimension in the first compressed service data to obtain the first service data.

In an eighth aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the above business data processing method when executing the computer program.

In a ninth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the above business data processing method.

In a tenth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the above business data processing method.

The service data processing method, the device, the computer equipment, the computer readable storage medium and the computer program product are used for obtaining first compressed service data according to the service data processing request, obtaining a binary coding result from the first compressed service data, wherein the binary coding result comprises a data coding system and binary coding range information corresponding to data belonging to a target dimension in the first compressed service data, decoding the binary coding range information according to the data coding system to obtain target data belonging to the target dimension in the first compressed service data, and performing data decompression on the first compressed service data based on the target data belonging to the target dimension in the first compressed service data to obtain decompressed first service data. For the first compressed service data comprising the binary coding result, decoding the included binary coding range information according to the data coding system in the binary coding result, and performing data decompression based on the target data belonging to the target dimension obtained by decoding to obtain the first service data, so that the decompression processing of the first service data is simplified, and the decompression processing efficiency of the service data is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are needed in the description of the embodiments of the present application or the related technologies will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other related drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.

FIG. 1 is an application environment diagram of a business data processing method in one embodiment;

FIG. 2 is a flow chart of a business data processing method in one embodiment;

FIG. 3 is a flow diagram of a binary encoding in one embodiment;

FIG. 4 is a flow chart of a method for processing business data according to another embodiment;

FIG. 5 is a flow chart of Cartesian calculations performed after two types of service data access according to an embodiment;

FIG. 6 is a schematic diagram of data compression based on binary encoding in one embodiment;

FIG. 7 is a flow chart of a Cartesian calculation of read data according to an embodiment;

FIG. 8 is a flow chart of a Cartesian calculation of read data according to another embodiment;

FIG. 9 is a block diagram of a business data processing device in one embodiment;

FIG. 10 is a block diagram showing a construction of a service data processing apparatus according to still another embodiment;

FIG. 11 is an internal block diagram of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The business data processing method provided by the embodiment of the application can be applied to an application environment shown in figure 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that needs to be processed by the server 104, and the data storage system may be integrated on the server 104, or may be placed on a cloud or other network server.

In one aspect, the terminal 102 may send service data to be compressed to the server 104, where the service data to be compressed may include first service data, the first service data includes target data belonging to a target dimension, and the target data is range data including at least two values. In the different business system scenarios, the target dimension may correspond to different data dimensions, for example, the target dimension may be at least one of various types of dimensions such as a salary level, an employee job level, a customer satisfaction, a product quality, a sales performance, an employee performance, a customer value, and the like. For the first service data, the server 104 may determine a data coding system based on a value range of the target data in the first service data, and code the target data in the first service data according to the data coding system to obtain a binary coding result including information of the data coding system and the corresponding binary coding range of the target data, and the server 104 may perform data compression on the first service data based on the binary coding result to obtain first compressed service data corresponding to the first service data, so as to implement data compression processing on the first service data. In some embodiments, the server 104 may feed back the first compressed service data corresponding to the first service data to the terminal 102, so that the user may view the data compression result through the terminal 102. In some embodiments, the compression processing method for the service data may also be implemented by the server 104 or the terminal 102 separately.

On the other hand, the terminal 102 may send a service data processing request to the server 104 to request the server 104 to trigger data decompression processing. The server 104 may obtain first compressed service data according to the service data processing request, obtain a binary coding result from the first compressed service data, where the binary coding result includes a data coding bin and binary coding range information corresponding to data belonging to a target dimension in the first compressed service data, and the server 104 may decode the binary coding range information according to the data coding bin to obtain target data belonging to the target dimension in the first compressed service data, decompress the data according to the target data belonging to the target dimension in the first compressed service data, and obtain decompressed first service data, thereby implementing data decompression processing for the first service data. In some embodiments, the server 104 feeds back the decompressed first service data to the terminal 102. In some embodiments, the decompression processing method for the first compressed service data may also be implemented by the server 104 or the terminal 102 alone.

The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, projection devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The head-mounted device may be a Virtual Reality (VR) device, an augmented Reality (Augmented Reality, AR) device, smart glasses, or the like. The server 104 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud computing services.

In an exemplary embodiment, as shown in fig. 2, a service data processing method is provided, and an example of application of the method to the server in fig. 1 is described, which includes the following steps 202 to 208. Wherein:

step 202, obtaining first service data, wherein target data belonging to a target dimension in the first service data is range data, and the range data comprises at least two valued data.

The business data may be data in a business system, different business systems may include different business data, for example, salary data in an enterprise employee in a financial system, performance data of the enterprise employee in an enterprise assessment system, for example, examination performance data of a student in an education system, and account flow data of a user in a banking system. The first business number comprises data of a target dimension, and the target data belonging to the target dimension in the first business number is range data comprising at least two values. In different business systems, the target dimension may correspond to different data dimensions, as the target dimension may include, but is not limited to, at least one of various dimensions including, payroll level, employee job level, customer satisfaction, product quality, sales performance, employee performance, customer value, and the like. The range data may include at least two values, i.e., the data may include a range. For example, if the target dimension includes salary and the salary, the data in the salary and the salary dimension in the first service data includes at least two values, that is, the data in the salary and the salary dimension in the first service data is range data, for example, the data in the salary and the salary dimension in the first service data a may be "1 st grade to 2 nd grade 4 grade" range data, and specifically may include 9 values of "1 st grade, 2 nd grade, 1 st grade, 3 rd grade, 1 st grade, 4 th grade, 1 st grade, 5 th grade, 2 nd grade, 1 st grade, 2 nd grade, 3 rd grade, 2 nd grade, and" 4 nd grade.

Optionally, the server may acquire first service data that needs to be subjected to compression processing, where the first service data includes target data of a target dimension, and the target data is range data, that is, the target data includes at least two values. In some embodiments, the server may obtain first service data from a database of the service system that needs to be processed for compression.

Step 204, determining a data code system based on the value range of the target data in the first service data.

The target data comprises a plurality of values, and the value range of the target data can be determined according to each value of the target data. For example, when the target dimension is the employee performance, the target data belonging to the employee performance in the first business data B may include six data such as "first grade", "first grade second grade", "first grade third grade", "second grade" and "third grade second grade", that is, the value range of the target data may be determined to be "first grade" to "third grade second grade". The data coding system is a coding mode aiming at target data, and can be specifically adopted by coding aiming at the target data. The data code system may be determined based on a range of values of the target data, e.g., the corresponding data code system may be determined based on a number of values in the range of values.

Alternatively, for the first service data, the server may determine a value range of the target data based on each value of the target data, and the server may determine a corresponding data code system based on the value range. In some embodiments, the server may perform dimension division according to a relationship between the values of the target data in the first service data, and determine the corresponding data code system based on a dimension division result and a value range. For example, the range of the target data in the first service data C is "C1 to C30", that is, the target data may include 30 one-dimensional values, the server may determine that the data code system is thirty-system, and for another example, the range of the target data in the first service data D is "D1E1-D1E6, D2E1-D2E6, D3E1-D3E6, D4E1-D4E6, D5E1-D5E6", that is, the target data also includes 30 values, and the 30 values may be divided into two-dimensional data of 5 (D1, D2, D3, D4, D5) 6 (E1, E2, E3, E4, E5, E6), and the server may determine that the data code system is six-system.

And 206, coding the target data in the first service data according to the data coding system to obtain a system coding result, wherein the system coding result comprises the data coding system and the system coding range information corresponding to the target data.

The binary coding result is a coding result obtained by coding the target data, and the binary coding range information is range information obtained based on the corresponding binary coding result of each value of the target data, so that the value range of the target data can be represented by the binary coding range information. The corresponding target data can be restored by decoding through the combination of the binary coding range information and the corresponding data coding system.

For example, the server may encode the target data in the first service data according to the data encoding system, so as to obtain a corresponding encoding result, where the encoding result may include the data encoding system and the encoding range information corresponding to the target data. For example, the range of values of the target data in the first service data D is "D1E1-D1E6, D2E1-D2E6, D3E1-D3E6, D4E1-D4E6, D5E1-D5E6", after the data coding system is determined to be in the six-system, the target data is encoded according to the six-system, so that each value of the target data can be encoded into "00-05,10-15,20-25,30-35,40-45", and further converted into the decimal system, and the obtained binary encoding result can be "6, 0-29", wherein 6 represents that the data coding system is in the six-system, 0-29 represents that the binary encoding range information corresponding to the target data, and each sequence number in 0-29 can respectively correspond to the value of one target data.

In some embodiments, the value ranges of the target data in each first service data are different, so that corresponding data coding system can be determined according to the value ranges of the target data in each first service data, and coding processing can be performed on the first service data through the corresponding data coding system, so that compression processing can be performed on each first service data. In some embodiments, the server may also determine the data code system comprehensively based on the respective value ranges of all the first service data, where the comprehensively determined data code system may be used to perform compression processing on the target data in all the first service data respectively.

And step 208, carrying out data compression on the first service data based on the binary coding result to obtain first compressed service data corresponding to the first service data.

The first compressed service data is a compression result obtained after data compression is performed on the first service data. Optionally, the server may perform data compression on the first service data based on the result of the binary encoding, for example, the server may replace the target data in the first service data with the corresponding result of the binary encoding, thereby implementing data compression on the first service data and obtaining the first compressed service data. When the length of each character string of the target data is longer and the number of the target data is large, the data length of the target data is long, and after the target data is compressed according to the data coding scale, a data coding result comprising the data coding scale information corresponding to the data coding scale and the target data can be obtained, and the data length of the data coding result can be effectively compressed, so that the data compression of the first service data is realized. The server may obtain a data compression result based on the first service data, and store or transmit the data compression result. In some embodiments, after receiving the first compressed service data, the server performs data decompression on the first compressed service data to obtain a binary coding result, and the server may decode based on the data coding binary and the binary coding range information included in the binary coding result to restore to obtain target data, and obtain the first service data based on the target data, so as to implement decompression on the first compressed service data, and restore to obtain the original first service data.

In the service data processing method, for the first service data of which the target data belonging to the target dimension is the range data, determining a data coding system based on the value range of the target data in the first service data, coding the target data in the first service data according to the data coding system to obtain a system coding result comprising the data coding system and the information of the corresponding system coding range of the target data, and performing data compression on the first service data based on the system coding result to obtain the first compressed service data corresponding to the first service data. For the first service data of which the target data belonging to the target dimension is the range data, the data coding system determined according to the value range of the target data is coded to obtain a binary coding result comprising the data coding system and the binary coding range information corresponding to the target data, so that the compression processing of the first service data is realized, the compression processing of the service data is simplified, and the compression processing efficiency of the service data is improved.

In an exemplary embodiment, as shown in fig. 3, the process of performing the binary encoding, that is, encoding the target data in the first service data according to the data encoding system, obtains a binary encoding result, which includes steps 302 to 308. Wherein:

step 302, performing sequence mapping on the value range to obtain a value sequence range corresponding to the value range.

The sequence mapping is mapping processing of mapping a value range of target data in the first service data into a value sequence range, wherein the value sequence range is a sequence mapping result of the value range of the target data, and the value sequence range characterizes the value range of the target data in a value form. By mapping the target data into a numerical sequence range, the numerical sequence range can be coded according to a data coding system to carry out coding compression on the target data.

For example, the server may perform sequence mapping for the value range of the target data, e.g., the server may perform sequence mapping for the value range { a, B, C, D, E, F } of the target data to obtain a value sequence range corresponding to the value range of the target data, e.g., {1-6}, where each sequence number in 1-6 corresponds to each value in the value range, e.g., 1-table value a, 2-table value B, 3-table value C, 4-table value D, 5-table value E, 6-table value F. In some embodiments, the server may arrange the values of the target data into a sequence, map each value in the sequence to a value, and obtain a range of values based on the value corresponding to each value. For example, the server may map each value in the value range { A, B, C, D, E, F } of the target data to {10,20,30,40,50,60} respectively, resulting in the value sequence range {10-60}. In some embodiments, the server may also map directly to the range of values of the target data, e.g., the server may map the range of values { A-F } of the target data directly to {1-6}.

And step 304, carrying out the binary conversion on the numerical value sequence range according to the data coding system to obtain a binary conversion result of the target data under the data coding system.

The numerical sequence range is characterized by a numerical form aiming at the numerical range of target data, and can be subjected to binary conversion, so that the numerical sequence range is further subjected to coding compression according to a binary system. Optionally, the server may perform a binary conversion on the value sequence range according to the data coding system, and encode the value sequence range by converting the data coding system to obtain a binary conversion result of the target data under the data coding system, so that each value of the target data may be represented by a corresponding binary conversion result. In some embodiments, the server may record a mapping relationship between each value of the target data and each binary conversion result, so that decoding of the target data may be performed based on the mapping relationship.

Step 306, obtaining the binary coding range information corresponding to the target data based on the binary conversion result of the target data under the data coding system.

For example, the server may obtain, for each value of the target data, the binary conversion result corresponding to the target data, and the binary coding range information corresponding to the target data. For example, each value of the target data may include {XXXXXX11,XXXXXX12,XXXXXX13,XXXXXX14,XXXXXX21,XXXXXX22,XXXXXX23,XXXXXX24,XXXXXX31,XXXXXX32,XXXXXX33,XXXXXX34}, mapping the value range sequence to obtain a value sequence range {11-34}, where each value is specifically characterized as {11,12,13,14,21,22,23,24,31,32,33,34}, if the data code system is quaternary, after performing the binary conversion according to the quaternary pair, each value may be obtained as {00,01,02,03,10,11,12,13,20,21,22,23}, and the obtained binary code range information may include {00-23}. In some embodiments, further binary conversions may be performed for the binary coded range information, such as decimal conversions, whereby the update yields a binary coded range information of {00-11}.

And 308, obtaining a binary coding result according to the data coding system and the binary coding range information corresponding to the target data.

Optionally, the server may combine the data code system and the corresponding code range information of the target data to obtain the code result of the target data. For example, the data coding system is quaternary, the coding range information is {00-11}, and the coding result is {4:00-11}, so that each value "{XXXXXX11,XXXXXX12,XXXXXX13,XXXXXX14,XXXXXX21,XXXXXX22,XXXXXX23,XXXXXX24,XXXXXX31,XXXXXX32,XXXXXX33,XXXXXX34}" of the target data is compressed to the coding result of {4:00-11}, the data compression rate can be greatly improved, and the compression effect is ensured. In some embodiments, the server may perform a binary decoding on the binary encoding result based on the binary encoding result being "{4:00-11}" and a mapping relationship between each value of the target data and each respective binary conversion result, thereby restoring each value of the target data.

In this exemplary embodiment, after the server maps the value range to the value range through the sequence, the binary code is converted according to the data coding binary, the binary code range information is determined based on the obtained binary code conversion result, and the binary code result is obtained according to the data coding binary code and the binary code range information, so that the data coding binary code determined according to the value range of the target data is encoded, and the compression processing of the service data is simplified without complex compression algorithm processing, thereby improving the compression processing efficiency of the service data.

In an exemplary embodiment, the service data processing method further includes generating an update according to the data coding system, determining an updated data coding system, updating a result of the binary coding in the first compressed service data according to the updated data coding system to obtain updated first compressed service data, wherein the updated first compressed service data includes an updated result of the binary coding, and the updated result of the binary coding includes information of a range of the binary coding corresponding to the updated data coding system and the target data.

For example, the data code system for the target data may be updated according to actual needs, e.g., after the value range of the target data is changed, the data code system may be updated. In some embodiments, the server may detect a value range of the target data, and when it is determined that the value range of the target data changes, the server may determine, according to the changed value range, whether the data code system needs to be updated, and when the data code system needs to be updated, the server determines the updated data code system according to the updated value range. In some embodiments, the data encoded system update may also be actively triggered by the user, and the server may determine the updated data encoded system. The server may perform corresponding update on each first compressed service data according to the updated data coding system, and may specifically perform update on the binary coding result in the first compressed service data, thereby obtaining updated first compressed service data. After the first compressed service data is updated, wherein the updated system coding result is obtained by coding based on the updated data coding system, the updated first compressed service data comprises the updated system coding result, and the updated system coding result comprises the corresponding obtained system coding range information of the updated data coding system and the target data according to the updated data coding system.

In this exemplary embodiment, when the data coding system generates the update, the server obtains the updated first compressed service data according to the updated data coding system for the binary coding result in the first compressed service data, so that the corresponding update can be performed in time according to the change of the data coding system, and the accuracy of the first compressed service data is ensured.

In an exemplary embodiment, the service data processing method further includes obtaining second service data associated with the first service data, wherein the second service data comprises data belonging to a target dimension, performing serialized dimension reduction on the second service data according to the data dimension to obtain dimension reduced second service data, mapping the dimension reduced second service data belonging to the target dimension into sequence number information to obtain compressed second service data, and obtaining second compressed service data based on the compressed second service data and mapping relations between the compressed second service data belonging to the target dimension and all sequence number information.

The second service data are different service data with association relation with the first service data in the service system, and the first service data and the second service data comprise data of a target dimension, so that the first service data and the second service data can be associated based on the target dimension. For the first service data and the second service data which both comprise the data of the target dimension, the first service data and the second service data can be combined based on the target dimension, for example, the first service data and the second service data can be combined according to the target dimension, so that new service data is obtained. The second business data may include a plurality of data dimensions, where each data dimension may include corresponding data, for example, the second business data may be a student examination score in the educational system, and specifically may include an examination score of each subject, and each subject may be a data dimension, so that each second business data may include an examination score of each subject of the student. The sequence number information is a result obtained by mapping the data belonging to the target dimension in the second service data, so that the corresponding data is represented by the sequence number information. For example, data 1 in the target dimension is { XXXXXXXXX1}, which may be mapped to {1}, data 2 in the target dimension is { WWWWWWWWWW }, which may be mapped to 2, so that data compression for the first service data may be achieved through sequence number information. A mapping relationship can be set between the sequence number information and the corresponding represented data, so that the compressed data can be decompressed and restored based on the mapping relationship from the sequence number information query and restore to the corresponding data.

The server may, for example, be able to retrieve the second service data associated with the first service data, in particular, the second service data associated with the first service data, including the data belonging to the target dimension, from a query in a database of the service system. The server may perform data compression processing on the second service data according to the data dimension, specifically may perform serialization dimension reduction on the second service data according to the data dimension, for example, may perform serialization dimension reduction on the second service data based on JSON (JavaScript Object Notation, javaScript, object notation) mode, to obtain the second service data after dimension reduction. The server can map the data belonging to the target dimension to the sequence number information, so that the data belonging to the target dimension is mapped and compressed to obtain compressed second service data. In some embodiments, when the server maps the data belonging to the target dimension into sequence number information, a mapping relationship between each data and each sequence number information may also be recorded. The server can construct and obtain the second compressed service data based on the compressed second service data and the mapping relation between the data and the respective sequence number information, so as to realize the compression processing of the second service data.

In this exemplary embodiment, the server may perform serialized dimension reduction on the second service data including the target dimension data according to the data dimension, and map the data belonging to the target dimension into sequence number information, so as to implement compression processing on the second service data, obtain second compressed service data, and compress the second service data by using a mapping method of serialized dimension reduction and sequence number information, so as to ensure compression processing efficiency of the second service data.

In an exemplary embodiment, as shown in fig. 4, a service data processing method is provided, and an example of application of the method to the server in fig. 1 is described, which includes the following steps 402 to 408. Wherein:

step 402, acquiring a service data processing request, and acquiring first compressed service data according to the service data processing request.

The service data processing request is used for requesting to process the service data, and specifically can be obtained by sending the service data processing request to a server through a terminal by a user. The first compressed service data is obtained by compressing the first service data, and the first compressed service data is decompressed to obtain the first service data, so that corresponding service processing is performed based on the first service data. The target data belonging to the target dimension in the first service data is range data, and the range data is data comprising at least two values.

For example, the user may send a service data processing request to the server through the terminal, where the service data processing request may carry a data identifier or service data that needs to be processed, and the server may obtain, based on the service data processing request, first compressed service data that needs to be processed. In some embodiments, the service data processing request may be directly carried in the service data processing request, and the server may directly parse the service data processing request to obtain the first compressed service data. In some embodiments, the service data processing request may carry a data identifier, and the server may query based on the data identifier to obtain corresponding first compressed service data.

Step 404, obtaining a binary coding result from the first compressed service data, where the binary coding result includes a data coding bin and binary coding range information corresponding to data belonging to the target dimension in the first compressed service data.

The server may obtain a binary encoding result based on the first compressed service data, where the binary encoding result includes a data encoding bin and binary encoding range information, where the data encoding bin describes an encoding manner for target data belonging to a target dimension in the first service data, and the binary encoding range information is an encoding result corresponding to a value range of the target data. And decoding the target data in the first service data based on the data coding system and the binary coding range information, so as to realize decompression and restoration of the first service data.

Step 406, decoding the binary coding range information according to the data coding system to obtain target data belonging to the target dimension in the first compressed service data, wherein the target data is range data comprising at least two values.

Optionally, the server may decode the binary coding range information according to the data coding system to obtain target data belonging to the target dimension, so as to decode and restore the target data carried in the first compressed service data.

Step 408, based on the target data belonging to the target dimension in the first compressed service data, performing data decompression on the first compressed service data to obtain the first service data.

For example, the server may decompress the data for the first compressed service data based on the decoded target data, e.g., the server may replace the decoded target data for the binary coding result in the first compressed service data, thereby implementing data decompression for the first compressed service data and obtaining the first service data.

In the service data processing method, first compressed service data is obtained according to the service data processing request, a binary coding result is obtained from the first compressed service data, the binary coding result comprises a data coding binary and binary coding range information corresponding to data belonging to a target dimension in the first compressed service data, decoding is conducted on the binary coding range information according to the data coding binary to obtain target data belonging to the target dimension in the first compressed service data, and data decompression is conducted on the first compressed service data based on the target data belonging to the target dimension in the first compressed service data to obtain decompressed first service data. For the first compressed service data comprising the binary coding result, decoding the included binary coding range information according to the data coding system in the binary coding result, and performing data decompression based on the target data belonging to the target dimension obtained by decoding to obtain the first service data, so that the decompression processing of the first service data is simplified, and the decompression processing efficiency of the service data is improved.

In an exemplary embodiment, decoding is performed on the binary coding range information according to the data coding system to obtain target data belonging to a target dimension in the first compressed service data, wherein the decoding comprises determining a binary conversion result under the data coding system based on the binary coding range information, performing binary conversion on the binary conversion result according to the data coding system to obtain a numerical value sequence range, and performing mapping reduction on the numerical value sequence range to obtain target data belonging to the target dimension in the first compressed service data.

The method comprises the steps of representing each value of target data in a target dimension in first service data through respective corresponding binary conversion results, and encoding a binary number sequence range through a binary conversion method by using binary conversion result data. For example, the server may determine the binary conversion result under the data encoding scale information based on the binary encoding scale information, e.g., the server may spread each of the binary conversion results in the binary encoding scale information, thereby obtaining a binary conversion result corresponding to each of the values of the target data. The server can carry out the binary conversion on the binary conversion result according to the data coding system so as to obtain a numerical value sequence range, wherein the numerical value sequence range is a sequence mapping result of the numerical value range of the target data, and the numerical value sequence range represents the numerical value range of the target data in a numerical value form. The server may perform mapping reduction on the numerical sequence range to obtain target data belonging to the target dimension in the first compressed service data.

In this exemplary embodiment, the server decodes based on the data code system and the data code range information in the code system result, so as to decompress the first compressed service data, and does not need to decompress through a complex compression algorithm, thereby simplifying the decompression of the service data and improving the decompression efficiency of the service data.

In an exemplary embodiment, the service data processing method further includes obtaining second compressed service data according to a service data processing request, obtaining compressed second service data from the second compressed service data, and a mapping relation between data belonging to a target dimension and each serial number information in the compressed second service data, performing mapping reduction on serial number information corresponding to the target dimension in the second compressed service data according to the mapping relation to obtain dimension-reduced second service data, and performing dimension-increasing reduction on the dimension-reduced second service data according to a data dimension to obtain second service data, wherein the second service data comprises data belonging to the target dimension.

The second compressed service data is obtained by compressing the second service data, and the second service data comprises data belonging to a target dimension. In an exemplary embodiment, when the service data processing request further includes second compressed service data or includes a data identifier of the second compressed service data, the server may obtain the second compressed service data based on the service data processing request, and parse the second compressed service data to obtain a mapping relationship between the compressed second service data and data belonging to the target dimension and each sequence number information. The server can perform mapping restoration on sequence number information corresponding to the target dimension in the second compressed service data according to the mapping relation so as to restore the second service data after the dimension reduction, wherein the data belonging to the target dimension in the second service data after the dimension reduction is restored. The server can perform dimension-lifting restoration according to the data dimension based on the dimension-reduced second service data, so that data decompression aiming at the second compressed service data is realized, the second service data is obtained, and corresponding service processing is performed based on the second service data.

In this exemplary embodiment, the server may decompress data for the second compressed service data based on the mapping relationship of the data dimension and the sequence number information, and may determine the compression processing efficiency of the second service data.

In an exemplary embodiment, the business data method further comprises determining a business processing data range based on the business data processing request, traversing from the first business data according to the business processing data range to obtain first target business data, wherein data belonging to a target dimension in the first target business data comprises single valued data, obtaining second target business data from the second business data according to the business processing data range, wherein the data belonging to the target dimension in the second target business data is matched with the data belonging to the target dimension in the first target business data, and conducting business processing on the first target business data and the second target business data according to the business data processing request.

The service processing data range is used for screening service data so as to determine the service data to be processed, and specifically, the first service data and the second service data obtained by decompression can be screened so as to determine the data to be processed according to the service data processing request. The first target service data is first service data which is required to be processed currently according to the service processing data range, and the data value under the target dimension in the first target service data is single, namely the data belonging to the target dimension in the first target service data is 1 value. The second target service data is second service data which is required to be processed currently according to the service processing data range, the data belonging to the target dimension in the second target service data is matched with the data belonging to the target dimension in the first target service data, if the data belonging to the target dimension in the second target service data is the same as the data belonging to the target dimension in the first target service data, the first target service data and the second target service data can be processed jointly through the target dimension.

For example, the server may determine the service processing data range based on the service data processing request, e.g., the server may parse the service data processing request to determine the service processing data range based on the data range field of the service data processing request. The server may screen the first service data obtained by decompression based on the service processing data range to obtain first target service data, where data belonging to the target dimension in the first target service data is data including a single value, and where data belonging to the target dimension in the first target service data is split into a single value. For example, the target data values belonging to the target dimension in the first service data a include A, B and C, and the splitting may be performed according to each value of the target data, so as to split the first service data a into the first service data A1 (the data value belonging to the target dimension is a), the first service data A2 (the data value belonging to the target dimension is B), and the first service data A3 (the data value belonging to the target dimension is C), where the server may determine that the first service data A1 and the first service data A2 belong to the first target service data from the first service data a when the first service data A1 and the first service data A2 belong to the service processing data range.

The server may screen the second service data obtained by decompression according to the service processing data range, so as to screen second target service data from the second service data, where the second target service data is associated with the first target service data, and specifically, data belonging to the target dimension in the second target service data is matched with data belonging to the target dimension in the first target service data. For example, the data belonging to the target dimension in the second target service data B1 is the same as the data belonging to the target dimension in the first target service data A1, and for example, the difference between the data belonging to the target dimension in the second target service data B2 and the data belonging to the target dimension in the first target service data A2 does not exceed the difference threshold. The server can jointly perform corresponding service processing on the obtained first target service data and second target service data according to the service data processing request, for example, the server can splice the first target service data and the second target service data according to the target dimension, so as to obtain spliced target service data. The server can also feed back the spliced target service data to the user terminal so as to intuitively display the target service data in the terminal.

In this exemplary embodiment, the server screens the first target service data and the second target service data, which are matched, from the decompressed first service data and the decompressed second service data according to the service processing data range determined by the service data processing request, and performs corresponding service processing on the first target service data and the second target service data according to the service data processing request, so that the processing efficiency of the service data can be ensured.

The application also provides an application scene, which applies the service data processing method. Specifically, the application of the service data processing method in the application scene is as follows:

The application scenario relates to processing of employee compensation data in an enterprise financial system, in which, for range data comprising a plurality of values, in terms of data storage, the storage space occupation is effectively reduced by serializing the range data into text storage using JSON format, and simultaneously adopting a universal text compression tool to further shorten the text length. For example, the original information is "{17943334254:[17911232211,17911232212,17911232213],17943334255:[17911232211,17911232212,17911232213],17943334256:[17911232211,17911232212,17911232213]}"., the text is compressed by using an lz series algorithm (a dictionary coding-based data compression algorithm, by extracting all phrases and replacing the phrase itself with the subscript of the phrase in the whole text), and the possible compression result is "map:{17943334254,1794333425517943334256,17911232211,17911232212,17911232213},val:{0:[3,4,5],1:[3,4,5],2:[3,4,5]}"., when the data is used, the text is read into a memory, decompressed by using the compression algorithm, and the decompressed text is inversely sequenced into readable mapping table information. However, the data compression result must be decompressed and deserialized for use, so that the data is frequently decoded and converted, thereby wasting computing resources, resulting in lower processing efficiency of compression and decompression of the service data, and reducing processing efficiency of the service data.

Based on this, in the service data processing method provided by the application, the service data including the range data is compressed and decompressed by adopting the system, and the main points are how the original data is encoded, the encoding result is directly used for calculation (without decoding), and the decoding mode of the data. Furthermore, the business data processing method provided by the application understands the scope of the range information, and the range information is emphasized to be formed by one-dimensional, two-dimensional or multi-dimensional points, has the high-low level property of numerical values in mathematics, and has continuity. Furthermore, the range data is encoded by using a binary scheme, and the change of a carry system needs to be closely focused, so that the error identification code information is prevented from being obtained due to the failure of data decoding.

Specifically, as shown in fig. 5, in the application scenario, the service data of the service data processing method provided by the present application may include first service data (service data B) and second service data (service data a), where the target dimension includes a salary dimension such as salary, that is, data in the salary dimension such as salary in service data B includes a plurality of values. For the service data A, the server can compress through data reduction and primary key mapping to obtain compressed data A and store the compressed data A, and for the service data B, the server can compress through a binary coding mode to obtain compressed data B and store the compressed data B. When the Cartesian calculation processing is needed for the service data A and the service data B, the server can obtain the service data A from the database by carrying out row number and then carrying out data ascending and sequence number mapping decompression, and the server can obtain the service data B from the database by carrying out row number and then carrying out binary decoding for the service data B by calculating the total number needed to be obtained. The server can perform Cartesian calculation processing based on the service data A and the service data B obtained through decompression, and preview and display the calculation result.

When the dimension of the service data A is reduced, the dimension reduction is realized through data dimension reduction and main key mapping. The structure of the business data A is a three-dimensional data structure, wherein the three dimensions are items, salaries and the like, and salaries, and the three dimensions are shown in the following table 1. Projects such as overtime payroll, performance payroll, post payroll and the like, salary and the like are salary grade divisions such as 200 and the like and 100 and the like, and salary grades are further divisions of salary grade such as 1 grade and 2 grade.

TABLE 1

When the number of items=5, the number of salary etc. =200, the number of salary files=100, the total number of data lines of the service data a=5×200×100=10w, so that the time required for storing 10w lines of data is longer. In order to accelerate service response, the original three-dimension is reduced to one dimension by carrying out data dimension reduction on the service data A, only the dimension of the project is reserved, 2 dimensions of salary and the like and salary files are serialized into json character strings, and 10w data are converted into 5 rows after dimension reduction, so that the service response of storage and inquiry is accelerated.

Meanwhile, 19 digits of a main key with relatively long salary and the like and a salary file are mapped into a short serial number in a main key mapping mode, so that the aim of saving storage space is fulfilled, and the specific main key mapping relation of the dimensions of the salary and the like is shown in the following table 2.

TABLE 2

Salary etc	Main key	Sequence number
			1, Etc	1766825583079188488	1
2, Etc	1766825583079188489	2
			3, Etc	1766825583079188490	3

The primary key mapping of the salary dimension may be as shown in table 3 below.

TABLE 3 Table 3

Firewood grade	Main key	Sequence number
			1 St gear	1766825073790019584	1
2 Gear	1766825073790019585	2
			3 Gear	1766825073790019586	3
4 Th gear	1766825073790019587	4
			5 Th gear	1766825073790019588	5

The data structure obtained by compressing the service data a can be shown in table 4 below.

TABLE 4 Table 4

For the service data B, the data structure is a multidimensional data structure, the number of the attributes has uncertainty, the attributes such as posts, departments, job levels and the like, the service data B can comprise 2 attributes, 3 attributes, 10 attributes and the like, and salaries and the like and salary files are range data. The details are shown in tables 5 and 6 below.

TABLE 5

Post	Department(s)	Firewood such as firewood
			Software development	Long sand 1 part	1 Grade to 20 grade 8 grade
Software development	Shenzhen 1 part	6 Grade 1 grade to 20 grade 8 grade
			Product design	Long sand 1 part	1 Grade to 10 grade 8 grade
Product design	Shenzhen 1 part	3 Grade 1 grade to 10 grade 8 grade

TABLE 6

Post	Department(s)	Job level	Firewood such as firewood
				Software development	Long sand 1 part	T2	1 Grade to 20 grade 8 grade
Software development	Shenzhen 1 part	T3	6 Grade 1 grade to 20 grade 8 grade
				Product design	Long sand 1 part	P1	1 Grade to 10 grade 8 grade
Product design	Shenzhen 1 part	P2	3 Grade 1 grade to 10 grade 8 grade

Wherein, for range data, such as when the salary number is 3, the salary number is 8. The 1 st grade to 2 nd grade 8 can be {1 st grade, 2 nd grade, 1 st grade, 3 rd grade, 1 st grade, 4 th grade, 1 st grade, 5 th grade, 1 st grade, 6 th grade, 1 st grade, 7 th grade, 1 st grade, 8 th grade, 2 nd grade, 1 st grade, 2 nd grade, 3 nd grade, 2 nd grade, 4 th grade, 2 nd grade, 5 th grade, 2 nd grade, 6 th grade, 2 nd grade, 7 th grade, 2 nd grade, 8 th grade }, namely the data comprises a plurality of values, and belongs to range data.

Further, as shown in fig. 6, when performing the processes of binary encoding, calculation and decoding on the service data B, the service data B of 3 salary grades, such as 3 salary grades, may be encoded according to a binary system, so as to obtain an encoding result { rangeCode: "0-8", ranknum:3}. When updating the code scale, the server can obtain a new code result { rangeCode: "0-14", ranknum:6} based on the new scale after resolving the subscript based on the old scale, wherein the salary change represents the change of the carry scale, i.e. the change of the scale is represented. When the difference union calculation processing is performed on different business data B, the set operation can be directly converted into the combination calculation of the digital interval, so that the processing efficiency of the difference union calculation is improved. For range data query, the carry value can be decoded into the mapping relation of the service data id and the query response, so that the queried data is obtained.

Further, the server can greatly reduce the occupied space by converting salary ranges such as salary ranges based on an algorithm of a system, and meanwhile, the calculation performance of the server for poor calculation, union calculation and the like is better. And the paycheck number is used as the dynamic coding of paycheck range information such as math system paycheck and the like, so that the occupation of storage space is reduced. Meanwhile, the coding result supports rapid calculation of the equal-gear range (especially in the aspect of cross-union calculation), and the calculation memory consumption is saved. For example, in the range of 3-gear 3, etc., the data encoding process may include:

The certain range is (1 grade, etc. 1 grade, 3 grade, etc.), the corresponding identification number information is:

{

17943334254:[17911232211,17911232212,17911232213],

17943334255:[17911232211,17911232212,17911232213],

17943334256:[17911232211,17911232212,17911232213]

}

The identification number information is converted into a subscript value, and the result is "{0: [0,1,2],1: [0,1,2],2: [0,1,2] }. The number in brackets is considered the lower of the two digits and the number before the colon is considered the upper of the two digits, resulting in a nine ternary digit sequence [00,01,02,10,11,12,20,21,22]. I.e. when in the 3*3 dimension, a maximum of 9 digits of the data range is obtained. In the above process, virtually all numbers within the enumerated range, as the dimension value becomes large, its range can be recorded. Specifically, the range of [00,01,02,10,11,12,20,21,22] above can be calculated to be 0-8 (decimal), but in the practical situation, enumeration and evaluation are not needed in the encoding stage, because in the using process, the beginning and ending of the value range can be set, for example, the beginning range of the value is 1, etc., the 1 grade, the ending range of the value is 3, etc., the 3 grade can be set, the obtained subscript range is 01-22, then the subscript range is converted into decimal 0-8, and the stored form is "{ rangeCode:"0-8", ranknum:3}", and the stored information comprises the binary sum range, wherein the stored information can be interpreted as the range between 0 and 8 of 3.

Through this coding scheme, a huge id mapping table can be converted into a very short text string. And this text string is easily understood. In particular, data for a range can be easily targeted to compute intersections, union. For example, a multi-range data "1 grade, 3 grade, 2 grade, 1 grade, 4 grade, 1 grade, 5 grade, 3 grade, etc. because there may be overlapping between ranges, a union is needed to be calculated, and according to the storage form described above, the range data is actually taken to be" 0-14,6-18,24-32", the result of the union can be calculated very fast to be" 0-18;24-32", and similarly, the intersection can be calculated easily.

Further, when the data code system is changed from 3 to 6, if the user has shifted up or down the gear, the previously recorded result must not be accurate (the numbers are all 3), and the conversion needs to be performed by 6, and the conversion is performed by calculating the high-low index (high-order of 0=0 (0/3), low-order of 0=0 (0%3)) by dividing the value of the system by the modulus operation. Then the range of 0-8 is calculated to the lower elevation and lower position of the initial range to be 0/0 and 2/2 respectively, and then new values are calculated by using a system of 6 to obtain a start value of 0 (0 x 6+0) and an end value of 14 (2 x 6+2), so that the obtained new stored values are "{ rangeCode:"0-14", ranknum:6 }. When the service needs to query the range information, the server reads the stored range '0-3' (4 scale), and the server needs to tile the range data to obtain '0, 1,2, 3', and then obtains the high and low bits of each range value as '0= (0, 0), 1= (0, 1), 2= (0, 2), 3= (0, 3)', and then obtains the corresponding identification number as '17943334254:' [17911232211,17911232212,17911232213,17911232213] } according to the subscript of the high and low bits.

When the service data A and the service data B obtained by decompression are subjected to Cartesian calculation, the method can be realized based on a memory Cartesian number taking model. In terms of Cartesian computation, most of the prior art directly stores result data of the Cartesian computation, and then relieves the pressure of a database by means of a horizontal sub-table to solve the performance problem of interaction. And the storage result can not support the reading and identification of information by human eyes, and the readability is poor. The service data processing method provided by the application does not directly store the Cartesian calculation result, but only stores the original data, and then accurately reads part of the original data to the memory to perform Cartesian calculation to obtain the desired data. Because the data volume of the Cartesian calculation results is too large, it can reach billions or even billions.

As shown in fig. 5, in order to realize the preview effect after the service data a and the service data are subjected to the cartesian computation, that is, the service data B are completely spread and combined to display the service data a, and the cartesian product operation is required to be performed on each column of the service data B of each row. After the service data B is queried row by row from the database and "decoded in binary", the data structure of table 7 below can be obtained.

TABLE 7

On the display view, the data display is performed according to the page, and the current page number currentPage and the display data size pageSize of each page are transmitted. The invention dynamically calculates the Cartesian product in the memory through currentPage, pageSize, and returns the Cartesian calculation result to the view model for display.

1. Calculating Cartesian total number of lines

totalCount=row1Count+row2Count+row3Count+...

Row1 count=count (attribute a) ×count (attribute B) ×count (attribute Z) ×count (salary range such as salary);

the number of rows with row number 1 in table 7:

row1Count=3*2*2*2*(2*100)=4800;

count (salary range of the salary etc. =1 st gear of the 1 st gear to the 1 st gear of the 100 nd gear of the 1 st gear to the 2 nd gear of the 100 st gear=1×100+1×100=200;

the number of rows as row number 2 in tables 2-7:

row2Count=1*1*1*2*(159*100+1*98)=31996;

count (salary range) =1 st to 159 st, 100 st to 160 st+98 st to 160 st=159×100+1×98=15998.

2. Calculating Cartesian starting coordinates

The server performs coordinate calculation by taking the salary range as units, the attribute Z as tens, the attribute C as hundreds, the attribute B as thousands. Here, the bit to ten bit system is not necessarily a decimal one, but is determined according to the number of salary ranges such as the salary of the current line. The bit to ten-bit system of row number 1 is count (salary etc. salary range) =200 and the bit to ten-bit system of row number 2 is count (salary etc. salary range) =15998 as in tables 2 to 7. Of course, the decimal to hundred, hundred to kilo, etc. system is not fixed, and is determined according to the number of attribute values of the current row and the current attribute column. Ten to hundred digits of row number 1 are count (attribute Z) =2 and hundred to thousand digits of row number 12 are count (attribute C) =1 as in tables 2-7. The method comprises the following steps:

(1) Starting parameters STARTPARAM, ending parameters endParam may be derived from currentPage, pageSize.

(2) The line startRowIndex of the Cartesian fetch start coordinates is calculated from the STARTPARAM size.

① Calculating the total number row1Count of the row1, when STARTPARAM < row1Count, indicating that the Cartesian fetch starting coordinate is at row1, returning, otherwise entering the next row;

② Calculating the total number row2Count of the row2, when STARTPARAM < row1count+row2Count, indicating that the Cartesian fetch start coordinate is in row2, returning, otherwise entering the next row;

③ Calculating the total number row3Count of the row3, and when STARTPARAM < row1count+row2count+row3Count, indicating that the Cartesian fetch start coordinate is in row3, returning, otherwise entering the next row;

(3) The exact position of the cartesian fetch start coordinates is calculated to determine from which column of attribute values the fetch starts.

① If startRowIndex has been determined in (2) on n rows, then the starting position needs to be calculated on n rows:

rowCalCount=startParam-row1Count-row2Count-...-rown-1Count;

② Initializing the cartesian fetch coordinates cartesianCoordinate of line n as:

{

The attribute a is 0 and,

The attribute B is 0 and,

The attribute C is 0 and,

The attribute Z is 0 and,

Firewood grade 0

}

③ Add to the different cartesian coordinates cartesian, add rowCalCount:

cartesianCoordinate = cartesianCoordinate + rowCalCount. The result is a cartesian starting coordinate.

3. Fetch starting from the Cartesian fetch start coordinates, fetch endParam-STARTPARAM number or return data with no data already, and target the Cartesian calculation process. As shown in fig. 7, in the case of taking a number, the server may set a pointer 1 of a salary file such as a salary file to point to GR1, select A1 against attribute a, select B1 against attribute B, select C1 against attribute N, and select N1 against attribute N, so that the taken data is obtained by combining data selected by each attribute, and may specifically be { N1,.. then, the next data is read, as shown in fig. 8, the specific pointer 1 may switch to point to GR2, so as to obtain { N1,., C1, B1, A1, GR2}, so that after traversing for each attribute, each service data may be obtained.

In the service data processing method provided by the application, the Cartesian calculation result is not directly stored, but only the original data is stored, and the required data is obtained by precisely reading part of the original data to the memory and performing Cartesian calculation. In addition, in the calculation of the Cartesian memory, the characteristics of Cartesian coordinate taking, and different bit, ten bit, hundred bit and the like of the Cartesian coordinate taking are abstracted. The business data processing method provided by the application can effectively reduce the occupation of the storage space of data and quicken the response, and the memory Cartesian calculation is utilized to realize the 3-second response of billion data loading by utilizing the characteristic that the memory calculation speed is far greater than the speed of a magnetic reading disk.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a service data processing device for realizing the above related service data processing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of one or more service data processing devices provided below may refer to the limitation of the service data processing method in the above description, which is not repeated here.

In one exemplary embodiment, as shown in fig. 9, there is provided a service data processing apparatus 900, including a first service data acquisition module 902, a code scale determination module 904, a code processing module 906, and a compression processing module 908, wherein:

The first service data obtaining module 902 is configured to obtain first service data, where target data belonging to a target dimension in the first service data is range data, and the range data is data including at least two values;

the code system determining module 904 is configured to determine a data code system based on a value range of target data in the first service data;

the encoding processing module 906 is configured to encode target data in the first service data according to a data encoding system, so as to obtain a binary encoding result, where the binary encoding result includes binary encoding range information corresponding to the data encoding system and the target data;

The compression processing module 908 is configured to perform data compression on the first service data based on the binary encoding result, so as to obtain first compressed service data corresponding to the first service data.

In an exemplary embodiment, the encoding processing module 906 is further configured to perform sequence mapping on the value range to obtain a value sequence range corresponding to the value range, perform binary conversion on the value sequence range according to the data encoding system to obtain a binary conversion result of the target data under the data encoding system, obtain binary encoding range information corresponding to the target data based on the binary conversion result of the target data under the data encoding system, and obtain a binary encoding result according to the data encoding system and the binary encoding range information corresponding to the target data.

In an exemplary embodiment, the encoding processing module 906 is further configured to determine an updated data encoding system when the data encoding system generates an update, update a result of the encoding in the first compressed service data according to the updated data encoding system to obtain updated first compressed service data, where the updated first compressed service data includes an updated encoding result, and the updated encoding result includes the updated data encoding system and the target data corresponds to the obtained encoding range information according to the updated data encoding system.

In an exemplary embodiment, the system further comprises a second service data processing module, wherein the second service data processing module is used for acquiring second service data associated with the first service data, the second service data comprises data belonging to a target dimension, the second service data is subjected to sequential dimension reduction according to the data dimension to obtain the second service data after dimension reduction, the data belonging to the target dimension in the second service data after dimension reduction is mapped into sequence number information to obtain compressed second service data, and the second compressed service data is obtained based on the compressed second service data and the mapping relation between the data belonging to the target dimension in the compressed second service data and each sequence number information.

In an exemplary embodiment, as shown in fig. 10, there is provided a service data processing apparatus 1000, including a processing request acquisition module 1002, an encoding result acquisition module 1004, a decoding processing module 1006, and a decompression processing module 1008, wherein:

a processing request acquisition module 1002, configured to acquire a service data processing request, and acquire first compressed service data according to the service data processing request;

The encoding result obtaining module 1004 is configured to obtain a binary encoding result from the first compressed service data, where the binary encoding result includes a data encoding bin and binary encoding range information corresponding to data belonging to a target dimension in the first compressed service data;

A decoding processing module 1006, configured to decode the binary coded range information according to a data coding system to obtain target data belonging to a target dimension in the first compressed service data, where the target data is range data including at least two values;

The decompression processing module 1008 is configured to decompress data for the first compressed service data based on target data belonging to a target dimension in the first compressed service data, to obtain the first service data.

In an exemplary embodiment, the decoding processing module 1006 is further configured to determine a binary conversion result under the data coding system based on the binary coding range information, perform binary conversion on the binary conversion result according to the data coding system to obtain a value sequence range, and perform mapping reduction on the value sequence range to obtain target data belonging to the target dimension in the first compressed service data.

In an exemplary embodiment, the system further comprises a second service data processing module, which is used for obtaining second compressed service data according to a service data processing request, obtaining the compressed second service data from the second compressed service data, and the mapping relation between the data belonging to the target dimension and each serial number information in the compressed second service data, carrying out mapping reduction on the serial number information corresponding to the target dimension in the second compressed service data according to the mapping relation to obtain the second service data after dimension reduction, carrying out dimension-increasing reduction on the second service data after dimension reduction according to the data dimension to obtain the second service data, wherein the second service data comprises the data belonging to the target dimension.

In an exemplary embodiment, the method further comprises a joint processing module, wherein the joint processing module is used for determining a service processing data range based on a service data processing request, traversing from first service data according to the service processing data range to obtain first target service data, wherein data belonging to a target dimension in the first target service data comprises single valued data, obtaining second target service data from second service data according to the service processing data range, the data belonging to the target dimension in the second target service data is matched with the data belonging to the target dimension in the first target service data, and performing service processing on the first target service data and the second target service data according to the service data processing request.

The various modules in the service data processing device described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one exemplary embodiment, a computer device is provided, which may be a server, and the internal structure thereof may be as shown in fig. 11. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing data involved in the business data processing method. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a business data processing method.

It will be appreciated by those skilled in the art that the structure shown in FIG. 11 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are both information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to meet the related regulations.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile memory and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (RESISTIVE RANDOM ACCESS MEMORY, reRAM), magneto-resistive Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computation, an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) processor, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the present application.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A method for processing service data, the method comprising:

determining a data coding system based on the value range of the target data in the first service data;

Coding the target data in the first service data according to the data coding system to obtain a binary coding result, wherein the binary coding result comprises binary coding range information corresponding to the data coding system and the target data;

2. The method of claim 1, wherein said encoding said target data in said first service data according to said data encoding system to obtain a result of the encoding comprises:

performing sequence mapping on the value range to obtain a numerical value sequence range corresponding to the value range;

Performing the binary conversion on the numerical value sequence range according to the data coding system to obtain a binary conversion result of the target data under the data coding system;

Based on a binary conversion result of the target data under the data coding system, obtaining the corresponding binary coding range information of the target data;

and obtaining a binary coding result according to the data coding system and the binary coding range information corresponding to the target data.

3. The method according to claim 1, wherein the method further comprises:

When the data coding system generates update, determining the updated data coding system;

Updating the binary code result in the first compressed service data according to the updated data code system to obtain updated first compressed service data;

The updated first compressed service data comprises an updated system coding result, and the updated system coding result comprises a system coding range information obtained by the corresponding coding of the updated data coding system and the target data according to the updated data coding system.

4. A method according to any one of claims 1 to 3, characterized in that the method further comprises:

Acquiring second business data associated with the first business data, wherein the second business data comprises data belonging to the target dimension;

Performing serialization dimension reduction on the second service data according to the data dimension to obtain dimension reduced second service data;

Mapping data belonging to the target dimension in the second service data after dimension reduction into sequence number information to obtain compressed second service data;

And obtaining second compressed service data based on the compressed second service data and the mapping relation between the data belonging to the target dimension in the compressed second service data and each serial number information.

5. A method for processing service data, the method comprising:

decoding the binary coding range information according to the data coding system to obtain target data belonging to the target dimension in the first compressed service data, wherein the target data is range data comprising at least two values;

And based on target data belonging to the target dimension in the first compressed service data, performing data decompression on the first compressed service data to obtain first service data.

6. The method of claim 5, wherein decoding the binary coded range information according to the data code system to obtain target data belonging to the target dimension in the first compressed service data, comprises:

Determining a binary conversion result under the data coding system based on the binary coding range information;

performing binary conversion on the binary conversion result according to the data coding binary system to obtain a numerical sequence range;

And carrying out mapping reduction on the numerical sequence range to obtain target data belonging to the target dimension in the first compressed service data.

7. The method according to claim 5 or 6, characterized in that the method further comprises:

obtaining second compressed service data according to the service data processing request;

Obtaining compressed second service data from the second compressed service data and mapping relations between data belonging to the target dimension and each serial number information in the compressed second service data;

according to the mapping relation, mapping and restoring sequence number information corresponding to the target dimension in the second compressed service data to obtain second service data with reduced dimension;

and carrying out dimension-lifting restoration on the second service data subjected to dimension reduction according to the data dimension to obtain the second service data, wherein the second service data comprises data belonging to the target dimension.

8. The method of claim 7, wherein the method further comprises:

determining a service processing data range based on the service data processing request;

Traversing from the first service data according to the service processing data range to obtain first target service data, wherein the data belonging to the target dimension in the first target service data comprises single valued data;

Obtaining second target business data from the second business data according to the business processing data range, wherein the data belonging to the target dimension in the second target business data is matched with the data belonging to the target dimension in the first target business data;

and performing service processing on the first target service data and the second target service data according to the service data processing request.

9. A traffic data processing apparatus, the apparatus comprising:

the system comprises a first service data acquisition module, a second service data acquisition module and a first service data processing module, wherein the first service data acquisition module is used for acquiring first service data, target data belonging to a target dimension in the first service data is range data, and the range data is data comprising at least two values;

The code system determining module is used for determining a data code system based on the value range of the target data in the first service data;

The coding processing module is used for coding the target data in the first service data according to the data coding system to obtain a binary coding result, wherein the binary coding result comprises the data coding system and binary coding range information corresponding to the target data;

10. A traffic data processing apparatus, the apparatus comprising:

The decoding processing module is used for decoding the binary coding range information according to the data coding system to obtain target data belonging to the target dimension in the first compressed service data, wherein the target data is range data comprising at least two values;

and the decompression processing module is used for carrying out data decompression on the first compressed service data based on the target data belonging to the target dimension in the first compressed service data to obtain the first service data.

11. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 4 or 5 to 8 when the computer program is executed.

12. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 4 or 5 to 8.

13. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 4 or 5 to 8.