WO2018165959A1 - E-commerce data cleaning system and method - Google Patents

Abstract

The present invention provides an e-commerce data cleaning system and method. The method comprises: adjusting the operation state of an e-commerce platform to be a state of stopping data exchange; calculating the hash values of each group of e-commerce data on the e-commerce platform; classifying equal hash values into one group, saving the equal hash values of each group and the storage path of the e-commerce data corresponding to the equal hash values in said group into a hash association list; deleting the e-commerce data corresponding to the non-selected storage paths in the hash association list; modifying, in each hash association list, the identifiers corresponding to the deleted e-commerce data; adjusting the operation state of the e-commerce platform to be a data exchange state; and receiving a query instruction sent by a terminal, and sending, to the terminal, the e-commerce data corresponding to the selected storage path in the hash association list corresponding to the query instruction. The present invention can accelerate the access speed of the e-commerce platform and reduce the redundancy of the e-commerce platform.

Description

 E-commerce data cleaning system and method

 [0001] The present invention relates to the field of data processing technologies, and in particular, to an e-commerce data cleaning system and method.

 Background technique

 [0002] With the rapid development of electronic commerce, people are increasingly accustomed to using e-commerce websites to purchase goods.

 However, sellers who are making profits through online channels cannot use the existing Internet information to predict the hot products afterwards, in order to achieve more profit.

 [0003] Data mining is the process of mining effective knowledge from large amounts of data stored in databases, data warehouses, or other information repositories. Data mining extracts implicit, valuable, and understandable information from large amounts of data to guide people's activities. Data mining techniques mainly include association rules, classification rules, cluster analysis and sequence patterns.

 [0004] Further, more and more users now use mobile terminals (for example, mobile terminals such as mobile phones, tablet computers, etc.) to view e-commerce data and store e-commerce data, resulting in e-commerce platforms storing multiple identical The e-commerce data, thus resulting in redundancy of e-commerce data on the e-commerce platform e-commerce platform and reducing the access speed of the e-commerce platform website.

 technical problem

 [0005] A primary object of the present invention is to provide an e-commerce data cleaning system and method for solving the technical problem of cleaning redundant e-commerce data on an e-commerce platform.

 Problem solution

 Technical solution

[0006] In order to achieve the above object, the present invention provides an e-commerce data cleaning system, which runs in an e-commerce platform, and the e-commerce platform is connected to a terminal through a network, and the system includes: an adjustment module, configured to The running state of the commercial platform is adjusted to stop the data interaction state, so as to ensure that the terminal cannot perform data interaction operation on the e-commerce platform; the calculation module is configured to calculate a hash value of each group of e-commerce data on the e-commerce platform; a module for finding the same hash value from the calculated hash value, and grouping the same hash value into a group, and matching the same hash value in each group with the same hash value in the group Storage of e-commerce data The path is saved in a hash association list; the deletion module is configured to select a storage path in each hash association list, and delete the e-commerce data corresponding to the unselected storage path in the hash association list; a modification module, configured to modify an identifier corresponding to the deleted e-commerce data in each hash association list; the adjustment module is further configured to adjust an operation state on the e-commerce platform to a data interaction state, to ensure the terminal and the The e-commerce platform performs data interaction; the receiving module is configured to receive a query instruction sent by the terminal; and the sending module is configured to: when the storage path of the e-commerce data to be queried in the query instruction has a corresponding hash association list, The e-commerce data corresponding to the selected storage path in the corresponding hash association list is sent to the terminal.

 [0007] In another aspect, the present invention further provides an e-commerce data cleaning method, which is applied to an e-commerce platform, wherein the e-commerce platform is connected to a terminal through a network, and the method includes the following steps: running on an e-commerce platform The state is adjusted to stop the data interaction state to ensure that the terminal cannot perform data interaction operation on the e-commerce platform; calculate a hash value of each group of e-commerce data on the e-commerce platform; and find the same from the calculated hash value a hash value, and group the same hash values into a group, and store the same hash value in each group and the storage path of the e-commerce data corresponding to the same hash value in the group in a hash association list. Selecting a storage path in each hash association list, and deleting the e-commerce data corresponding to the unselected storage path in the hash association list; modifying the deleted e-commerce data in each hash association list Corresponding identification; adjusting the running state on the e-commerce platform to a data interaction state to ensure data interaction between the terminal and the e-commerce platform; Instructions; when the memory's data path electrically query instructions are to be queried association list corresponding to the hash, the hash linked list corresponding to the selected path is stored data corresponding to the electricity supplier to the terminal.

 Advantageous effects of the invention

 Beneficial effect

 [0008] The present invention adopts the above technical solution, and brings the technical effects as follows: The present invention uses the hash value comparison method to find the same e-commerce data, and deletes repeated e-commerce data, thereby accelerating the access speed of the e-commerce platform. , reducing the redundancy of the e-commerce platform.

 Brief description of the drawing

 DRAWINGS

1 is a schematic diagram of an application environment of an e-commerce data cleaning system of the present invention; 2 is a block diagram showing a preferred embodiment of an e-commerce data cleaning system of the present invention;

3 is a flow chart of a preferred embodiment of the e-commerce data cleaning method of the present invention;

4 is a schematic diagram of a hash association list of the present invention.

 BEST MODE FOR CARRYING OUT THE INVENTION

 BEST MODE FOR CARRYING OUT THE INVENTION

The specific embodiments, structures, features and functions of the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

 [0014] Referring to FIG. 1, FIG. 1 is a schematic diagram of an application environment of an e-commerce data cleaning system according to the present invention. The e-commerce data cleaning system 30 in the present invention runs on the e-commerce platform 3. The e-commerce platform 3 is communicatively coupled to a plurality of terminals 1 via a network 2. The e-commerce platform 3 stores various types of e-commerce data (e.g., user name, product text information, merchandise type information, courier information, warehousing information, merchandise pictures, shopping records, etc.). It should be noted that the e-commerce data stored in the e-commerce platform 3 is classified into multiple sets of data according to the classification of the time and place, for example, the data generated every second is classified into a set of data. The e-commerce platform 3 is a cluster composed of a plurality of servers.

 [0015] In this embodiment, the network 2 may be a wired communication network or a wireless communication network. The network is preferably a wireless communication network, including but not limited to a GSM network, a GPRS network, a CDMA network, a TD-S CDMA network, a WiMAX network, a TD-LTE network, an FDD-LTE network, and the like. .

 [0016] The terminal 1 may be, but not limited to, any other suitable portable touch screen electronic device such as a smart phone, a tablet computer, a personal digital assistant (PDA), a personal computer, an electronic signboard, and the like. The terminal 1 is configured to access the e-commerce platform 3 through the network 2, and obtain e-commerce data on the e-commerce platform 3. The terminal 1 is further configured to generate e-commerce data, and send the generated e-commerce data to the e-commerce platform 3 for storage.

 [0017] Referring to FIG. 2, it is a block diagram of a preferred embodiment of the e-commerce data cleaning system of the present invention. In the present embodiment, the e-commerce data cleaning system 30 is applied to the e-commerce platform 3. The e-commerce platform 3 includes, but is not limited to, an e-commerce data cleaning system 30, a storage unit 31, a processing unit 32, and a communication unit 33.

[0018] The storage unit 31 may be a read only memory unit ROM, an electrically erasable storage unit EEPRO M, flash memory unit FLASH or solid hard disk.

 [0019] The processing unit 32 may be a central processing unit (CPU), a microcontroller (MCU), a data processing chip, or an information processing unit having a data processing function.

 [0020] The communication unit 33 is a communication interface with a remote communication function, for example, communication supporting wireless communication technologies such as GSM, GPR S, WCDMA, CDMA, TD-SCDMA, WiMAX, TD-LTE, FDD-LTE. Interface, support wired communication interface of USB and network cable.

[0021] The cleaning system 30's data electrical include, but are not limited to, the adjustment module 311, calculation module 312, a searching module 313, a deleting module _314, modification module 315, a receiving module 316 and a transmitting module 317, the present invention A module referred to refers to a series of computer program instructions that can be executed by the processing unit 32 of the e-commerce platform 3 and that are capable of performing a fixed function, which is stored in the storage unit 31 of the e-commerce platform 3.

 [0022] The adjustment module 311 is configured to adjust the running state of the e-commerce platform 3 to stop the data interaction state, so as to ensure that the terminal 1 cannot perform data interaction operation on the e-commerce platform 3. In this embodiment, the terminal 1 stops the data interaction operation (for example, writing, reading, querying, etc.) on the e-commerce platform 3, and can perform redundancy on the e-commerce data in the e-commerce platform 3. The cleaning is performed to prevent the e-commerce platform 3 from cleaning up the redundancy. Since the terminal 1 writes (or queries or reads) the data to the e-commerce platform 3, it interferes with the cleaning of the redundancy. The adjustment module 31 is configured to monitor a program (or software) related to the terminal 1 running in the e-commerce platform 1, and prevent a program (or software) associated with the terminal 1 from performing data interaction with the e-commerce platform 3. In other embodiments, the adjustment module 311 can also ensure that the terminal 1 cannot perform data interaction operation on the e-commerce platform 3 by suspending the communication connection with the terminal 1 (for example, sending an "e-commerce" to the terminal 1. In the platform 3 redundancy cleanup, please check the "Information" later. In order to prevent the user from accessing the e-commerce platform, the e-commerce platform 3 performs state adjustments at preset times (for example, when the user visit amount is small at 3 am).

 [0023] The calculation module 312 is configured to calculate a hash value of each group of e-commerce data on the e-commerce platform 3. Specifically, the calculation module 312 traverses the e-commerce data on the e-commerce platform 3 and calculates a hash value of each set of e-commerce data through a hash algorithm. The hash algorithm may be, but is not limited to, the SHA256 algorithm.

[0024] The searching module 313 is configured to search for the same hash value from the calculated hash values, and group the same hash values into a group, and the same hash value in each group is the same as the same in the group. The storage path of the e-commerce data corresponding to the hash value is stored in a hash association list. In general, due to the uniqueness of the hash value, if the same hash value is found, it indicates that the same e-commerce data exists in the e-commerce platform 3. In this embodiment, the The hash association list includes a plurality of identical hash values and a storage path of each hash value corresponding to the e-commerce data. It should be noted that the hash values in the hash association list are the same. If there are multiple sets of identical e-commerce data in the e-commerce platform 3, the search module 313 obtains multiple sets of identical hash values and generates a hash association list for each set of identical hash values. As shown in FIG. 4, the hash association list includes at least four fields, where the four fields are a file name (a name indicating an e-commerce data) and a hash value (a hash value indicating an e-commerce data). ), a storage path (representing the storage path of the e-commerce data on the e-commerce platform 3) and a logo (indicating whether the e-commerce data is retained in the e-commerce platform 3, where Y is reserved and N is deleted). Further, for the sake of convenience, the letter B in FIG. 4 merely refers to the hash value, and does not mean that the hash value is B, and the letters T1 to T9 in FIG. 4 are merely referring to the storage path, and do not indicate T1 to T9 are storage paths.

[0025] The deleting module 314 is configured to select one storage path in each hash association list, and delete the e-commerce data corresponding to the unselected storage path in the hash association list. In this embodiment, the deleting module ₃ 14 selects the storage path T1 and deletes the e-commerce data corresponding to the remaining storage paths T2 to T9 in the e-commerce platform 3.

 The modification module 315 is configured to modify the identifier corresponding to the deleted e-commerce data in each hash association list. Specifically, as shown in FIG. 4, the modification module 315 modifies the identifier corresponding to T2 to T9 in the hash association list from Y to N.

[0027] The adjustment module 311 is further configured to adjust an operation state on the e-commerce platform 3 to a data interaction state to ensure that the terminal 1 performs data interaction with the e-commerce platform ₃ . In this embodiment, the adjustment module 311 can stop monitoring the program (or software) related to the terminal 1 in the e-commerce platform 1. In other embodiments, the adjustment module 311 can also ensure that the terminal 1 can perform data interaction operations on the e-commerce platform 3 by re-communicating with the terminal 1 (for example, resending "electricity" to the terminal 1. The business platform 3 redundancy cleaning is completed, welcome to query "information".

The receiving module 316 is configured to receive a query instruction sent by the terminal 1. Generally, the terminal 1 queries the e-commerce data on the e-commerce platform 3, and the query instruction includes a storage path of the e-commerce data to be queried. The sending module 317 is configured to determine the query instruction to be checked. Whether the storage path of the quotient e-commerce data has a corresponding hash association list. The sending module 317 parses the query instruction and obtains the e-commerce data to be queried. a storage path, comparing the storage path of the e-commerce data to be queried with each hash association list in the e-commerce platform 3, if there is a hash association list in the e-commerce platform 3, the e-commerce to be queried The storage path of the data, the hash association list is a hash association list corresponding to the storage path of the e-commerce data to be queried

[0030] The sending module 317 is further configured to send the e-commerce data corresponding to the selected storage path in the corresponding hash association list to the terminal 1. Specifically, as shown in FIG. 4, if the storage path of the e-commerce data to be queried is T2, the sending module 317 sends the e-commerce data 存储 corresponding to the storage path T1 to the terminal 1.

 [0031] Referring to FIG. 3, it is a flow chart of a preferred embodiment of the e-commerce data cleaning method of the present invention. In this embodiment, the e-commerce data cleaning method is applied to an e-commerce platform 3, and the method includes the following steps:

 [0032] Step S11: The adjustment module 311 adjusts the running state of the e-commerce platform 3 to stop the data interaction state, so as to ensure that the terminal 1 cannot perform data interaction operation on the e-commerce platform 3. In this embodiment, the terminal 1 stops performing data interaction operations (for example, writing, reading, querying, etc.) on the e-commerce platform 3, and can perform redundancy on the e-commerce data in the e-commerce platform 3. The cleaning is performed to prevent the e-commerce platform 3 from cleaning up the redundancy. Since the terminal 1 writes (or queries or reads) the data to the e-commerce platform 3, it interferes with the cleaning of the redundancy. The adjustment module 311 can monitor the program (or software) related to the terminal 1 running in the e-commerce platform 1, and prevent the program (or software) associated with the terminal 1 from performing data interaction with the e-commerce platform 3. In other embodiments, the adjustment module 311 can also ensure that the terminal 1 cannot perform data interaction operation on the e-commerce platform 3 by suspending the communication connection with the terminal 1 (for example, sending an "e-commerce" to the terminal 1. In the platform 3 redundancy cleanup, please check the "Information" later.

 [0033] Step S12: The calculation module 312 calculates a hash value of each group of e-commerce data on the e-commerce platform 3. Specifically, the calculation module 312 traverses the e-commerce data on the e-commerce platform 3, and calculates a hash value of each set of e-commerce data through a hash algorithm. The hash algorithm may be, but is not limited to, the SHA256 algorithm.

[0034] Step S13: The searching module 313 searches for the same hash value from the calculated hash values, and groups the same hash values into a group, and sets the same hash value in each group and the group. The storage path of the e-commerce data corresponding to the same hash value is stored in a hash association list. In general, due to the uniqueness of the hash value, if the same hash value is found, it indicates that the same e-commerce data exists in the e-commerce platform 3. In this embodiment, the hash association list includes a plurality of identical hash values and a storage path of each hash value corresponding to the e-commerce data. It should be noted that the hash values in the hash association list are the same. The electric commerce If there are multiple sets of identical e-commerce data in station 3, then the lookup module 313 obtains multiple sets of identical hash values and generates a hash association list for each set of identical hash values. As shown in FIG. 4, the hash association list includes at least four fields, where the four fields are a file name (a name indicating an e-commerce data) and a hash value (a hash value indicating an e-commerce data). ), a storage path (representing the storage path of the e-commerce data on the e-commerce platform 3) and a logo (indicating whether the e-commerce data is retained in the e-commerce platform 3, where Y is reserved and N is deleted). Further, for the sake of convenience, the letter B in FIG. 4 merely refers to the hash value, and does not mean that the hash value is B, and the letters T1 to T9 in FIG. 4 are merely referring to the storage path, and do not indicate T1 to T9 are storage paths.

 [0035] Step S14: The deleting module 314 selects one storage path in each hash association list, and deletes the e-commerce data corresponding to the unselected storage paths in the hash association list. In this embodiment, the deletion module 314 selects the storage path T1, and deletes the e-commerce data corresponding to the remaining storage paths Τ2 to Τ9 in the e-commerce platform 3.

 [0036] Step S15: The modification module 315 modifies the identifier corresponding to the deleted e-commerce data in each hash association list. Specifically, as shown in FIG. 4, the modification module 315 modifies the identifier corresponding to Τ2 to Τ9 in the hash association list from Υ to Ν.

 [0037] Step S16: The adjustment module 311 adjusts the running state of the e-commerce platform 3 to a data interaction state to ensure that the terminal 1 performs data interaction with the e-commerce platform 3. In this embodiment, the adjustment module 311 can stop monitoring the program (or software) related to the terminal 1 in the e-commerce platform 1. In other embodiments, the adjustment module 311 can also ensure that the terminal 1 can perform data interaction operations on the e-commerce platform 3 by re-communicating with the terminal 1 (for example, resending "electricity" to the terminal 1. The business platform 3 redundancy cleaning is completed, welcome to query "information".

 [0038] Step S17: The receiving module 316 receives the query instruction sent by the terminal 1. Generally, the terminal 1 queries the e-commerce data on the e-commerce platform 3, and the query instruction includes a storage path of the e-commerce data to be queried.

[0039] Step S18: If the storage path in the query instruction has a corresponding hash association list, the sending module 317 sends the e-commerce data corresponding to the selected storage path in the corresponding hash association list to the terminal 1 . Specifically, the sending module 317 first determines whether the storage path of the e-commerce data to be queried in the query instruction has a corresponding hash association list. The sending module 317 parses the query instruction and obtains the number of e-commerce to be queried According to the storage path, the storage path of the e-commerce data to be queried is compared with each hash association list in the e-commerce platform 3, if there is a hash association list in the e-commerce platform 3 containing the to-be-queried The storage path of the e-commerce data, the hash association list is a hash association list corresponding to the storage path of the e-commerce data to be queried. If the storage path in the query instruction does not have a corresponding hash association list, the e-commerce data in the e-commerce platform 3 is directly traversed by the storage path in the query instruction (omitted in FIG. 3), and the process ends. As shown in FIG. 4, if the storage path of the e-commerce data to be queried is T2, the sending module 317 sends the e-commerce data 存储 corresponding to the storage path Τ 1 to the terminal 1.

Industrial applicability

 The invention uses the hash value comparison method to find the same e-commerce data, and deletes the repeated e-commerce data, accelerates the access speed of the e-commerce platform, and reduces the redundancy of the e-commerce platform.

Claims

Claim  [Claim 1] An e-commerce data cleaning system, which runs in an e-commerce platform, wherein the e-commerce platform is communicatively connected to a terminal through a network, and the system includes:  An adjustment module, configured to adjust a running state on the e-commerce platform to stop the data interaction state, to ensure that the terminal cannot perform data interaction operation on the e-commerce platform; and a calculation module, configured to calculate each group of e-commerce on the e-commerce platform a hash value of the data; a lookup module for finding the same hash value from the calculated hash value, and grouping the same hash value into a group, the same hash value in each group and the group The storage path of the e-commerce data corresponding to the same hash value is stored in a hash association list; the deletion module is configured to select a storage path in each hash association list, and delete the hash association list. E-commerce data corresponding to the selected storage path;  a modifying module, configured to modify, in each hash association list, an identifier corresponding to the deleted e-commerce data;  The adjusting module is further configured to adjust an operating state on the e-commerce platform to a data interaction state to ensure data interaction between the terminal and the e-commerce platform;  a receiving module, configured to receive a query instruction sent by the terminal, and a sending module, configured to: when the storage path of the e-commerce data to be queried in the query instruction has a corresponding hash association list, associate the corresponding hash The e-commerce data corresponding to the selected storage path in the list is sent to the terminal.  [Claim 2] The e-commerce data cleaning system according to claim 1, wherein the hash association list includes four fields, and the four fields are respectively a file name, a hash value, and a storage. Path and logo.  [Claim 3] The e-commerce data cleaning system according to claim 1, wherein the calculating module calculates a hash value of each group of e-commerce data on the e-commerce platform as follows: traversing the e-commerce platform The e-commerce data on the above, and the hash value of each group of e-commerce data is calculated by a hash algorithm. [Claim 4] The e-commerce data cleaning system according to claim 1, wherein the adjusting module adjusts the running state on the e-commerce platform to stop the data interaction state as follows: monitoring the operation in the e-commerce platform Terminal-related programs or software, and blocks programs related to the terminal Or the software performs data interaction with the e-commerce platform.  [Claim 5] The e-commerce data clearing system according to claim 1, wherein the sending module determines whether a storage path of the e-commerce data to be queried in the query instruction has a corresponding hash association list. The method is as follows: parse the query instruction and obtain a storage path of the e-commerce data to be queried, and compare the storage path of the e-commerce data to be queried with each hash association list in the e-commerce platform, if the e-commerce A hash association list in the platform includes a storage path of the e-commerce data to be queried, and the hash association list is a hash association list corresponding to the storage path of the e-commerce data to be queried.  [Claim 6] A method for cleaning an e-commerce data, which is applied to an e-commerce platform, wherein the e-commerce platform is connected to a terminal through a network, and the method comprises the following steps:  Adjusting the running state of the e-commerce platform to stop the data interaction state, so as to ensure that the terminal cannot perform data interaction operation on the e-commerce platform;  Calculating a hash value of each group of e-commerce data on the e-commerce platform;  Find the same hash value from the calculated hash value, and group the same hash value into a group, and compare the same hash value in each group with the same hash value in the group. The storage path is saved in a hash association list;  Selecting a storage path in each hash association list, and deleting the e-commerce data corresponding to the unselected storage paths in the hash association list;  Modifying, in each hash association list, an identifier corresponding to the deleted e-commerce data; adjusting an operation state on the e-commerce platform to a data interaction state to ensure data interaction between the terminal and the e-commerce platform;  Receiving a query command sent by the terminal; and  When the storage path of the e-commerce data to be queried in the query instruction has a corresponding hash association list, the e-commerce data corresponding to the selected storage path in the corresponding hash association list is sent to the final mountain. [Claim 7] The method for cleaning an e-commerce data according to claim 6, wherein the hash association list includes four fields, and the four fields are a file name, a hash value, and a storage. Path and logo. [Claim 8] The e-commerce data clearing method according to claim 6, wherein the calculating the hash value of each group of e-commerce data on the e-commerce platform comprises the following steps: traversing the e-commerce platform The e-commerce data on the above, and the hash value of each group of e-commerce data is calculated by a hash algorithm.  [Claim 9] The method for cleaning an e-commerce data according to claim 6, wherein the step of adjusting an operating state on the e-commerce platform to stop the data interaction state comprises the following steps: monitoring operation in the e-commerce platform Programs or software related to the terminal, and prevent the terminal-related programs or software from performing data interaction with the e-commerce platform.  [Claim 10] The e-commerce data clearing method according to claim 6, wherein the step of determining whether the storage path of the e-commerce data to be queried in the query instruction has a corresponding hash association list includes The following steps are performed: parsing the query instruction and obtaining a storage path of the e-commerce data to be queried, comparing the storage path of the e-commerce data to be queried with each hash association list in the e-commerce platform, if the e-commerce A hash association list in the platform includes a storage path of the e-commerce data to be queried, and the hash association list is a hash association list corresponding to the storage path of the e-commerce data to be queried.