CN111984898A - Tag push method, device, electronic device and storage medium based on big data - Google Patents

Abstract

The invention relates to the technical field of big data, and proposes a method, device, electronic device and storage medium for label pushing based on big data, including: collecting raw data from multiple preset data sources and cleaning according to a preset data cleaning strategy , obtain sample data; extract multi-dimensional target features from the sample data of each node and classify according to the preset classification model to obtain the initial label of each node; perform cluster analysis on the initial label of each node to form Label system of different objects; when the target node is detected to be triggered, the initial label in the label system of the target node is pushed. The invention obtains the initial label of each node by cleaning the original data and extracting multi-dimensional target features, and clusters them into a label system of different objects, thereby improving the accuracy of label recommendation. In addition, the present invention also relates to the technical field of blockchain, and the initial label is stored in the blockchain node.

Description

Tag push method, device, electronic device and storage medium based on big data

technical field

The present invention relates to the technical field of big data, in particular to a method, device, electronic device and storage medium for pushing labels based on big data.

Background technique

The traditional labeling technology is to extract based on the data source, obtain the user's behavior data by means such as burying points, and put various labels according to the user's behavior habits and basic information, and the traditional labels are derived from the business or product. From a perspective, relying on experience to carry out related dimension combinations and threshold settings, most labels are uninterested.

The existing tag library of the execution business system is based on the execution of the business process. The tags in the tag library contain a large number of manual operations, and most of the tags are derived from the simple collection and sorting of data, and there is no need for each business node. The data is cleaned and processed, and valuable information is extracted to create a tag library, which makes it impossible for users to quickly obtain the desired information and data according to the recommended tags. Before determining each step, a large amount of information and data needs to be checked. rate is low.

SUMMARY OF THE INVENTION

In view of the above, it is necessary to propose a big data-based label push method, device, electronic device and storage medium. By cleaning the original data and extracting multi-dimensional target features, the initial label of each node is obtained, and the labels of different objects are clustered. The label system improves the accuracy of label recommendation.

A first aspect of the present invention provides a method for pushing labels based on big data, and the method for pushing labels based on big data includes:

Collect a plurality of raw data from a plurality of preset data sources, wherein each of the raw data corresponds to a node identifier;

Perform data cleaning on each of the raw data according to a preset data cleaning strategy to obtain sample data;

Extract multi-dimensional target features from the sample data corresponding to each node, classify the multi-dimensional target features according to a preset classification model, and obtain the initial label of each node;

Perform cluster analysis on the initial label of each node to form a label system of different objects;

When it is detected that the target node among the multiple nodes is triggered, the initial label in the label system corresponding to the target node is pushed.

Preferably, the cluster analysis of the initial labels of each node to form a label system for different objects includes:

According to the k-means clustering algorithm, the initial label of each node is clustered to obtain a plurality of objects;

Taking any one of the multiple objects as the target object, the target object and the initial label corresponding to the target object are set as the label system corresponding to the target object.

Preferably, after performing cluster analysis on the initial label of each node to form a label system of different objects, the method further includes:

Monitor the click-through rate and conversion rate of each initial tag in a preset period in real time;

Judging whether the click-through rate of each initial label is greater than the corresponding click-through rate threshold, and judging whether the conversion rate of each initial label is greater than the corresponding conversion rate threshold;

When the click-through rate of each initial tag is greater than or equal to the corresponding click-through rate threshold, and the conversion rate of each initial tag is greater than or equal to the corresponding conversion rate threshold, the initial tags are divided into popular tags;

When the click rate of each initial tag is smaller than the corresponding click rate threshold, or the conversion rate of each initial tag is smaller than the corresponding conversion rate threshold, the initial tag is classified as a useless tag.

Preferably, performing data cleaning on the original data according to a preset data cleaning strategy, and obtaining sample data includes:

Identify the node ID of each raw data;

obtaining a preset data cleaning strategy corresponding to the node identifier;

Clean the original data corresponding to the node identifier according to the preset data cleaning strategy;

Converting the cleaned raw data into structured data of a preset type;

Classifying the structured data according to the node identifiers to obtain sample data, and storing the sample data in a preset database.

Preferably, the multi-dimensional target features are extracted from the sample data corresponding to each node, and the multi-dimensional target features are classified according to a preset classification model, and the initial label obtained for each node includes:

Read the sample data of each node from the preset database according to the node identification of each node and the query language HQL grammar rule oriented to the node identification;

Extract multi-dimensional target features from the read sample data of each node according to a preset algorithm;

The multi-dimensional target features are input into the preset classification model for classification to obtain the initial label of each node, wherein the initial label is stored in the blockchain node.

Preferably, the extraction of multi-dimensional target features from the read sample data of each node according to a preset algorithm includes:

Extract the first feature from the read sample data of each node according to the preset feature dimension;

Process the read sample data of each node through the trained model to obtain the second feature;

The first feature and the second feature are combined to obtain a multi-dimensional target feature.

Preferably, after the initial label in the label system corresponding to the target node is pushed, the method further includes:

When monitoring the user's reprocessing instruction for the pushed initial label, parse the reprocessing instruction to obtain the user's reprocessing condition;

Inputting the reprocessing conditions into the preset classification model to obtain a new label, and combining the new label and the pushed initial label to obtain an advanced label;

Push the advanced tag.

A second aspect of the present invention provides a big data-based label pushing device, the big data-based label pushing device comprising:

a collection module, configured to collect a plurality of raw data from a plurality of preset data sources, wherein each of the raw data corresponds to a node identifier;

a cleaning module, configured to perform data cleaning on each of the raw data according to a preset data cleaning strategy to obtain sample data;

A classification module, used for extracting multi-dimensional target features from the sample data corresponding to each node, classifying the multi-dimensional target features according to a preset classification model, and obtaining the initial label of each node;

an analysis module for performing cluster analysis on the initial label of each node to form a label system of different objects;

The push module is configured to push the initial label in the label system corresponding to the target node when it is detected that the target node is triggered.

A third aspect of the present invention provides an electronic device, the electronic device includes a processor, and the processor is configured to implement the big data-based tag push method when executing a computer program stored in a memory.

A fourth aspect of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for pushing tags based on big data is implemented.

To sum up, the method, device, terminal and storage medium for label pushing based on big data according to the present invention clean the original data collected from different data sources through a preset data cleaning strategy on the one hand. Clean the sample data of each node, delete the problem data, ensure the consistency and integrity of the obtained sample data, and improve the quality of the sample data. On the other hand, by extracting the multi-dimensional objects in the sample data of each node feature, the multi-dimensional target features are input into the preset classification model for classification to obtain the initial label of each node, which improves the efficiency of calculating the initial label, and at the same time, the initial label of each node is clustered into different objects The labeling system improves the accuracy of label recommendation.

In addition, by monitoring the click rate and conversion rate of each initial tag in real time within a preset period, deleting useless tags, and continuously optimizing the entire tag system through training and learning, ensuring the timeliness of the initial tags in the tag system and improving the recommended tags. 's accuracy.

Description of drawings

FIG. 1 is a flowchart of a method for pushing tags based on big data according to Embodiment 1 of the present invention.

FIG. 2 is a structural diagram of a big data-based label push device provided in Embodiment 2 of the present invention.

FIG. 3 is a schematic structural diagram of an electronic device according to Embodiment 3 of the present invention.

Detailed ways

In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments may be combined with each other under the condition of no conflict.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.

Example 1

FIG. 1 is a flowchart of a method for pushing tags based on big data according to Embodiment 1 of the present invention.

In this embodiment, the method for pushing labels based on big data can be applied to electronic devices. For electronic devices that need to push labels based on big data, the method based on the method of the present invention can be directly integrated on the electronic device. The function of tag push of big data, or running in electronic equipment in the form of Software Development Kit (SKD).

As shown in FIG. 1 , the method for pushing labels based on big data specifically includes the following steps. According to different requirements, the order of the steps in this flowchart can be changed, and some can be omitted.

S11: Collect a plurality of raw data from a plurality of preset data sources, wherein each of the raw data corresponds to a node identifier.

In this embodiment, the original data includes: basic information of the subject to be executed, basic information of the case, information of the subject of execution, property information, etc., wherein the person to be executed refers to Lao Lai, and the information of the person to be executed mainly includes: name , ID number, age, gender, occupation and unit, etc.; execution subject information mainly includes: case number, identity information of the person subject to execution, user information, links involved, return status, operation time, etc.; property information refers to the name of the person subject to execution All property under it, such as bank deposits, real estate, vehicles, etc. Take real estate as an example: the province and city where the real estate is located, floor, orientation, area, etc. The data source may be an execution business system, and raw data is collected from each process node of the execution business system.

S12: Perform data cleaning on each of the original data according to a preset data cleaning strategy to obtain sample data.

In this embodiment, a data cleaning strategy may be set in advance according to the cleaning conditions of the labels corresponding to each node, and the preset data cleaning strategy may be cleaning of missing values, format content cleaning, logical error cleaning, and non-required data cleaning, After the raw data is collected, the raw data is cleaned according to the preset data cleaning strategy to obtain sample data.

In this embodiment, the preset data cleaning strategy corresponding to the missing value cleaning is to directly delete data records with missing values or to complete data records with missing values.

Exemplarily, the target labels of data records with missing values are mainly concentrated in one or several categories. If these data records are deleted, a large amount of feature information will be lost in the corresponding classified data samples, resulting in overfitting of the model or inaccurate classification. , and the preset data cleaning strategy used is to complete data records with missing values.

In this embodiment, the preset data cleaning strategy corresponding to the format content cleaning is that the display format of time, date, numerical value, full-width, etc. is inconsistent, the content contains characters that should not exist, and the content and the field should have content. Inconsistent data is cleaned.

Exemplarily, when the display formats such as time, date, numerical value, full-width, etc. are inconsistent, the preset data cleaning strategy is to process the display formats such as time, date, numerical value, and full-width into a consistent format; When there are characters that should not exist, the preset data cleaning strategy is to use semi-automatic verification and semi-manual methods to find out possible problems and remove unnecessary characters, such as Chinese characters in ID numbers.

In this embodiment, the preset data cleaning strategy corresponding to the logic error cleaning is deduplication, removal of unreasonable values, and correction of contradictory content.

Exemplarily, the preset data cleaning strategy set for deduplication is to delete duplicate fields and keep only one; the preset data cleaning strategy set for removing unreasonable values, such as age 200, delete the unreasonable value corresponding to the age. Reasonable value; the preset data cleaning strategy set for correcting conflicting content is to determine which field provides more reliable information based on the data source of the field, and remove or reconstruct unreliable fields. For example, the ID number is 1101031980XXXXXXXX, Then fill in the age of 18, and you need to judge the ID number and age which are more reliable to reconstruct or delete contradictory content.

In this embodiment, the preset data cleaning policy corresponding to the non-required data cleaning refers to deleting unnecessary fields.

Preferably, performing data cleaning on the original data according to a preset data cleaning strategy, and obtaining sample data includes:

Identify the node ID of each raw data;

obtaining a preset data cleaning strategy corresponding to the node identifier;

Clean the original data corresponding to the node identifier according to the preset data cleaning strategy;

Converting the cleaned raw data into structured data of a preset type;

Classifying the structured data according to the node identifiers to obtain sample data, and storing the sample data in a preset database.

In this embodiment, the preset database may be the hive database, which is a data warehouse tool based on Hadoop, which can store structured data, provide a complete SQL query function, and can convert SQL statements into MapReduce tasks for running , the original data is cleaned and converted into structured data of a preset type through a preset data cleaning strategy, the structured data is classified according to the node identifier to obtain sample data, and the sample Data is stored in a preset database.

Further, the method also includes:

placing the problem data that does not conform to the preset data cleaning strategy in the original data in the problem database;

When the cleaning instruction is not received within the preset time period, the processing of the problem data is ended;

Also delete the problem data.

In this embodiment, in the process of cleaning data through the preset data cleaning strategy, if problem data occurs, the problem data may be stored in the problem database, and if no cleaning instruction is received within the preset time period, determine The problem data can be deleted.

In this embodiment, the original data is cleaned by the preset data cleaning strategy, and the problem data is deleted, so as to ensure the consistency and integrity of the obtained sample data, and improve the quality of the sample data.

S13: Extract multi-dimensional target features from the sample data corresponding to each node, classify the multi-dimensional target features according to a preset classification model, and obtain an initial label of each node.

In this embodiment, since different nodes correspond to different data and different nodes correspond to different sample data, multi-dimensional target features are extracted from the sample data corresponding to each node, and trained in the pre-trained classification model The multi-dimensional target feature obtains the initial label of each node, and organizes the initial labels of multiple nodes to form a label library.

Preferably, the multi-dimensional target features are extracted from the sample data corresponding to each node, and the multi-dimensional target features are classified according to a preset classification model, and the initial label obtained for each node includes:

Read the sample data of each node from the preset database according to the node identification of each node and the query language HQL grammar rule oriented to the node identification;

Extract multi-dimensional target features from the read sample data of each node according to a preset algorithm;

The multi-dimensional target features are input into the preset classification model for classification to obtain the initial label of each node, wherein the initial label is stored in the blockchain node.

In this embodiment, the sample data of different nodes is different, the sample data of the corresponding node is read from the preset database using the query language HQL syntax rules, and the sample data of each node is read from the preset algorithm by using the preset algorithm. Multi-dimensional target features are extracted from the algorithm, the preset algorithm is the prior art, and the present invention will not describe in detail here.

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned initial label, the above-mentioned initial label can also be stored in a node of a blockchain.

In this embodiment, the initial label of each node is obtained by inputting the multi-dimensional target feature into a preset classification model for classification, which improves the efficiency of obtaining the initial label by calculation.

Further, the extraction of multi-dimensional target features from the read sample data of each node according to a preset algorithm includes:

Extract the first feature from the read sample data of each node according to the preset feature dimension;

Process the read sample data of each node through the trained model to obtain the second feature;

The first feature and the second feature are combined to obtain a multi-dimensional target feature.

In this implementation, the multi-dimensional target features include basic features and behavioral features, where the basic features are descriptions of natural attributes of the executed object, for example, the gender and age of the executed object; the behavioral feature is the executed object behaviors, such as no property, no house, no car, etc.

In this embodiment, different from the traditional labeling system, the labeling system analyzes the raw data of each node in the business process, cleans the raw data to obtain the sample data of each node, and extracts each node's sample data. The multi-dimensional target feature in the node's sample data obtains the initial label of each node, which ensures the accuracy of the initial label.

S14: Perform cluster analysis on the initial labels of each node to form label systems of different objects.

In this embodiment, the objects can be classified according to the subject dimension to which the initial label belongs, for example, it can be classified into a case label, an enforcee label, a property information label, etc.; the objects can also be classified according to the strategy model of the application of the initial label To points, for example, can be divided into property control model tags and so on.

Preferably, the cluster analysis of the initial labels of each node to form a label system for different objects includes:

According to the k-means clustering algorithm, the initial label of each node is clustered to obtain a plurality of objects;

Taking any one of the multiple objects as the target object, the target object and the initial label corresponding to the target object are set as the label system corresponding to the target object.

In this embodiment, the k-means clustering algorithm is an iterative clustering analysis algorithm. The steps are to randomly select K objects as the initial cluster centers, and then calculate the difference between each object and each seed cluster center. The distance between each object is assigned to the cluster center closest to it; the cluster center and the objects assigned to them represent a cluster; each time a sample is assigned, the cluster center of the cluster will be Some objects are recalculated; this process is repeated until a preset termination condition is met, where the preset termination condition can be that no objects are reassigned to different clusters, no cluster centers change again, and the sum of squared errors is local. minimum.

In this embodiment, by using the k-means clustering algorithm to cluster the initial labels to obtain multiple objects, the accuracy of obtaining label systems for different objects is improved.

Further, after performing cluster analysis on the initial label of each node to form a label system of different objects, the method further includes:

Monitor the click-through rate and conversion rate of each initial tag in a preset period in real time;

Judging whether the click-through rate of each initial label is greater than the corresponding click-through rate threshold, and judging whether the conversion rate of each initial label is greater than the corresponding conversion rate threshold;

When the click-through rate of each initial tag is greater than or equal to the corresponding click-through rate threshold, and the conversion rate of each initial tag is greater than or equal to the corresponding conversion rate threshold, the initial tags are divided into popular tags;

When the click rate of each initial tag is smaller than the corresponding click rate threshold, or the conversion rate of each initial tag is smaller than the corresponding conversion rate threshold, the initial tag is classified as a useless tag.

Further, the method also includes:

When the initial label is a popular label, retain the initial label;

When the initial label is a useless label, the initial label is deleted.

In this embodiment, the conversion rate refers to the rate at which initial tags are converted into advanced tags.

In this implementation, the click-through rate and conversion rate of each initial tag are monitored in real time within a preset period, useless tags are deleted, and the entire tag system is continuously optimized through training and learning to ensure the timeliness of the initial tags in the tag system, while improving the accuracy of the recommended labels.

S15: When it is detected that the target node is triggered, push the initial label in the label system corresponding to the target node.

Exemplarily, when a node that triggers property control is detected, the property control model label in the label system corresponding to the property control model is pushed to assist the user in making decisions quickly and determine the priority of property control, such as freezing first. The bank deposit of the person subject to execution or the local real estate of the person subject to execution should be seized first, which improves the efficiency of the user's case handling.

In this embodiment, the labels are directly given to different application models, thereby reducing the model strategy operation process.

Further, after the initial label in the label system corresponding to the target node is pushed, the method further includes:

When monitoring the user's reprocessing instruction for the pushed initial label, parse the reprocessing instruction to obtain the user's reprocessing condition;

Inputting the reprocessing conditions into the preset classification model to obtain a new label, and combining the new label and the pushed initial label to obtain an advanced label;

Push the advanced tag.

In this implementation, by parsing the reprocessing instruction fed back by the user, inputting the reprocessing condition into the preset classification model to obtain a new tag, and combining the new tag with the pushed initial tag The calculation obtains advanced tags, which responds to users in a timely manner, improves the timeliness of recommended tags, and improves the efficiency of case handling.

To sum up, the method for pushing labels based on big data described in this embodiment collects raw data from multiple preset data sources, wherein the raw data corresponds to a node identifier; The original data is cleaned according to the preset data cleaning strategy, and the sample data of multiple nodes is obtained; the multi-dimensional target features are extracted from the sample data of each node, and the multi-dimensional target features are classified according to the preset The model is classified to obtain the initial label of each node; the initial label of each node is clustered to form a label system of different objects; when the target node in the multiple nodes is monitored to be triggered, push The initial label in the label system corresponding to the target node.

In the method for pushing labels based on big data described in this embodiment, on the one hand, the original data collected from different data sources is cleaned through a preset data cleaning strategy, and the sample data of each node is obtained by cleaning the original data. , delete the problem data, ensure the consistency and integrity of the obtained sample data, and improve the quality of the sample data. On the other hand, by extracting the multi-dimensional target features in the sample data of each node, the multi-dimensional target The feature is input into the preset classification model to classify the initial label of each node, which improves the efficiency of calculating the initial label. At the same time, the initial label of each node is clustered into a label system of different objects, which improves the recommendation of labels. 's accuracy.

In addition, by monitoring the click rate and conversion rate of each initial tag in real time within a preset period, deleting useless tags, and continuously optimizing the entire tag system through training and learning, ensuring the timeliness of the initial tags in the tag system and improving the recommended tags. 's accuracy.

Embodiment 2

FIG. 2 is a structural diagram of a big data-based label push device provided in Embodiment 2 of the present invention.

In some embodiments, the big data-based tag pushing apparatus 20 may include a plurality of functional modules composed of program code segments. The program codes of each program segment in the big data-based label pushing apparatus 20 may be stored in the memory of the electronic device and executed by the at least one processor to execute (details described in FIG. 1 ) based on big data 's tag push.

In this embodiment, the big data-based label pushing apparatus 20 may be divided into multiple functional modules according to the functions performed by the big data-based label pushing apparatus 20 . The functional modules may include: a collection module 201 , a cleaning module 202 , a classification module 203 , an analysis module 204 , a monitoring module 205 , a judgment module 206 and a push module 207 . The modules referred to in the present invention refer to a series of computer program segments that can be executed by at least one processor and can perform fixed functions, and are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

Collection module 201: used to collect multiple raw data from multiple preset data sources, wherein each of the raw data corresponds to a node identifier.

In this embodiment, the original data includes: basic information of the subject to be executed, basic information of the case, information of the subject of execution, property information, etc., wherein the person to be executed refers to Lao Lai, and the information of the person to be executed mainly includes: name , ID number, age, gender, occupation and unit, etc.; execution subject information mainly includes: case number, identity information of the person subject to execution, user information, links involved, return status, operation time, etc.; property information refers to the name of the person subject to execution All property under it, such as bank deposits, real estate, vehicles, etc. Take real estate as an example: the province and city where the real estate is located, floor, orientation, area, etc. The data source may be an execution business system, and raw data is collected from each process node of the execution business system.

Cleaning module 202 : for performing data cleaning on each of the raw data according to a preset data cleaning strategy to obtain sample data.

In this embodiment, a data cleaning strategy may be set in advance according to the cleaning conditions of the labels corresponding to each node, and the preset data cleaning strategy may be cleaning of missing values, format content cleaning, logical error cleaning, and non-required data cleaning, After the raw data is collected, the raw data is cleaned according to the preset data cleaning strategy to obtain sample data.

In this embodiment, the preset data cleaning strategy corresponding to the missing value cleaning is to directly delete data records with missing values or to complete data records with missing values.

Exemplarily, the target labels of data records with missing values are mainly concentrated in one or several categories. If these data records are deleted, a large amount of feature information will be lost in the corresponding classified data samples, resulting in overfitting of the model or inaccurate classification. , and the preset data cleaning strategy used is to complete data records with missing values.

In this embodiment, the preset data cleaning strategy corresponding to the format content cleaning is that the display format of time, date, numerical value, full-width, etc. is inconsistent, the content contains characters that should not exist, and the content and the field should have content. Inconsistent data is cleaned.

Exemplarily, when the display formats such as time, date, numerical value, full-width, etc. are inconsistent, the preset data cleaning strategy is to process the display formats such as time, date, numerical value, and full-width into a consistent format; When there are characters that should not exist, the preset data cleaning strategy is to use semi-automatic verification and semi-manual methods to find out possible problems and remove unnecessary characters, such as Chinese characters in ID numbers.

In this embodiment, the preset data cleaning strategy corresponding to the logic error cleaning is deduplication, removal of unreasonable values, and correction of contradictory content.

Exemplarily, the preset data cleaning strategy set for deduplication is to delete duplicate fields and keep only one; the preset data cleaning strategy set for removing unreasonable values, such as age 200, delete the unreasonable value corresponding to the age. Reasonable value; the preset data cleaning strategy set for correcting conflicting content is to determine which field provides more reliable information based on the data source of the field, and remove or reconstruct unreliable fields. For example, the ID number is 1101031980XXXXXXXX, Then fill in the age of 18, and you need to judge the ID number and age which are more reliable to reconstruct or delete contradictory content.

In this embodiment, the preset data cleaning policy corresponding to the non-required data cleaning refers to deleting unnecessary fields.

Preferably, the cleaning module 202 performs data cleaning on the raw data according to a preset data cleaning strategy, and the obtained sample data includes:

Identify the node ID of each raw data;

obtaining a preset data cleaning strategy corresponding to the node identifier;

Clean the original data corresponding to the node identifier according to the preset data cleaning strategy;

Converting the cleaned raw data into structured data of a preset type;

Classifying the structured data according to the node identifiers to obtain sample data, and storing the sample data in a preset database.

In this embodiment, the preset database may be the hive database, which is a data warehouse tool based on Hadoop, which can store structured data, provide a complete SQL query function, and can convert SQL statements into MapReduce tasks for running , the original data is cleaned and converted into structured data of a preset type through a preset data cleaning strategy, the structured data is classified according to the node identifier to obtain sample data, and the sample Data is stored in a preset database.

Further, in the process of data cleaning, the problem data in the original data that does not conform to the preset data cleaning strategy is placed in the problem database; when the re-cleaning instruction is not received within the preset time period, the process ends. Processing of the problem data; deleting the problem data at the same time.

In this embodiment, in the process of cleaning data through the preset data cleaning strategy, if problem data occurs, the problem data may be stored in the problem database, and if no cleaning instruction is received within the preset time period, determine The problem data can be deleted.

In this embodiment, the original data is cleaned by the preset data cleaning strategy, and the problem data is deleted, so as to ensure the consistency and integrity of the obtained sample data, and improve the quality of the sample data.

Classification module 203: used to extract multi-dimensional target features from the sample data corresponding to each node, classify the multi-dimensional target features according to a preset classification model, and obtain an initial label of each node.

In this embodiment, since different nodes correspond to different data and different nodes correspond to different sample data, multi-dimensional target features are extracted from the sample data corresponding to each node, and trained in the pre-trained classification model The multi-dimensional target feature obtains the initial label of each node, and organizes the initial labels of multiple nodes to form a label library.

Preferably, the classification module 203 extracts multi-dimensional target features from the sample data corresponding to each node, classifies the multi-dimensional target features according to a preset classification model, and obtains the initial value of each node. Labels include:

Read the sample data of each node from the preset database according to the node identification of each node and the query language HQL grammar rule oriented to the node identification;

Extract multi-dimensional target features from the read sample data of each node according to a preset algorithm;

The multi-dimensional target features are input into the preset classification model for classification to obtain the initial label of each node, wherein the initial label is stored in the blockchain node.

In this embodiment, the sample data of different nodes is different, the sample data of the corresponding node is read from the preset database using the query language HQL syntax rules, and the sample data of each node is read from the preset algorithm by using the preset algorithm. Multi-dimensional target features are extracted from the algorithm, the preset algorithm is the prior art, and the present invention will not describe in detail here.

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned initial label, the above-mentioned initial label can also be stored in a node of a blockchain.

In this embodiment, the initial label of each node is obtained by inputting the multi-dimensional target feature into a preset classification model for classification, which improves the efficiency of obtaining the initial label by calculation.

Further, the extraction of multi-dimensional target features from the read sample data of each node according to a preset algorithm includes:

Extract the first feature from the read sample data of each node according to the preset feature dimension;

Process the read sample data of each node through the trained model to obtain the second feature;

The first feature and the second feature are combined to obtain a multi-dimensional target feature.

In this implementation, the multi-dimensional target features include basic features and behavioral features, where the basic features are descriptions of natural attributes of the executed object, for example, the gender and age of the executed object; the behavioral feature is the executed object behaviors, such as no property, no house, no car, etc.

In this embodiment, different from the traditional labeling system, the labeling system analyzes the raw data of each node in the business process, cleans the raw data to obtain the sample data of each node, and extracts each node's sample data. The multi-dimensional target feature in the node's sample data obtains the initial label of each node, which ensures the accuracy of the initial label.

Analysis module 204: used to perform cluster analysis on the initial label of each node to form a label system of different objects.

In this embodiment, the objects can be classified according to the subject dimension to which the initial label belongs, for example, it can be classified into a case label, an enforcee label, a property information label, etc.; the objects can also be classified according to the strategy model of the application of the initial label To points, for example, can be divided into property control model tags and so on.

Preferably, the analysis module 204 performs cluster analysis on the initial label of each node to form a label system of different objects, including:

According to the k-means clustering algorithm, the initial label of each node is clustered to obtain a plurality of objects;

Taking any one of the multiple objects as the target object, the target object and the initial label corresponding to the target object are set as the label system corresponding to the target object.

In this embodiment, the k-means clustering algorithm is an iterative clustering analysis algorithm. The steps are to randomly select K objects as the initial cluster centers, and then calculate the difference between each object and each seed cluster center. The distance between each object is assigned to the cluster center closest to it; the cluster center and the objects assigned to them represent a cluster; each time a sample is assigned, the cluster center of the cluster will be Some objects are recalculated; this process is repeated until a preset termination condition is met, where the preset termination condition can be that no objects are reassigned to different clusters, no cluster centers change again, and the sum of squared errors is local. minimum.

In this embodiment, by using the k-means clustering algorithm to cluster the initial labels to obtain multiple objects, the accuracy of obtaining label systems for different objects is improved.

Further, after the analysis module 204 performs cluster analysis on the initial labels of each node to form a label system of different objects, the monitoring module 205 is used to monitor the click-through rate and the click rate of each initial label in a preset period in real time. Conversion rate.

Judgment module 206: for judging whether the click rate of each initial tag is greater than the corresponding click rate threshold, and judging whether the conversion rate of each initial tag is greater than the corresponding conversion rate threshold.

In this embodiment, when the click rate of each initial tag is greater than or equal to the corresponding click rate threshold, and the conversion rate of each initial tag is greater than or equal to the corresponding conversion rate threshold, the The initial tags are divided into popular tags.

In this embodiment, when the click rate of each initial tag is smaller than the corresponding click rate threshold, or the conversion rate of each initial tag is smaller than the corresponding conversion rate threshold, the initial tags are divided into for useless labels.

Further, after dividing the initial label into useless labels and popular labels, determine the type of the initial label, when the initial label is a popular label, keep the initial label; when the initial label is useless When tagging, delete the initial tag.

In this embodiment, the conversion rate refers to the rate at which initial tags are converted into advanced tags.

In this implementation, the click-through rate and conversion rate of each initial tag are monitored in real time within a preset period, useless tags are deleted, and the entire tag system is continuously optimized through training and learning to ensure the timeliness of the initial tags in the tag system, while improving the accuracy of the recommended labels.

Pushing module 207: configured to push the initial label in the label system corresponding to the target node when monitoring that the target node is triggered.

Exemplarily, when a node that triggers property control is detected, the property control model label in the label system corresponding to the property control model is pushed to assist the user in making decisions quickly and determine the priority of property control, such as freezing first. The bank deposit of the person subject to execution or the local real estate of the person subject to execution should be seized first, which improves the efficiency of the user's case handling.

In this embodiment, the labels are directly given to different application models, thereby reducing the model strategy operation process.

Further, after the push module 207 pushes the initial label in the label system corresponding to the target node, when monitoring the user's reprocessing instruction for the pushed initial label, parse the reprocessing instruction to obtain the User's reprocessing conditions; inputting the reprocessing conditions into the preset classification model to obtain a new label, and combining the new label and the pushed initial label to obtain an advanced label; the Advanced tab.

In this implementation, by parsing the reprocessing instruction fed back by the user, inputting the reprocessing condition into the preset classification model to obtain a new tag, and combining the new tag with the pushed initial tag The calculation obtains advanced tags, which responds to users in a timely manner, improves the timeliness of recommended tags, and improves the efficiency of case handling.

To sum up, the device for pushing labels based on big data described in this embodiment collects raw data from multiple preset data sources, wherein the raw data corresponds to a node identifier; The original data is cleaned according to a preset data cleaning strategy to obtain sample data of multiple nodes; multi-dimensional target features are extracted from the sample data of each node, and the multi-dimensional target features are classified according to a preset The model is classified to obtain the initial label of each node; the initial label of each node is clustered to form a label system of different objects; when the target node in the multiple nodes is monitored to be triggered, push The initial label in the label system corresponding to the target node.

In the method for pushing labels based on big data described in this embodiment, on the one hand, the original data collected from different data sources is cleaned through a preset data cleaning strategy, and the sample data of each node is obtained by cleaning the original data. , delete the problem data, ensure the consistency and integrity of the obtained sample data, and improve the quality of the sample data. On the other hand, by extracting the multi-dimensional target features in the sample data of each node, the multi-dimensional target The feature is input into the preset classification model to classify the initial label of each node, which improves the efficiency of calculating the initial label. At the same time, the initial label of each node is clustered into a label system of different objects, which improves the recommendation of labels. 's accuracy.

In addition, by monitoring the click rate and conversion rate of each initial tag in real time within a preset period, deleting useless tags, and continuously optimizing the entire tag system through training and learning, ensuring the timeliness of the initial tags in the tag system and improving the recommended tags. 's accuracy.

Embodiment 3

Referring to FIG. 3 , it is a schematic structural diagram of an electronic device according to Embodiment 3 of the present invention. In a preferred embodiment of the present invention, the electronic device 3 includes a memory 31 , at least one processor 32 , at least one communication bus 33 and a transceiver 34 .

Those skilled in the art should understand that the structure of the electronic device shown in FIG. 3 does not constitute a limitation of the embodiments of the present invention, and may be a bus-type structure or a star-shaped structure, and the electronic device 3 may also include a ratio more or less other hardware or software, or a different arrangement of components is shown.

In some embodiments, the electronic device 3 is an electronic device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits , programmable gate arrays, digital processors and embedded devices. The electronic device 3 may also include a client device, which includes but is not limited to any electronic product that can perform human-computer interaction with a client through a keyboard, a mouse, a remote control, a touchpad, or a voice-activated device, for example, Personal computers, tablets, smartphones, digital cameras, etc.

It should be noted that the electronic device 3 is only an example, and other existing or future electronic products that can be adapted to the present invention should also be included within the protection scope of the present invention, and are incorporated herein by reference .

In some embodiments, the memory 31 is used to store program codes and various data, such as the big data-based tag push device 20 installed in the electronic device 3 , and to achieve high speed during the operation of the electronic device 3 , Automatically complete program or data access. Described memory 31 comprises read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM) , One-time Programmable Read-Only Memory (One-time Programmable Read-Only Memory, OTPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM), CompactDisc Read-Only Memory , CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

In some embodiments, the at least one processor 32 may be composed of integrated circuits, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits packaged with the same function or different functions, including one Or a combination of multiple central processing units (Central Processing Units, CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The at least one processor 32 is the control core (Control Unit) of the electronic device 3, and uses various interfaces and lines to connect various components of the entire electronic device 3, by running or executing the program stored in the memory 31 or module, and call the data stored in the memory 31 to perform various functions of the electronic device 3 and process data, for example, to perform the function of tag push based on big data.

In some embodiments, the at least one communication bus 33 is configured to enable connection communication between the memory 31 and the at least one processor 32 and the like.

Although not shown, the electronic device 3 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 32 through a power management device, so as to be implemented by the power management device Manage charging, discharging, and power management functions. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 3 may further include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

It should be understood that the embodiments are only used for illustration, and are not limited by this structure in the scope of the patent application.

The above-mentioned integrated units implemented in the form of software functional modules may be stored in a computer-readable storage medium. The above-mentioned software function modules are stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, an electronic device, or a network device, etc.) or a processor (processor) to execute the methods described in the various embodiments of the present invention. part.

In a further embodiment, with reference to FIG. 2 , the at least one processor 32 can execute the operating device of the electronic device 3 and various installed applications (such as the big data-based label pushing device 20 ), Program code and the like, for example, the above-mentioned respective modules.

Program codes are stored in the memory 31, and the at least one processor 32 can call the program codes stored in the memory 31 to perform related functions. For example, each module described in FIG. 2 is a program code stored in the memory 31 and executed by the at least one processor 32, so as to realize the functions of the various modules to achieve tag push based on big data the goal of.

In an embodiment of the present invention, the memory 31 stores a plurality of instructions, and the plurality of instructions are executed by the at least one processor 32 to achieve the purpose of tag push based on big data.

Specifically, for the specific implementation method of the above instruction by the at least one processor 32, reference may be made to the description of the relevant steps in the embodiment corresponding to FIG. 1 , and details are not described herein.

The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division manners in actual implementation.

The modules described as separate components may or may not be physically separated, and components shown as modules may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

It will be apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, but that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of the invention is to be defined by the appended claims rather than the foregoing description, which are therefore intended to fall within the scope of the claims. All changes within the meaning and range of the equivalents of , are included in the present invention. Any reference signs in the claims shall not be construed as limiting the involved claim. Furthermore, it is clear that the word "comprising" does not exclude other units or, and the singular does not exclude the plural. Several units or means recited in the device claims can also be realized by one unit or means by means of software or hardware. The terms first, second, etc. are used to denote names and do not denote any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent substitutions can be made without departing from the spirit and scope of the technical solutions of the present invention.

Claims (10)

1. a big data-based label push method, is characterized in that, described big data-based label push method comprises: Collect a plurality of raw data from a plurality of preset data sources, wherein each of the raw data corresponds to a node identifier; Perform data cleaning on each of the raw data according to a preset data cleaning strategy to obtain sample data; Extract multi-dimensional target features from the sample data corresponding to each node, classify the multi-dimensional target features according to a preset classification model, and obtain the initial label of each node; Perform cluster analysis on the initial label of each node to form a label system of different objects; When it is detected that the target node is triggered, the initial label in the label system corresponding to the target node is pushed. 2. The method for pushing labels based on big data as claimed in claim 1, wherein said performing cluster analysis on the initial label of each node to form a label system of different objects comprises: According to the k-means clustering algorithm, the initial label of each node is clustered to obtain a plurality of objects; Taking any one of the multiple objects as the target object, the target object and the initial label corresponding to the target object are set as the label system corresponding to the target object. 3. The method for pushing labels based on big data as claimed in claim 1, characterized in that, after the initial label of each node is subjected to cluster analysis to form a label system of different objects, the method further comprises: Monitor the click-through rate and conversion rate of each initial tag in a preset period in real time; Judging whether the click-through rate of each initial label is greater than the corresponding click-through rate threshold, and judging whether the conversion rate of each initial label is greater than the corresponding conversion rate threshold; When the click-through rate of each initial tag is greater than or equal to the corresponding click-through rate threshold, and the conversion rate of each initial tag is greater than or equal to the corresponding conversion rate threshold, the initial tags are divided into popular tags; When the click rate of each initial tag is smaller than the corresponding click rate threshold, or the conversion rate of each initial tag is smaller than the corresponding conversion rate threshold, the initial tag is classified as a useless tag. 4. The method for pushing labels based on big data according to claim 1, wherein, performing data cleaning on the raw data according to a preset data cleaning strategy, and obtaining sample data comprising: Identify the node ID of each raw data; obtaining a preset data cleaning strategy corresponding to the node identifier; Clean the original data corresponding to the node identifier according to the preset data cleaning strategy; Converting the cleaned raw data into structured data of a preset type; Classifying the structured data according to the node identifiers to obtain sample data, and storing the sample data in a preset database. 5. The method for pushing labels based on big data as claimed in claim 1, wherein the multi-dimensional target feature is extracted from the sample data corresponding to each node, and the multi-dimensional target feature is preset according to the The classification model is classified, and the initial label of each node is obtained including: Read the sample data of each node from the preset database according to the node identification of each node and the query language HQL grammar rule oriented to the node identification; Extract multi-dimensional target features from the read sample data of each node according to a preset algorithm; The multi-dimensional target features are input into the preset classification model for classification to obtain the initial label of each node, wherein the initial label is stored in the blockchain node. 6. The big data-based label push method according to claim 5, wherein the extraction of multi-dimensional target features from the read sample data of each node according to a preset algorithm comprises: Extract the first feature from the read sample data of each node according to the preset feature dimension; Process the read sample data of each node through the trained model to obtain the second feature; The first feature and the second feature are combined to obtain a multi-dimensional target feature. 7. The method for pushing labels based on big data according to claim 1, wherein after the initial label in the label system corresponding to the target node is pushed, the method further comprises: When monitoring the user's reprocessing instruction for the pushed initial label, parse the reprocessing instruction to obtain the user's reprocessing condition; Inputting the reprocessing conditions into the preset classification model to obtain a new label, and combining the new label and the pushed initial label to obtain an advanced label; Push the advanced tag. 8. A big data-based label pushing device, wherein the big data-based label pushing device comprises: a collection module, configured to collect a plurality of raw data from a plurality of preset data sources, wherein each of the raw data corresponds to a node identifier; a cleaning module, configured to perform data cleaning on each of the raw data according to a preset data cleaning strategy to obtain sample data; A classification module, used for extracting multi-dimensional target features from the sample data corresponding to each node, classifying the multi-dimensional target features according to a preset classification model, and obtaining the initial label of each node; an analysis module for performing cluster analysis on the initial label of each node to form a label system of different objects; The push module is configured to push the initial label in the label system corresponding to the target node when it is detected that the target node is triggered. 9. An electronic device, characterized in that the electronic device comprises a processor, and the processor is configured to implement the big data-based tag according to any one of claims 1 to 7 when executing the computer program stored in the memory push method. 10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the computer program according to any one of claims 1 to 7 is implemented. Label push method for big data.

Images