JP7580593B2 - How to train an end-to-end sensitive text recall model, How to recall sensitive text - Google Patents

Description

Priority information

This application is filed based on a Chinese patent application with Chinese patent application number "2022106332414" and filing date of June 6, 2022, and claims priority to said Chinese patent application, the entire contents of which are hereby incorporated by reference into this application.

The present disclosure relates to the field of data processing technology, specifically to the field of artificial intelligence technology such as deep learning, and more specifically to a method for training an end-to-end sensitive text recall model, and a sensitive text recall method.

Text within an application is one of the main ways to convey information to users, but sensitive text containing harmful or illegal information can bring a bad user experience, pose regulatory risks, harm social trends, and ultimately lead to users abandoning the application product. Word list recall can promptly recall sensitive text in text information, thereby ensuring product safety and improving user experience.

The present disclosure provides a method for training an end-to-end sensitive text recall model, a sensitive text recall method, an apparatus, a device, and a storage medium.

According to an embodiment of the first aspect of the present disclosure, a method for training an end-to-end sensitive text recall model is provided, the method including the steps of: obtaining a preset word list and a first random text corpus in a sensitive text block scene, where text corresponding to a term in the preset word list is sensitive text; constructing positive sample data based on the preset word list and constructing negative sample data based on the first random text corpus; performing iterative training on an initial text classification model based on the positive sample data and the negative sample data by a human evaluation method and a multi-sample splice sampling method to obtain a text classification model whose model indicator reaches a target standard after training; and generating an end-to-end sensitive text recall model based on model parameters of the text classification model whose model indicator reaches the target standard, where the end-to-end sensitive text recall model has acquired word list recall ability through learning.

In some embodiments, the step of performing iterative training of an initial text classification model based on the positive sample data and the negative sample data using a human evaluation method and a multi-sample splice sampling method includes the steps of: dividing the positive sample data and the negative sample data into a training set and a validation set as training samples; training a text classification model based on the training set and the validation set to obtain an optimal model; obtaining a test set, evaluating the optimal model based on the test set to obtain a model evaluation result; updating the training samples based on the model evaluation result and the test set using a human evaluation method and a multi-sample splice sampling method; and re-dividing the updated training samples into a training set and a validation set, and training a text classification model based on the training set and the validation set to obtain an optimal model until the model index reaches a target standard after training is completed.

In some embodiments, the test set includes a recall sample and a second random text corpus, and the step of evaluating the optimal model based on the test set to obtain a model evaluation result includes the steps of: inputting the recall sample from the test set into the optimal model and obtaining a first prediction result output from the optimal model; determining a recall rate of the optimal model based on the first prediction result and actual label information corresponding to the recall sample; inputting the second random text corpus from the test set into the optimal model and obtaining a second prediction result output from the optimal model; and determining a precision rate of the optimal model based on the second prediction result and actual label information corresponding to the second random text corpus.

In some embodiments, the step of updating the training samples based on the model evaluation results and the test set using a human evaluation scheme and a multi-sample splice sampling scheme includes the steps of: in response to the recall being less than a first threshold, obtaining a first human evaluation of the examples predicted to be negative examples from the first prediction results, and adding the examples of the recall samples that are mispredicted to be negative examples from the recall samples to the sample set to be updated based on the first human evaluation; and/or in response to the precision being less than a second threshold, obtaining a second human evaluation of the examples predicted to be positive examples from the second prediction results, and adding the text corpus of the examples of the second random text corpus that are mispredicted to be positive examples from the second random text corpus to the sample set to be updated based on the second human evaluation; splicing every N samples in the sample set to be updated into one sample, and updating the sample obtained after the splicing process to the training sample, where N is an integer greater than 1.

In some embodiments, N is 3.

In some embodiments, the text classification model includes a first long short-term memory network (LSTM) layer, an average pooling layer, a second LSTM layer, a max pooling layer, a spliced concat layer, a reduced dropout layer, and a classification layer, the first LSTM layer extracts sample text features, the average pooling layer pools the text features to obtain first path features, and the second LSTM layer calculates features for the last hidden layer output of the first LSTM layer. Extraction is performed, and the extracted features are input to the max pooling layer, which pools the output of the second LSTM layer to obtain a second path feature, the splice concat layer splices the first path feature and the second path feature to obtain a splice feature, the reduced dropout layer performs a dropout operation on the splice feature, and the classification layer classifies the features output from the reduced dropout layer to obtain a classification prediction value.

According to an embodiment of a second aspect of the present disclosure, there is provided a sensitive text recall method, the method including the steps of obtaining a target text, and predicting the target text based on a pre-trained end-to-end sensitive text recall model to determine whether to recall the target text, the end-to-end sensitive text recall model having acquired word list recall capability by training, the end-to-end sensitive text recall model being trained using the method of the first aspect.

In some embodiments, the step of predicting the target text based on the pre-trained end-to-end sensitive text recall model and determining whether to recall the target text includes the steps of extracting text features of the target text through the first long short-term memory network (LSTM) layer, pooling the text features through the average pooling layer to obtain first path features, and performing feature extraction on the last hidden layer output of the first LSTM layer through the second LSTM layer to extract the extracted features. The method includes inputting features to the max pooling layer, pooling the output of the second LSTM layer through the max pooling layer to obtain a second path feature, splicing the first path feature and the second path feature to obtain a spliced feature, and performing a Dropout operation on the spliced feature through the reduced Dropout layer, classifying the features output from the reduced Dropout layer through the classification layer to obtain a classification prediction value, and determining whether to recall the text to be processed based on the prediction value.

According to an embodiment of the third aspect of the present disclosure, there is provided an apparatus for training an end-to-end sensitive text recall model, the apparatus comprising: an acquisition module for acquiring a preset word list and a first random text corpus in a sensitive text block scene, where text corresponding to a term in the preset word list is sensitive text; a construction module for constructing positive sample data based on the preset word list, where the construction module constructs negative sample data based on the first random text corpus; and a processing module for performing iterative training on an initial text classification model based on the positive sample data and the negative sample data by a human evaluation method and a multi-sample splice sampling method, to obtain a text classification model whose model indicator reaches a target standard after training is completed, and generating an end-to-end sensitive text recall model based on model parameters of the text classification model whose model indicator reaches the target standard, where the end-to-end sensitive text recall model has acquired word list recall ability through learning.

In some embodiments, the processing module specifically performs the steps of dividing the positive sample data and the negative sample data into a training set and a validation set as training samples, training a text classification model based on the training set and the validation set to obtain an optimal model, obtaining a test set, evaluating the optimal model based on the test set to obtain a model evaluation result, updating the training samples based on the model evaluation result and the test set through a human evaluation method and a multi-sample splice sampling method, re-dividing the updated training samples into a training set and a validation set, and training a text classification model based on the training set and the validation set until the model index reaches the target standard after training is completed to obtain an optimal model.

In some embodiments, the test set includes a recall sample and a second random text corpus, and the processing module specifically inputs the recall sample from the test set into the optimal model, obtains a first prediction result output from the optimal model, and determines a recall rate of the optimal model based on the first prediction result and actual label information corresponding to the recall sample, inputs the second random text corpus from the test set into the optimal model, obtains a second prediction result output from the optimal model, and determines a precision rate of the optimal model based on the second prediction result and actual label information corresponding to the second random text corpus.

In some embodiments, the processing module specifically, in response to the recall being less than a first threshold, obtains a first human evaluation result of an example predicted to be a negative example from the first prediction result, and adds a sample of an example of the recall sample that is mispredicted to be a negative example to the sample set to be updated based on the first human evaluation result; and/or, in response to the precision being less than a second threshold, obtains a second human evaluation result of an example predicted to be a positive example from the second prediction result, and adds a text corpus of an example of the second random text corpus that is mispredicted to be a positive example to the sample set to be updated based on the second human evaluation result, splices every N samples from the sample set to be updated into one sample, and updates the sample obtained after the splicing process to the training sample, where N is an integer greater than 1.

In some embodiments, N is 3.

In some embodiments, the text classification model includes a first long short-term memory network (LSTM) layer, an average pooling layer, a second LSTM layer, a max pooling layer, a spliced concat layer, a reduced dropout layer, and a classification layer, the first LSTM layer extracts sample text features, the average pooling layer pools the text features to obtain first path features, and the second LSTM layer applies a feature to the last hidden layer output of the first LSTM layer. Feature extraction is performed, and the extracted features are input to the max pooling layer, which pools the output of the second LSTM layer to obtain a second path feature, the splice concat layer splices the first path feature and the second path feature to obtain a spliced feature, the reduced dropout layer performs a dropout operation on the spliced feature, and the classification layer classifies the features output from the reduced dropout layer to obtain a classification prediction value.

According to a fourth embodiment of the present disclosure, there is provided a sensitive text recall device, the device comprising: an acquisition module for acquiring a target text; and a prediction module for predicting the target text based on a pre-trained end-to-end sensitive text recall model to determine whether to recall the target text, the end-to-end sensitive text recall model having acquired word list recall capability through learning, the end-to-end sensitive text recall model being trained using a method according to any of the embodiments of the first aspect of the present disclosure.

In some embodiments, the prediction module specifically extracts text features of the target text through the first long short-term memory network (LSTM) layer, pools the text features through the average pooling layer to obtain first path features, performs feature extraction on the last hidden layer output of the first LSTM layer through the second LSTM layer, inputs the extracted features to the max pooling layer, pools the output of the second LSTM layer through the max pooling layer to obtain second path features, splices the first path features and the second path features to obtain splice features, performs a dropout operation on the splice features through the reduced dropout layer, classifies the features of the reduced dropout layer output through the classification layer to obtain a classification prediction value, and determines whether to recall the target text based on the prediction value.

According to an embodiment of the fifth aspect of the present disclosure, there is provided an electronic device, comprising at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor can perform a method according to any of the embodiments of the first or second aspects.

According to an embodiment of the sixth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing the computer to perform a method according to any of the embodiments of the first or second aspect.

According to an embodiment of the seventh aspect of the present disclosure there is provided a computer program which , when running on a computer, causes the computer to perform a method according to any of the embodiments of the first or second aspect.

According to the technology disclosed herein, positive sample data and negative sample data are constructed based on a word list and a large amount of actual data, and an iterative training process is performed on a text classification model based on the constructed positive sample data and negative sample data to generate an end-to-end sensitive text recall model, so that the end-to-end sensitive text recall model can learn the word list recall ability, thereby improving the knowledge generalization ability of the end-to-end sensitive text recall model and improving the recall ability of the model for sensitive text, thereby enabling the end-to-end sensitive text recall model to realize word list recall and improve the word list generalization ability.

It should be understood that the contents described in this section are not intended to identify key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

The drawings are used for a better understanding of the present technical solution, and are not intended to limit the present disclosure.
FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure. 1 is a schematic flowchart of model training provided by an embodiment of the present disclosure. FIG. 13 is a schematic diagram according to a second embodiment of the present disclosure. FIG. 1 is a schematic diagram of an architecture of a text classification model provided by an embodiment of the present disclosure. FIG. 13 is a schematic diagram according to a third embodiment of the present disclosure. FIG. 1 is a schematic diagram of a training apparatus for an end-to-end sensitive text recall model provided by an embodiment of the present disclosure. 1 is a schematic diagram of a sensitive text recall device provided by an embodiment of the present disclosure; FIG. 2 is a block diagram of an electronic device provided by an embodiment of the present disclosure.

Hereinafter, exemplary embodiments of the present disclosure will be described in conjunction with the drawings, and various details of the embodiments of the present disclosure will be included therein for ease of understanding, and they should be regarded as merely exemplary. Therefore, it should be recognized that those skilled in the art can make various changes and modifications to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, the following description will omit the description of well-known functions and structures. In the description of the present disclosure, unless otherwise specified, "/" means "or", for example, the description A/B can represent A or B, and "and/or" in this specification describes a related relationship between related objects and indicates that three relationships can exist. For example, the description A and/or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

Using word list blocking text to block sensitive information (such as information that violates relevant laws and regulations) in text is an important means to eliminate harmful information, but the generalizability of the word list policy used in the related technology is poor. For example, "We love XX" can be used as a term to recall the text "We love XX, we love YY", but cannot recall the text "We love XX very much", which is very similar in meaning to the above text. Therefore, the embodiment of the present disclosure proposes a method for training an end-to-end sensitive text recall model, and can build a sensitive text recall model with high knowledge generalization ability, and based on this model, end-to-end text prediction and recall can be realized, and semantically similar expression text can be effectively recalled.

Referring to FIG. 1, FIG. 1 is a schematic diagram of a method for training an end-to-end sensitive text recall model according to a first embodiment of the present disclosure. As shown in FIG. 1, the method may include, but is not limited to, the following steps S101 to S104.

Step S101: Obtain a predefined word list and a first random text corpus in a sensitive text block scene.

In an embodiment of the present disclosure, the pre-defined word list includes corresponding sensitive text terms, which may include multi-word terms and single-word terms, and the first random text corpus includes, but is not limited to, text captured by the pre-defined word list or text that has been evaluated by a human to be normal.

For example, obtain a pre-defined word list consisting of terms corresponding to sensitive text that needs to be blocked in a sensitive text blocking scene, and then obtain a first random text corpus according to the actual situation.

Step S102: construct positive sample data based on a predefined word list, and construct negative sample data based on a first random text corpus.

For example, delimiters of multi-word terms and single-word terms in a preset word list are removed, and the remaining text is used as a model positive sample to evaluate the proportion of harmful corpus in the first random text corpus; if the proportion of harmful corpus in the first random corpus is equal to or less than a preset threshold (e.g., 1%), the text in the first random text corpus is directly randomly extracted as a negative sample; if the proportion of harmful corpus in the first random corpus is greater than the preset threshold, the first random text corpus is examined to remove harmful corpus in the first random text corpus; if the proportion of harmful corpus in the first random corpus is equal to or less than the preset threshold, the text in the first random corpus after processing is randomly extracted as a negative sample.

Step S103: Based on the positive sample data and the negative sample data, an iterative training process is performed on the initial text classification model by using a human evaluation method and a multi-sample splice sampling method, and a text classification model whose model index reaches the target standard after the training is completed is obtained.

In the embodiment of the present disclosure, the text classification model includes, but is not limited to, TextCNN (Text Convolutional Neural Networks), FastText for fast text classification, and BERT (Bidirectional Encoder Representations from Transformers). The multi-sample splice sampling method refers to splicing a preselected number (e.g., three) of texts from a plurality of human - evaluated marked-up texts into one sample. For example, three texts each from a plurality of human - evaluated marked-up texts are spliced into one sample.

For example, an iterative training process is performed on an initial text classification model based on positive sample data and negative sample data, and during the iterative training process, at each preset number of training steps (e.g., 100 steps), the current model's indicators are calculated to evaluate the effectiveness of the current model, the training samples are updated using a multi-sample splice sampling method, the model's loss value is calculated based on a loss function, the gradient is inversely calculated based on the loss value to optimize the model parameters, and the updated training samples are used to train the model after parameter optimization. The above steps are repeated until the model's indicators reach the target standard, and this model is regarded as a text classification model.

In the embodiments of the present disclosure, the target indicator refers to a preset indicator for determining whether the model achieves the target effect, and the model indicator includes, but is not limited to, the model precision and model recall.

In the embodiment of the present disclosure, the formula for calculating the precision rate can be expressed as follows:

Accuracy is the precision rate, TP is the number of samples for which the model predicts positive samples as positive samples, FN is the number of samples for which the model predicts positive samples as negative samples, FP is the number of samples for which the model predicts negative samples as positive samples, and TN is the number of samples for which the model predicts negative samples as negative samples.

In the embodiment of the present disclosure, the formula for the recall can be expressed as follows:

recall is the recall, TP is the number of samples for which the model predicts a positive sample as a positive sample, and FN is the number of samples for which the model predicts a positive sample as a negative sample.

In an embodiment of the present disclosure, the calculation formula for the loss value can be expressed as follows:

L is the loss value, i is the i-th sample, yi is the label of sample i, positive samples are 1, negative samples are 0, and pi is the probability of predicting sample i as a positive sample.

Step S104: Generate an end-to-end sensitive text recall model based on the model parameters of the text classification model whose model indicators reach the target standard.

In the embodiment of the present disclosure, the end-to-end sensitive text recall model has already learned the word list recall capability.

For example, an end-to-end sensitive text recall model is generated using a pre-configured neural network model structure based on the model parameters of a text classification model whose model metrics reach a target standard.

In an embodiment of the present disclosure, the pre-configured neural network model structure may be the same as the text classification model described above.

By implementing the embodiments of the present disclosure, an end-to-end sensitive text recall model can be generated by performing iterative training on a text classification model based on the constructed positive and negative sample data, thereby improving the knowledge generalization ability of the end-to-end sensitive text recall model and improving the recall ability of the model for sensitive text.

In the embodiment of the present disclosure, the text classification model can learn the word list recall ability by constructing positive and negative samples. However, since the generalization ability of the neural network is high and it may recall many texts that do not meet the word list recall needs, the embodiment of the present disclosure designed a model training flow in an offline training environment to ensure that the model can more accurately learn the appropriate generalization ability. As an example, refer to FIG. 2, which is a schematic flowchart of model training provided by the embodiment of the present disclosure. As shown in FIG. 2, the embodiment of the present disclosure trains the classification model by constructing positive and negative samples, and in the model iteration process, samples that the model misrecalls are added to the positive/negative samples in a human evaluation manner based on the model's precision and recall indicators, thereby improving the model indicator.

As an example, refer to FIG. 3, which is a schematic diagram of a model training method according to a second embodiment of the present disclosure. As shown in FIG. 3, the implementation process of performing iterative training on an initial text classification model based on the positive sample data and the negative sample data through a human evaluation method and a multi-sample splice sampling method may include, but is not limited to, the following steps S301 to S305.

Step S301: Divide the positive sample data and negative sample data into a training set and a validation set as training samples.

For example, the positive sample data and the negative sample data are randomly divided in a predetermined ratio to obtain a training set and a validation set.

In some embodiments, for example, the ratio of the number of sample data in the training set and the validation set is 9:1, and the positive samples and negative samples are randomly divided in the above ratios, with 90% of the positive sample data and 90% of the negative sample data being the training set, and the remaining 10% of the positive sample data and 10% of the negative sample data being the validation set.

Step S302: Train a text classification model based on the training set and validation set to obtain an optimal model.

For example, a text classification model is trained based on a training set, and the model is tested for the precision and recall effects of the current training step model at each preset training step (e.g., 100 steps) using a validation set, and the precision and recall effects of different training step models are compared to obtain an optimal model trained using the current training set and the validation set.

Step S303: Obtain a test set, evaluate the optimal model based on the test set, and obtain a model evaluation result.

For example, obtain a test set containing a recall set and random data, recall the test set based on the optimal model obtained in the previous step, calculate the current index of the model based on the model output, and take this index as the evaluation result of the optimal model.

In some embodiments, the test set may include a recall sample and a second random text corpus, and the step of evaluating the optimal model based on the test set and obtaining a model evaluation result may include the steps of: inputting the recall sample from the test set into the optimal model and obtaining a first prediction result output from the optimal model; determining a recall rate of the optimal model based on the first prediction result and actual label information corresponding to the recall sample; inputting the second random text corpus from the test set into the optimal model and obtaining a second prediction result output from the optimal model; and determining a precision rate of the optimal model based on the second prediction result and actual label information corresponding to the second random text corpus.

In an embodiment of the present disclosure, the recall samples are pre-acquired to test the recall ability of the model, and the acquisition method of the second random text corpus may be the same as the acquisition method of the first random text corpus.

Note that the text in the first random text corpus is different from the text in the second random text corpus.

As an example, a text obtained by a pre-set word list or a text with a normal artificial screening result can be randomly split into two parts, one part as a first random text corpus and the other part as a second random text corpus, ensuring that the text in the first random text corpus is different from the text in the second random text corpus.

For example, a recall sample from the test set is input as input data into the optimal model to predict label information for each sample in the recall sample, the predicted label information for each sample is obtained as a first prediction result, and the recall of the optimal model is calculated based on the first prediction result and the actual label information corresponding to the recall sample using the above-mentioned recall calculation formula; a second random text corpus from the test set is input as input data into the optimal model to predict label information for each sample in the second random text corpus, the predicted label information for each sample is obtained as a second prediction result, and the precision of the optimal model is calculated based on the second prediction result and the actual label information corresponding to the second random text corpus using the above-mentioned precision calculation formula.

In the embodiment of the present disclosure, the formula for calculating the precision rate can be expressed as follows:

Precision is the precision rate, TP is the number of samples that predict positive samples as positive samples, and FP is the number of samples that predict negative samples as positive samples.

Step S304: Based on the model evaluation results and the test set, update the training samples using the human evaluation method and the multi-sample splice sampling method.

For example, if the model evaluation results do not reach the expected results, the human evaluation model predicts data that is a positive example, and the data that is actually classified as a negative example is added to the training sample to update the training sample.

In some embodiments, the step of updating the training samples using the human evaluation method and the multi-sample splice sampling method based on the model evaluation results and the test set can include the steps of: obtaining a first human evaluation result predicted to be a negative example of the first prediction result in response to the recall rate being smaller than a first threshold, and adding a sample of the recall sample that is mispredicted to be a negative example based on the first human evaluation result to the sample set to be updated; and/or obtaining a second human evaluation result predicted to be a positive example of the second prediction result in response to the precision rate being smaller than a second threshold, and adding a text corpus of the second random text corpus that is mispredicted to be a positive example based on the second human evaluation result to the sample set to be updated; splicing every N samples in the sample set to be updated into one sample, and updating the sample obtained by performing the splicing process to the training sample, where N is an integer greater than 1.

In some embodiments, N is 3. In some embodiments, in response to the recall of the current model being less than a first threshold, obtaining a first prediction result in which the model prediction is a negative example, performing human evaluation on the samples predicted to be negative examples from the first prediction result, selecting positive samples from the samples that were mispredicted to be negative examples, adding the positive samples that were mispredicted to be negative examples to a sample set to be updated, splicing every third sample in the sample set to be updated into one sample, updating the sample obtained by performing the splicing process to a training sample, and in response to the precision of the current model being greater than or equal to a second threshold, not processing the second prediction result.

In some other embodiments, in response to the recall of the current model being equal to or greater than a first threshold, the first prediction result is not processed. In response to the precision of the current model being less than a second threshold, a second prediction result is obtained in which the model prediction is a positive example, samples predicted to be positive examples in the second prediction result are manually evaluated, negative samples among the samples that are mispredicted to be positive examples are selected, the negative samples mispredicted to be positive examples are added to a sample set to be updated, every third sample in the sample set to be updated is spliced into one sample, and the sample obtained by performing the splicing process is updated to a training sample.

In some other embodiments, in response to the recall rate of the current model being less than a first threshold, a first prediction result in which the model prediction is a negative example is obtained, a sample predicted to be a negative example from the first prediction result is evaluated by a human, a positive sample from the above samples that is mispredicted as a negative example is selected, and the positive sample mispredicted as a negative example is added to the sample set to be updated; in response to the precision rate of the current model being less than a second threshold, a second prediction result in which the model prediction is a positive example is obtained, a sample predicted to be a positive example from the second prediction result is evaluated by a human, a negative sample from the above samples that is mispredicted as a positive example is selected, and the negative sample mispredicted as a positive example is added to the sample set to be updated; every third sample in the sample set to be updated is spliced into one sample, and the sample obtained by performing the splicing process is updated to a training sample.

In some other embodiments, in response to the recall rate of the current model being equal to or greater than the first threshold, if the precision rate of the current model is equal to or greater than the second threshold, it is determined that the index of the current model has reached the target standard. Step S305: Repartition the updated training samples into a training set and a validation set, and train the text classification model based on the training set and the validation set until the model index reaches the target standard after the training is completed, thereby obtaining an optimal model.

For example, the updated training samples are re-divided at a preset ratio to obtain new training sets and validation sets, and the new training sets and validation sets are used to execute step S302 again, and the subsequent steps are executed based on the actual situation to retrain the text classification model until the model indicators after training reach the target standard.

Note that, according to an embodiment of the present disclosure, a text classification model can be trained offline using model version iteration to obtain an end-to-end sensitive text recall model. The end-to-end sensitive text recall model can be deployed on a server to directly recognize text in linked applications to recall sensitive text, thereby achieving end-to-end sensitive text recall.

In an embodiment of the present disclosure, the text classification model may include a first long short-term memory network (LSTM) layer, an average pooling layer, a second LSTM layer, a max pooling layer, a spliced Concat layer, a reduced Dropout layer, and a classification layer. As an example, refer to FIG. 4, which is a schematic diagram of the architecture of a text classification model provided by an embodiment of the present disclosure. As shown in FIG. 4, the first LSTM layer extracts the text features of the samples, the mean-pooling layer pools the text features to obtain the first path features, the second LSTM layer extracts features from the output of the last hidden layer in the first LSTM layer (i.e., hn shown in FIG. 4) and inputs the extracted features to the max-pooling layer, which pools the output of the second LSTM layer to obtain the second path features, the concat layer splices the first path features and the second path features to obtain spliced features, the Dropout layer performs a Dropout operation on the spliced features, and the classification layer classifies the features output from the Dropout layer to obtain classified prediction values.

In addition, the Dropout layer can effectively prevent the occurrence of overfitting. The Dropout function is a special activation function. During the training phase of the text classification model, the ratio of the number of weights activated by the Dropout layer to the total number of weights in the Dropout layer should be kept at the retention probability keep_prob (generally 0.5), and during the prediction phase, keep_prob = 1.

Referring to FIG. 5, FIG. 5 is a schematic diagram of a sensitive text recall method according to a third embodiment of the present disclosure. As shown in FIG. 5, the method may include, but is not limited to, the following steps S501 to S502.

Step S501: Obtain the text to be processed.

For example, text information within a related application can be obtained as the text to be processed.

Step S502: Predict the text to be processed based on the pre-trained end-to-end sensitive text recall model, and determine whether to recall the text to be processed.

In an embodiment of the present disclosure, the end-to-end sensitive text recall model has already been trained to have word list recall capability, and the end-to-end sensitive text recall model is trained in a manner provided by any of the embodiments of the present disclosure.

For example, the text to be processed is input into a pre-trained end-to-end sensitive text recall model to predict the text and determine whether the text contains sensitive text, thereby determining whether the text to be processed should be recalled.

By implementing an embodiment of the present disclosure, it is possible to predict the text to be processed based on a pre-trained end-to-end sensitive text recall model and determine whether to recall the text to be processed, thereby improving the recall for sensitive text.

Referring to FIG. 6, FIG. 6 is a schematic diagram of a training apparatus for an end-to-end sensitive text recall model provided by an embodiment of the present disclosure. As shown in FIG. 6, the apparatus includes an acquisition module 601, a construction module 602, and a processing module 603.

The acquisition module 601 acquires a preset word list and a first random text corpus in a sensitive text block scene, and the text corresponding to the terms in the preset word list is the sensitive text. The construction module 602 constructs positive sample data according to the preset word list and constructs negative sample data according to the first random text corpus. The processing module 603 performs iterative processing training on the initial text classification model based on the positive sample data and the negative sample data by a human evaluation method and a multi-sample splice sampling method, to obtain a text classification model whose model indicator reaches the target standard after the training is completed, and generates an end-to-end sensitive text recall model based on the model parameters of the text classification model whose model indicator reaches the target standard, and the end-to-end sensitive text recall model has already obtained the word list recall ability through learning.

In some embodiments, the processing module 603 specifically performs the steps of dividing the positive sample data and the negative sample data into a training set and a validation set as training samples, training a text classification model based on the training set and the validation set to obtain an optimal model, obtaining a test set, evaluating the optimal model based on the test set, obtaining a model evaluation result, updating the training samples based on the model evaluation result and the test set through a human evaluation method and a multi-sample splice sampling method, redividing the updated training samples into a training set and a validation set, and training a text classification model based on the training set and the validation set until the model index reaches the target standard after training is completed to obtain an optimal model.

In some embodiments, the test set includes a recall sample and a second random text corpus, and the processing module 603 specifically inputs the recall sample from the test set into the optimal model, obtains a first prediction result output from the optimal model, determines a recall rate of the optimal model based on the first prediction result and actual label information corresponding to the recall sample, inputs the second random text corpus from the test set into the optimal model, obtains a second prediction result output from the optimal model, and determines a precision rate of the optimal model based on the second prediction result and actual label information corresponding to the second random text corpus.

In some embodiments, the processing module 603 specifically, in response to the recall being less than a first threshold, obtains a first human evaluation result that is predicted to be a negative example of the first prediction result, and adds a sample of the recall sample that is mispredicted to be a negative example based on the first human evaluation result to the sample set to be updated; and/or, in response to the precision being less than a second threshold, obtains a second human evaluation result that is predicted to be a positive example of the second prediction result, and adds a text corpus of the second random text corpus that is mispredicted to be a positive example based on the second human evaluation result to the sample set to be updated, splices every N samples in the sample set to be updated into one sample, and updates the sample obtained by performing the splicing process to a training sample, where N is an integer greater than 1.

In some embodiments, N is 3.

In some embodiments, the text classification model includes a first long short-term memory network (LSTM) layer, an average pooling layer, a second LSTM layer, a max pooling layer, a splice concat layer, a reduced dropout layer, and a classification layer, where the first LSTM layer extracts text features of the samples, the average pooling layer pools the text features to obtain first path features, the second LSTM layer extracts features from the output of the last hidden layer in the first LSTM layer and inputs the extracted features to the max pooling layer, the max pooling layer pools the output of the second LSTM layer to obtain second path features, the splice concat layer splices the first path features and the second path features to obtain splice features, the reduced dropout layer performs a dropout operation on the splice features, and the classification layer classifies the features output from the reduced dropout layer to obtain classified prediction values.

According to the device of the embodiment of the present disclosure, a text recall model can be obtained by performing iterative training on the text classification model based on the constructed positive sample data and negative sample data, thereby improving the knowledge generalization ability of the text recall model and improving the recall ability of the model for sensitive text.

Referring to FIG. 7, FIG. 7 is a schematic diagram of a sensitive text recall device provided by an embodiment of the present disclosure. As shown in FIG. 7, the device includes an acquisition module 701 and a prediction module 702. The acquisition module 701 acquires a target text, and the prediction module 702 predicts the target text based on a pre-trained end-to-end sensitive text recall model to determine whether to recall the target text, where the end-to-end sensitive text recall model has already acquired word list recall ability through learning, and the end-to-end sensitive text recall model is trained by a method described in any of the embodiments of the present disclosure.

In some embodiments, the prediction module 702 specifically extracts text features of the target text through a first long short-term memory network (LSTM) layer, pools the text features through an average pooling layer to obtain a first path feature, extracts features from the output of the last hidden layer in the first LSTM layer through a second LSTM layer, inputs the extracted features to a max pooling layer, pools the output of the second LSTM layer through a max pooling layer to obtain a second path feature, splices the first path feature and the second path feature to obtain a spliced feature, performs a Dropout operation on the spliced feature through a reduced Dropout layer, classifies the features output from the reduced Dropout layer through a classification layer to obtain a classified prediction value, and determines whether to recall the target text based on the prediction value.

According to an embodiment of the device disclosed herein, it is possible to predict the text to be processed based on a pre-trained end-to-end sensitive text recall model and determine whether to recall the text to be processed, thereby improving the recall for sensitive text.

The specific method for executing the operations of each module in the device of the above embodiment has already been described in detail in the embodiment relating to the method, and will not be described in detail here.

According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a computer-readable storage medium, and a computer program.

As shown in FIG. 8, a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device can be used to realize any of the end-to-end sensitive text recall model training methods or sensitive text recall methods of the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the description herein and/or the implementation of the present disclosure as sought.

As shown in FIG. 8, the electronic device includes one or more processors 801, memory 802, and interfaces for connecting each component, including high-speed and low-speed interfaces. Each component may be connected to each other by different buses and mounted on a common motherboard, or mounted in other manners as needed. The processor can process instructions executed in the electronic device, including instructions stored in the memory for displaying graphical information of a GUI on an external input/output device (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses can be used, along with multiple memories, if necessary. Similarly, multiple electronic devices can be connected, each providing a partial required operation (e.g., a server array, a blade server, or a multi-processor system). In FIG. 8, one processor 801 is taken as an example.

Memory 802 is a non-transitory computer-readable storage medium provided by the present disclosure. The memory stores instructions executed by at least one processor, causing the at least one processor to execute the method for training an end-to-end sensitive text recall model or the sensitive text recall method provided by the present disclosure. The non-transitory computer-readable storage medium of the present disclosure stores computer instructions for a computer to execute the method for training an end-to-end sensitive text recall model or the sensitive text recall method provided by the present application.

The memory 802, as a non-transitory computer-readable storage medium, stores non-transitory software programs, non-transitory computer-executable programs, and modules, such as computer instructions/modules (e.g., the acquisition module 601, the construction module 602, and the processing module 603 shown in FIG. 6) corresponding to the end-to-end sensitive text recall model training method in the embodiment of the present disclosure, or computer instructions/modules (e.g., the acquisition module 701 and the prediction module 702 shown in FIG. 7) corresponding to the sensitive text recall method. The processor 801 executes the non-transitory software programs, instructions, and modules stored in the memory 802 to perform various functional applications and data processing of the server, i.e., to realize the end-to-end sensitive text recall model training method or the sensitive text recall method in the embodiment of the above method.

The memory 802 may include a storage program area and a storage data area. The storage program area may store an operating system, an application required for at least one function. The storage data area may store data created by using the end-to-end sensitive text recall model training method or the electronic device of the sensitive text recall method, and the like. The memory 802 may also include high-speed random access memory, and may further include non-transient memory, such as at least one magnetic disk storage device, flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 802 may selectively include memory configured remotely with respect to the processor 801, and these remote memories may be connected to the end-to-end sensitive text recall model training method or the electronic device of the sensitive text recall method via a network. Examples of the above networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The electronic device of the end-to-end sensitive text recall model training method or the sensitive text recall method may further include an input device 803 and an output device 804. The processor 801, the memory 802, the input device 803, and the output device 804 may be connected via a bus or other methods, and FIG. 8 illustrates an example in which they are connected via a bus.

The input device 803 can receive input numeric or character information and generate key signal inputs related to user settings and function control of the end-to-end sensitive text recall model training method or the electronic device of the sensitive text recall method, for example, a touch screen, a keypad, a mouse, a track pad, a touch pad, a wand, one or more mouse buttons, a track ball, a joystick, etc. The output device 804 can include a display device, an auxiliary lighting device (e.g., LED), a haptic feedback device (e.g., vibration motor), etc. The display device can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device can be a touch screen.

Various embodiments of the systems and techniques described herein may be realized in digital electronic circuitry systems, integrated circuit systems, application specific integrated circuits (ASICs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs that may be executed and/or interpreted in a programmable system having at least one programmable processor, which may be an application specific or general purpose programmable processor, and that may receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

These computing programs (also referred to as programs, software, software applications, or code) include the implementation of these computing programs in machine instructions for a programmable processor, in high-level process and/or object-oriented programming languages, and/or in assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic disks, optical disks, memories, programmable logic devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including machine-readable media that receive machine instructions that are machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and/or data to a programmable processor.

To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other types of devices can also provide interaction with a user, for example, the feedback provided to the user can be any form of sensing feedback (e.g., vision feedback, auditory feedback, or haptic feedback) and can receive input from the user in any form (including acoustic, speech, or haptic input).

The systems and techniques described herein may be implemented on a computing system having a back-end component (e.g., a data server), or a computing system having a middleware component (e.g., an application server), or a computing system having a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with the embodiments of the systems and techniques described herein), or any combination of such back-end, middleware, and front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

The computer system may include a client and a server. The client and server are generally remote from each other and typically interact with each other via a communication network. The relationship between the client and the server is generated by a computer program that is executed on a corresponding computer and has a client-server relationship with each other. The server may be a cloud server, also called a cloud computing server or cloud host, which is a host product in a cloud computing service system and solves the deficiencies of difficult management and weak business scalability that exist in traditional physical hosts and VPS services (abbreviated as "Virtual Private Server", or "VPS"). The server may be a server of a distributed system or a server incorporating blockchain.

According to the technical proposal of the embodiment of the present disclosure, an iterative training process can be performed on the text classification model based on the constructed positive sample data and negative sample data to obtain a text recall model, thereby improving the knowledge generalization ability of the text recall model and improving the recall ability of the model for sensitive text.

In addition, the description of the above-mentioned method for training the end-to-end sensitive text recall model also applies to the device, electronic device, computer-readable storage medium, and computer program of the embodiments of the present disclosure, and therefore the description thereof will be omitted here.

It should be understood that steps can be rearranged, added, or removed using the various types of flows shown above. For example, the steps described in this disclosure may be performed in parallel, sequentially, or in a different order, but are not limited herein as long as the technical solution disclosed in this disclosure can achieve the desired results. The specific embodiments described above do not limit the scope of protection of the present disclosure.

It should be understood that those skilled in the art may make various modifications, combinations, subcombinations, and substitutions according to design requirements and other factors. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims (13)

Obtaining a preset word list and a first random text corpus in a sensitive text block scene, where text corresponding to a term in the preset word list is sensitive text;
constructing positive sample data based on the predefined word list and constructing negative sample data based on the first random text corpus;
According to the positive sample data and the negative sample data, iteratively train the initial text classification model through a human evaluation method and a multi-sample splice sampling method to obtain a text classification model whose model index reaches a target standard after training;
generating an end-to-end sensitive text recall model based on the model parameters of the text classification model whose model indicators reach a target standard, the end-to-end sensitive text recall model having acquired word list recall ability through training;
A method for training an end-to-end sensitive text recall model, including: performing an iterative training process on an initial text classification model based on the positive sample data and the negative sample data through a human evaluation scheme and a multi-sample splice sampling scheme;
Dividing the positive sample data and the negative sample data into a training set and a validation set as training samples;
training a text classification model based on the training set and the validation set to obtain an optimal model;
obtaining a test set and evaluating the optimal model based on the test set to obtain a model evaluation result;
updating the training samples according to the model evaluation results and the test set through human evaluation and multi-sample splice sampling;
Re-dividing the updated training samples into a training set and a validation set, and training a text classification model based on the training set and the validation set until the model index reaches a target standard after the training is completed, thereby obtaining an optimal model;
2. The method for training an end-to-end sensitive text recall model of claim 1, comprising: the test set includes a recall sample and a second random text corpus, and evaluating the best fit model based on the test set to obtain a model evaluation result;
inputting the recall sample of the test set into the optimal model and obtaining a first prediction result output from the optimal model;
determining a recall rate of the optimal model based on the first prediction result and actual label information corresponding to the recall sample;
inputting the second random text corpus of the test set into the best-fit model and obtaining a second prediction result output from the best-fit model;
determining a fit rate of the optimal model based on the second prediction result and actual label information corresponding to the second random text corpus;
The method for training an end-to-end sensitive text recall model of claim 2, comprising: updating the training samples according to the model evaluation results and the test set through a human evaluation scheme and a multi-sample splice sampling scheme;
In response to the recall rate being less than a first threshold, obtaining a first human evaluation result of the first prediction result of the examples that are predicted to be negative examples, and adding samples of the recall samples that are mispredicted to be negative examples to a sample set to be updated based on the first human evaluation result; and/or
in response to the precision being less than a second threshold, obtaining a second human evaluation of the examples predicted to be positive examples from the second prediction results, and obtaining text corpora mispredicted to be positive examples from the second random text corpus based on the second human evaluation;
4. The method of claim 3, further comprising: splicing every N samples in the sample set to be updated into one sample, and updating the sample obtained after the splicing process to the training sample, where N is an integer greater than 1. The method for training an end-to-end sensitive text recall model according to claim 4, wherein N is 3. the text classification model includes a first long short-term memory network (LSTM) layer, an average pooling layer, a second LSTM layer, a max pooling layer, a spliced concat layer, a reduced dropout layer, and a classification layer;
The first LSTM layer extracts text features of the samples;
the average pooling layer pools the text features to obtain first path features;
The second LSTM layer performs feature extraction on the output of the last hidden layer of the first LSTM layer, and inputs the extracted features to the max pooling layer;
the max pooling layer pools the output of the second LSTM layer to obtain second path features;
the splice concat layer splices the first path feature and the second path feature to obtain a splice feature;
the reduced Dropout layer performs a Dropout operation on the splice feature;
The method for training an end-to-end sensitive text recall model according to claim 1 , wherein the classification layer performs classification processing on the features output from the reduced Dropout layer to obtain a classification prediction value. obtaining a text to be processed;
predicting the target text based on a pre-trained end-to-end sensitive text recall model to determine whether the target text is recalled;
Including,
The end-to-end sensitive text recall model is trained to obtain word list recall capability, and the end-to-end sensitive text recall model is trained according to the method of claim 1 . predicting the target text based on the pre-trained end-to-end sensitive text recall model to determine whether to recall the target text,
extracting text features of the target text through a first L STM layer;
pooling the text features through an average pooling layer to obtain first path features;
performing feature extraction on the output of the last hidden layer of the first LSTM layer through a second LSTM layer, and inputting the extracted features into a max pooling layer;
pooling the output of the second LSTM layer through the max pooling layer to obtain second path features;
splicing the first path feature and the second path feature to obtain a splice feature, and performing a Dropout operation on the splice feature via the reduced Dropout layer;
processing the features of the output of the reduced Dropout layer through a classification layer to obtain a predicted classification value;
determining whether to recall the subject text based on the predicted value;
8. The method of claim 7, comprising: An acquisition module for acquiring a preset word list and a first random text corpus in a sensitive text block scene, wherein text corresponding to a term in the preset word list is sensitive text;
a construction module for constructing positive sample data based on the predefined word list, and a construction module for constructing negative sample data based on the first random text corpus;
According to the positive sample data and the negative sample data, an iterative training process is performed on the initial text classification model through a human evaluation method and a multi-sample splice sampling method, and a text classification model whose model index reaches a target standard after training is completed is obtained;
A processing module for generating an end-to-end sensitive text recall model based on the model parameters of the text classification model whose model indicators reach a target standard, wherein the end-to-end sensitive text recall model has a word list recall ability through learning;
An apparatus for training an end-to-end sensitive text recall model, comprising: an acquisition module for acquiring a text to be processed;
a prediction module for predicting the target text based on a pre-trained end-to-end sensitive text recall model to determine whether the target text is recalled;
Equipped with
The end-to-end sensitive text recall model is trained to obtain word list recall capability, and the end-to-end sensitive text recall model is trained according to the method of any one of claims 1 to 6. At least one processor;
a memory communicatively coupled to the at least one processor;
Equipped with
An electronic device comprising: an input/output section for inputting a first input/output signal to the at least one processor; and a second input/output section for outputting a first input/output signal to the at least one processor; and a memory for storing instructions executable by the at least one processor; the memory storing instructions executable by the at least one processor; A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising:
A non-transitory computer readable storage medium having computer instructions that cause a computer to perform the method of any one of claims 1 to 6 or the method of claims 7 or 8. A computer program which, when executed on a computer , causes the computer to carry out the method according to any one of claims 1 to 6 or the method according to claim 7 or 8.

Images