CN116894073A - Sensitive data identification method, device and storage medium - Google Patents

Abstract

This application discloses a sensitive data identification method, device and storage medium. The method specifically includes: the electronic device acquires a first data set and a second data set. The first data set includes N fields of sensitive data, and the second data set includes M fields of data to be detected; wherein the first data set is The pre-stored annotated sensitive data set, and the second data set is the collected unlabeled data set. The electronic device merges the data of fields with the same character type in the first data set and the second data set to obtain a third data set. The third data set includes data of S fields. The electronic device clusters the data in the third data set to obtain R-type data. The electronic device determines the distribution difference between the sensitive data and the data to be detected in the R-type data of the third data set. If the distribution difference is less than the first preset threshold, it determines that the data in the third data set is sensitive data. This method can improve the efficiency of sensitive data identification.

Description

Sensitive data identification method, device and storage medium

Technical field

The present invention relates to the field of computer technology, and in particular to a sensitive data identification method, device and storage medium.

Background technique

Structured data refers to data stored in databases, tabular text files (for example, excel, csv, etc.). Structured data can usually be identified by fields, and you can determine whether the data corresponding to the field is sensitive data through the fields of the data. For example, if a certain type of numerical data is stored in a bank's database, and the field of this data is "amount", it can be determined that the data in this field is the user's sensitive data. However, in some cases, there may be missing data fields, so it is necessary to judge whether it is sensitive data through the data itself. Currently, sensitive data identification is mainly done manually, which is less efficient.

Contents of the invention

This application provides a sensitive data identification method, device and storage medium to solve the problem of low efficiency in sensitive data identification.

In the first aspect, this application provides a sensitive data identification method. The method can be applied to electronic devices with processing capabilities. The method specifically includes: the electronic device acquires a first data set and a second data set. The first data set includes N fields of sensitive data, and the second data set includes M fields. The data to be detected; among them, the first data set is a pre-stored annotated sensitive data set, the second data set is a collected unlabeled data set, and M and N are both positive integers. The electronic device merges the data of fields with the same character type in the first data set and the second data set to obtain a third data set. The third data set includes data of S fields, and S is a positive integer. The electronic device clusters the data in the third data set to obtain R type data, where R is a positive integer. The electronic device determines the distribution difference between the sensitive data and the data to be detected in the R-type data of the third data set. If the distribution difference is less than the first preset threshold, it determines that the data in the third data set is sensitive data.

In this embodiment of the present application, by merging the M fields in the second data set with the same character type (for example, floating point numbers, letters, etc.) as the N fields in the first data set, the fields in the second data set can be merged. The fields are initially screened, and fields that are obviously not sensitive data are deleted to obtain the third data set. The electronic device further clusters the data in the third data set to obtain R-type data, and can determine whether the data in the third data set is based on the distribution difference between the sensitive data in the third data set and the data to be detected in the R-type data. Sensitive data. When identifying sensitive data, by merging the first data set and the second data set, data that is obviously not sensitive data can be deleted and the identification efficiency can be improved. The merged third data set is further clustered to obtain the distribution of sensitive data and data to be detected, and finally the identification results are obtained. It is more efficient to identify sensitive data through the distribution.

Optionally, the electronic device merges the data of the fields with the same character type in the first data set and the second data set, including: the electronic device merges the first field among the N fields of the first data set and the second data set. The second fields among the M fields are sampled respectively to obtain a plurality of first sampled data and a plurality of second sampled data. The electronic device determines whether character types of the plurality of first sampling data and the plurality of second sampling data are the same. If the character types are the same, the electronic device merges the data in the first field with the data in the second field, and the merged data is the data in any field in the third data set.

In the embodiment of this application, sampling is used to compare whether the character types of the two fields in the first data set and the second data set are the same. If they are the same, the data of the two fields are merged into the data of one field in the third data set. . The efficiency of merging can be improved through sampling.

Optionally, the electronic device determines whether the character types of the plurality of first sampling data and the plurality of second sampling data are the same, including: the electronic device determines statistical parameters of the plurality of first sampling data and the plurality of second sampling data respectively, and the statistics Parameters include mean and/or variance. The electronic device determines whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same based on whether the difference in statistical parameters of the plurality of first sampled data and the plurality of second sampled data is less than a second preset threshold.

In the embodiment of the present application, since the electronic device has a certain accuracy in determining the character type, when determining whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same, it can be determined based on the statistical parameters of the sampled data. If the difference in statistical parameters is less than the second preset threshold, it can be considered that the statistical distributions of the plurality of first sampling data and the plurality of second sampling data are approximately the same, and the distribution of the plurality of first sampling data and the plurality of second sampling data can be determined. Character types are the same.

Optionally, the electronic device determines the distribution difference between the sensitive data in the third data set and the data to be detected in the R-type data, including: the electronic device determines the first set based on the proportion of the sensitive data in each type of data in the R-type data. , the first set is used to characterize the distribution of sensitive data in R type data. The electronic device determines the second set according to the proportion of each type of data in the R type data of the data to be detected, and the second set is used to characterize the distribution of the data to be detected in the R type data. The electronic device determines that the Euclidean distance between the first set and the second set is the distribution difference between the sensitive data and the data to be detected in the R-type data.

In this embodiment of the present application, by clustering the data in the third data set, a first set corresponding to sensitive data and a second set of data to be detected are obtained. Whether the data to be detected in the third data set is sensitive data can be determined based on the Euclidean distance between the first set and the second set, which is more efficient.

Optionally, before the electronic device clusters the data in the third data set, the method further includes: the electronic device converts the character data in the third data set into numerical data.

In this embodiment of the present application, converting the character data in the third data set into numerical data can improve the processing efficiency of the electronic device.

In a second aspect, this application provides a sensitive data identification device. The device includes: an acquisition module, a merging module, a clustering module and a determining module. Wherein, the acquisition module is used to acquire a first data set and a second data set. The first data set includes N fields of sensitive data, and the second data set includes M fields of data to be detected; wherein, the first data set is pre- The stored labeled sensitive data set, the second data set is the collected unlabeled data set, M and N are both positive integers. The merging module is used to merge the data of fields with the same character type in the first data set and the second data set to obtain a third data set. The third data set includes data of S fields, and S is a positive integer. The clustering module is used to cluster the data in the third data set to obtain R type data, where R is a positive integer. The determination module is used to determine the distribution difference between the sensitive data and the data to be detected in the R-type data of the third data set. If the distribution difference is less than the first preset threshold, it is determined that the data in the third data set is sensitive data.

Optionally, the merging module is specifically configured to: respectively sample the first field among the N fields of the first data set and the second field among the M fields of the second data set to obtain multiple first sampled data and Multiple second sampled data. Determine whether the character types of the plurality of first sampling data and the plurality of second sampling data are the same. If the character types are the same, the data in the first field and the data in the second field are merged, and the merged data is the data in any field in the third data set.

Optionally, the merging module is specifically configured to determine statistical parameters of multiple first sampling data and multiple second sampling data respectively, where the statistical parameters include mean and/or variance. Whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same is determined based on whether the difference in statistical parameters of the plurality of first sampled data and the plurality of second sampled data is less than a second preset threshold.

Optionally, the determination module is specifically configured to: determine the first set according to the proportion of sensitive data in each type of data in the R type data, and the first set is used to characterize the distribution of sensitive data in the R type data. The second set is determined according to the proportion of each type of data in the R type data of the data to be detected, and the second set is used to characterize the distribution of the data to be detected in the R type data. The Euclidean distance between the first set and the second set is determined to be the distribution difference between the sensitive data and the data to be detected in the R-type data.

Optionally, the clustering module is also used to convert character data in the third data set into numerical data.

In a third aspect, embodiments of the present application provide an electronic device, which includes a processor and a memory communicatively connected to the processor. Wherein, the memory stores computer execution instructions, and the instructions are executed by the processor, so that the processor can execute the method described in any one of the above first aspects.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium that stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor is caused to execute the above-described first aspect. any of the methods described.

In a fifth aspect, embodiments of the present application provide a computer program product. The computer program product includes a computer program, which is stored in a computer-readable storage medium. The processor can read the computer program from the computer-readable storage medium, and the processor executes the computer program. The method described in any one of the above first aspects can be implemented as a computer program.

Description of the drawings

Figure 1 is a schematic flow chart of a sensitive data identification method provided by an embodiment of the present application;

Figure 2 is a schematic structural diagram of a sensitive data identification device provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to better understand the solutions provided by the embodiments of the present invention, some technical concepts involved in the embodiments of the present invention are first introduced. It should be noted that in the technical solution of this application, the collection, dissemination, and use of data are all in compliance with relevant national laws and regulations.

As mentioned above, the structured data in the database table can be identified by the fields of the data to determine whether the data is sensitive data. However, in some cases there may be missing data fields, which need to be identified through the data itself. Let’s take the following tables 1 and 2 as an example for explanation.

Table 1

Table 2

Table 1 above shows the storage situation of data in the database table under normal circumstances. It can be seen that the data fields in Table 1 are clear, and you can know whether the data is sensitive data based on the data fields. For example, the "name" field includes the user's name, and there is sensitive data "Hu_Jingtao" (name of the head of state). For another example, the data under the "amount" field is sensitive data because it involves user privacy. You can quickly know whether the data is sensitive data through the fields corresponding to the data. As shown in Table 2, when a data field is missing (the missing field is represented by "?"), it is impossible to directly determine whether the data is sensitive data. For example, the meanings represented by the data "4256.27", "54212.65", and "1684.74" in Table 2 cannot be determined intuitively.

The simpler way is to use manual methods to identify one by one, but manual methods have low identification efficiency when the amount of data is large. There are currently some machine learning algorithms for identifying sensitive data, but these machine learning algorithms need to input the data to be identified into the model for identification one by one, and the identification efficiency is also low.

In view of this, embodiments of the present application provide a sensitive data identification method. By merging the data to be detected and the sensitive data, and clustering the merged data, the distribution of sensitive data in the data domain to be detected can be determined based on the clustering. The difference determines whether the data to be detected is sensitive data. Since sensitive data is identified based on the distribution of data, the identification efficiency is high.

Please refer to Figure 1, which shows a schematic flow chart of the sensitive data identification method provided by the embodiment of the present application. The process shown in Figure 1 takes electronic equipment performing sensitive data identification as an example. The electronic device can be any device with data storage function. For example, the electronic device can be a terminal, such as a smart phone, a tablet computer, a desktop computer, etc.

For the convenience of description, in the following introduction, the data set with clear fields can be distinguished from the data set with missing fields, and the data set with clear fields is called the first data set, as shown in Table 1. The data set with missing fields is called the second data set, as shown in Table 2. That is, the data in the second data set needs to be identified. It should be understood that the "first" and "second" mentioned here are only used to distinguish the data sets and are not used to limit the size, content, order, timing, priority or importance of the data sets.

S101. The electronic device obtains a first data set and a second data set. The first data set includes N fields of sensitive data, and the second data set includes M fields of data to be detected. The first data set is pre-stored. The labeled sensitive data set, the second data set is the collected unlabeled data set, M and N are both positive integers.

The electronic device acquires the first data set and the second data set. Among them, the data in the first data set is sensitive data with clearly identified fields. The first data set may be a data set pre-collected by the electronic device. The electronic device can manually identify the pre-collected data set and mark the sensitive data fields of the data set. After the electronic device labels the first data set, it can obtain N fields of sensitive data, where N is a positive integer. The electronic device can pre-store the annotated first data set.

The second data set is a data set collected by electronic devices but not labeled. Some or all of the fields in the second data set may be missing. Therefore, the data in the second data set may contain both sensitive data and non-sensitive data, and the sensitive data in the second data set needs to be identified. For ease of expression, the data in the second data set is called data to be detected. The number of fields in the second data set is M, that is to say, the second data set includes M fields of data to be detected.

S102. The electronic device merges the data of fields with the same character type in the first data set and the second data set to obtain a third data set. The third data set includes data of S fields, and S is a positive integer.

In this embodiment of the present application, since the efficiency of identifying itself directly through the data in the second data set is low, considering that the first data set is known annotated sensitive data, the first data set can be used to identify itself. Identification of the second data set. The first data set includes N fields of data, the second data set includes M fields of data to be detected, the N fields of data in the first data set are all sensitive data, and the M fields of the second data set are to be detected. Not all detection data is sensitive. The character type of the data in the second data set may be completely different from the data in the first data set, or it may be the same as the data character type in the first data set, but is not sensitive data. Therefore, when the electronic device identifies the data to be detected in the second data set, it can ignore fields that are obviously not sensitive data, thereby reducing the amount of data to be detected that needs to be identified and improving detection efficiency. For example, if there is data "10:34", "09:42", "16:23"... under an unlabeled field in the second data set, you can judge that the field is "time" from the character type of the data. , the data under this field is non-sensitive data. For another example, the second data set summarizes the data "SAFGJAIOJIAJDO", "OPQJTEQIOTAD", "POQTEUPODFDAF"... under an unlabeled field. The meaning of the data under this field is obviously not sensitive data.

In order to identify whether the data character type in the second data set is the same as the character type in the first data set, the electronic device may compare a certain field (for example, the first field) of the N fields in the first data set with the second one through sampling. One field (for example, the second field) among the M fields of the data set is compared one by one.

The electronic device may sample the first data set to obtain a plurality of first sampling data, and sample the second data set to obtain a plurality of second sampling data. If the fields to which the plurality of second sampled data collected by the electronic device belong are not sensitive data fields, their character types may be completely different from the plurality of first sampled data. For example, the character types of the plurality of second sampling data are English character strings, such as "SAFGJAIOJIAJDO", "OPQJTEQIOTAD", etc., and the character types of the plurality of first sampling data are floating point numbers, such as "4256.27", "54212.65", etc. Then it can be determined that the character types of the plurality of first sampled data and the plurality of second sampled data are different. It should be understood that the quantity of sampling data should be within a reasonable range. If the quantity is too small, the data collected may not be representative. Too much quantity may affect the processing efficiency of the electronic equipment. The specific quantity can be determined according to actual needs. This application The examples are not specifically limited.

Specifically, the electronic device can determine whether the character types of the plurality of first sampling data and the plurality of second sampling data are the same through the statistical parameters of the plurality of first sampling data and the plurality of second sampling data. The statistical parameters include mean and/or variance. That is to say, the electronic device can calculate the mean and/or variance of multiple first sampling data and the mean and/or variance of multiple second sampling data. Since statistical parameters can characterize the distribution of data, if multiple first sampling data are used, The statistical parameters of the sampled data and the plurality of second sampled data can determine that the more similar the distributions of the two are, the higher the probability that the character types of the two are the same. For example, if multiple first sampling data and multiple second sampling data are both "height", then the mean and/or variance of multiple first sampling data, the mean and/or variance of multiple second sampling data calculated by the electronic device Or the variances should be similar and both are within the height range of normal people, then the electronic device can determine that the character types of the two are the same. For another example, if the plurality of first sampling data and the plurality of second sampling data are all letter-type data "names", the electronic device can vectorize the plurality of first sampling data and the plurality of second sampling data through word2vec. , the mean and/or variance of the vectorized plurality of first sampled data and the mean and/or variance of the plurality of second sampled data may be calculated. If the two are similar, the electronic device may determine that the character types of the two are the same. On the other hand, if the character type of the plurality of first sampling data is a floating point number and the character type of the plurality of second sampling data is a string of English letters, then the statistics of the plurality of first sampling data and the plurality of second sampling data The parameters are obviously different, and the electronic device can determine that the character types of the two are different.

Since the statistical parameters can characterize the degree of approximation of the distribution of multiple first sampling data and multiple second sampling data, that is, there may be differences in the distributions of multiple first sampling data and multiple second sampling data. The electronic device can set a second preset threshold for the statistical parameters of the plurality of first sampling data and the plurality of second sampling data. When the difference in the statistical parameters of the two is less than the second preset threshold, it means that the distribution of the two is sufficient. Similar, it can be considered that the character types of the two are the same. The second preset threshold is a preset threshold for the mean and/or variance of the sampled data. As an example, the second preset threshold for the mean may be 2, and the second preset threshold for the variance may be 3. That is to say, if the difference between the means of the plurality of first sampling data and the plurality of second sampling data is less than 2, and the difference in the variance is less than 3, it can be considered that the characters of the plurality of first sampling data and the plurality of second sampling data Same type. Correspondingly, if the difference between the means of the plurality of first sampled data and the plurality of second sampled data is greater than 2 or the difference of the variances is less than 3, it can be considered that the character types of the plurality of first sampled data and the plurality of second sampled data are not the same. Similarly, the data corresponding to the second field can be ignored.

The electronic device can compare each field of the second data set with each field of the first data set one by one, and merge the fields corresponding to data with the same character type. It should be noted that when the electronic device determines whether the character types are the same, it determines by sampling multiple first sampling data and multiple second sampling data. When merging, it determines whether the first sampling data corresponds to The data in the first field and the data in the second field corresponding to the second sampled data are merged. That is to say, it is judged through sampling whether the data character types of the two fields are the same, and when merging, all the data in the two fields are merged. The electronic device merges the data in the first field with the data in the second field, and the merged data is the data in any field in the third data set. The electronic device merges the data of fields with the same character type among the N fields in the first data set and the M fields in the second data set to obtain a third data set. The number of fields in the third data set can be recorded as S, S is a positive integer. It should be understood that not all data in the third data set is sensitive data, and further judgment is required on the data in the third data set.

S103. The electronic device clusters the data in the third data set to obtain R type data, where R is a positive integer.

The electronic device may use a density-based spatial clustering of applications of noise (DBSCAN) algorithm to cluster the data in the third data set. In order to facilitate the understanding of this scheme, the DBSCAN algorithm is briefly introduced next.

The DBSCAN algorithm is a density-based clustering algorithm that generally assumes that categories can be determined by the tightness of the sample distribution. The DBSCAN algorithm requires the selection of a distance metric. For the data set to be clustered, the distance between any two points reflects the density between the points. The distance can be used to determine whether the points can be clustered into the same category. That is to say, if the points are If the distance between them meets certain conditions, it can be considered that the two points are closely connected and can be classified into the same category. The DBSCAN algorithm requires the user to input two parameters: one parameter is the radius (EPS), which represents the range of the circular field centered on a given point P. Another parameter is the minimum number of points (minpts) in the domain centered on point P. If the condition is met: the number of points in the area with point P as the center and radius EPS is not less than minpts, then point P is called the core point. You can continue to search for points within the area with a radius of EPS through point P until the above conditions cannot be met. Then you can group the found points into the same category, that is, a cluster.

The electronic device can calculate the distance between samples in the third data set through the DBSCAN algorithm. By setting the parameter radius EPS, the closely connected samples in the third data set can be determined, and the closely connected samples can be classified into one category, thus obtaining a Cluster categories. By classifying all groups of closely connected samples into different categories, we obtain the final results of all clustering categories. The electronic device can cluster the third data set through a clustering algorithm to obtain R clusters, that is, R categories. Then it can be considered that R type data is obtained after clustering the third data set. It should be noted that before the electronic device clusters the data in the third data set, the character data in the third data set can be converted into numerical data, which can improve the efficiency of the electronic device in clustering the third data set. .

S104. The electronic device determines the distribution difference between the sensitive data and the data to be detected in the R-type data of the third data set. If the distribution difference is less than the first preset threshold, it determines that the data in the third data set is sensitive data.

The data of one field among the S fields of the third data set is only data of the same character type. However, the character types are the same, and the parameters represented by them are not exactly the same. That is to say, the non-sensitive data in the second data set may be merged in the third data set, and eventually the data in one of the S fields in the third data set may contain more non-sensitive data. For example, in the aforementioned step S102, when merging the first data set and the second data set, the floating-point numbers whose fields in the first data set are of the same character type as those in the second data set are merged, and the third data set obtained is The data in one of the S fields of the data set may include "amount" or "height", "weight" and other data in the first data set and the second data set. Therefore, the data in the third data set can be further identified through the R-type data obtained by clustering the third data set.

Since the third data set includes both the sensitive data in the first data set and the data to be detected in the second data set, after clustering the data in the third data set, the sensitive data in the third data set and the data to be detected are in R Distributions in class data may differ. For example, if the fields of non-sensitive data in the second data set are merged into the data to be detected, it may result in more non-sensitive data in the data to be detected. Non-sensitive data will generate a clustering category that is far away from other data during clustering, that is, a relatively independent cluster will be generated, which will lead to a large difference in the distribution of sensitive data and data to be detected. Therefore, the electronic device can determine the distribution difference between the sensitive data and the data to be detected in the R-type data of the third data set. If the distribution difference is too large, it can be considered that the data to be detected includes non-sensitive fields, and not all the data to be detected is is sensitive data.

Since the electronic device clusters the third data set to generate R-type data, the distribution of this type of data can be determined based on the proportion of each type of data in the R-type data. Data proportions can be expressed as fractions or percentages. Taking the proportion as a score as an example, the electronic device can obtain R scores representing the proportion, and the R proportional scores can be represented by (n1, n2...nr). This set of R fractions representing proportions can be used to represent the distribution of data in R-type data. It should be understood that the sum of each element of the set should be 1, that is, Σni=1.

The electronic device can determine the first set based on the proportion of sensitive data in each type of data in the R type data to represent the distribution of sensitive data in the R type data. The first set can be used (a1, a2...ar) express. Similarly, the electronic device can determine the second set according to the proportion of the data to be detected in each type of data in the R type data to characterize the distribution of the data to be detected in the R type data. The second set can be used as (b1 ,b2...br) means. The electronic device can determine the distribution difference between the sensitive data and the data to be detected in the R-type data based on the Euclidean distance between the first set and the second set. The electronic device may set a first preset threshold to determine whether the distribution difference between the first set and the second set is too large. For example, the electronic device may set the first preset threshold to 4. When the distribution difference between the first set and the second set is greater than or equal to the first preset threshold, it can be considered that the difference between the data to be detected and the sensitive data is large, and the data to be detected can be considered not to be sensitive data. Correspondingly, when the distribution difference between the first set and the second set is less than the first preset threshold, the difference between the data to be detected and the sensitive data can be considered to be small enough, and the data to be detected can be considered to be sensitive data. It should be understood that the embodiments of the present application do not limit the specific value of the first preset threshold. The electronic device can set the first preset threshold small enough to ensure that the distribution of the data to be detected and the sensitive data are as consistent as possible. This can ensure the accuracy of identifying the data to be detected as sensitive data.

Referring to Figure 2, based on the same inventive concept, an embodiment of the present application provides a sensitive data identification device 200. The device 200 includes: an acquisition module 201, a merging module 202, a clustering module 203 and a determining module 204. Among them, the acquisition module 201 is used to acquire a first data set and a second data set. The first data set includes N fields of sensitive data, and the second data set includes M fields of data to be detected; wherein, the first data set is The pre-stored labeled sensitive data set, the second data set is the collected unlabeled data set, M and N are both positive integers. The merging module 202 is used to merge data of fields with the same character type in the first data set and the second data set to obtain a third data set. The third data set includes data of S fields, where S is a positive integer. The clustering module 203 is used to cluster the data in the third data set to obtain R type data, where R is a positive integer. The determination module 204 is used to determine the distribution difference between the sensitive data and the data to be detected in the R-type data of the third data set. If the distribution difference is less than the first preset threshold, it is determined that the data in the third data set is sensitive data.

Optionally, the merging module 202 is specifically configured to: respectively sample the first field among the N fields of the first data set and the second field among the M fields of the second data set to obtain multiple first sampled data. and multiple second sampled data. Determine whether the character types of the plurality of first sampling data and the plurality of second sampling data are the same. If the character types are the same, the data in the first field and the data in the second field are merged, and the merged data is the data in any field in the third data set.

Optionally, the merging module 202 is specifically configured to determine statistical parameters of a plurality of first sampling data and a plurality of second sampling data respectively, where the statistical parameters include means and/or variances. Whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same is determined based on whether the difference in statistical parameters of the plurality of first sampled data and the plurality of second sampled data is less than a second preset threshold.

Optionally, the determination module 204 is specifically configured to: determine a first set according to the proportion of sensitive data in each type of data in the R type data, and the first set is used to characterize the distribution of sensitive data in the R type data. The second set is determined according to the proportion of each type of data in the R type data of the data to be detected, and the second set is used to characterize the distribution of the data to be detected in the R type data. The Euclidean distance between the first set and the second set is determined to be the distribution difference between the sensitive data and the data to be detected in the R-type data.

Optionally, the clustering module 203 is also used to convert character data in the third data set into numerical data.

Referring to Figure 3, based on the same inventive concept, an embodiment of the present application provides an electronic device 300. The electronic device includes: at least one processor 301, at least one memory 302, and computer program instructions stored in the memory. When the computer program When the instructions are executed by the processor, the sensitive data identification method as mentioned above is implemented.

Optionally, the processor 301 may be a central processing unit, an Application Specific Integrated Circuit (ASIC), one or more integrated circuits for controlling program execution, or an on-site computer. The hardware circuit developed by programmable gate array (English: Field Programmable GateArray, abbreviation: FPGA) can be a baseband processor.

Optionally, the read-write lock operating device also includes a memory 302 connected to at least one processor 301. The memory 302 may include a read-only memory (English: Read Only Memory, ROM for short), a random access memory (English: Random Access Memory (abbreviated as: RAM) and disk memory. The memory 302 is used to store data required when the processor 301 is running. The number of memories 302 is one or more. The memory 302 is also shown in FIG. 3 . However, it should be noted that the memory 302 is not a required functional module, so it is shown with a dotted line in FIG. 3 .

Based on the same inventive concept, embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores computer instructions. When the computer instructions are run on a computer, they cause the computer to execute the aforementioned sensitive data identification method.

In the specific implementation process, computer-readable storage media include: Universal Serial Bus flash drive (USB), mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory ( Various storage media that can store program code, such as Random Access Memory (RAM), magnetic disks, or optical disks.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above function allocation can be completed by different functional modules according to needs, that is, The internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working processes of the systems, devices and units described above, reference can be made to the corresponding processes in the foregoing method embodiments, which will not be described again here.

In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be The combination can either be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the application. The aforementioned storage media include: Universal Serial Bus flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk Or various media such as CDs that can store program code.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the scope of the present invention. In this way, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims (10)

1. A sensitive data identification method, characterized by including: Obtain a first data set and a second data set, the first data set includes N fields of sensitive data, and the second data set includes M fields of data to be detected; wherein the first data set is pre- The stored labeled sensitive data set, the second data set is the collected unlabeled data set, M and N are both positive integers; Merge the data of fields with the same character type in the first data set and the second data set to obtain a third data set, where the third data set includes data of S fields, where S is a positive integer; Cluster the data in the third data set to obtain R type data, where R is a positive integer; Determine the distribution difference between the sensitive data and the data to be detected in the R-type data of the third data set. If the distribution difference is less than a first preset threshold, determine that the data in the third data set is sensitive. data. 2. The method of claim 1, wherein merging data of fields with the same character type in the first data set and the second data set includes: The first field among the N fields of the first data set and the second field among the M fields of the second data set are respectively sampled to obtain multiple first sampled data and multiple second sampled data. ; Determine whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same; If the character types are the same, the data in the first field and the data in the second field are merged, and the merged data is the data in any field in the third data set. 3. The method of claim 2, wherein determining whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same includes: Determine statistical parameters of the plurality of first sampling data and the plurality of second sampling data respectively, where the statistical parameters include mean and/or variance; Determine whether the character type of the plurality of first sampled data and the plurality of second sampled data is based on whether the difference in statistical parameters of the plurality of first sampled data and the plurality of second sampled data is less than a second preset threshold. same. 4. The method of claim 1, wherein determining the distribution difference between the sensitive data of the third data set and the data to be detected in the R-type data includes: Determine a first set according to the proportion of the sensitive data in each type of data in the R-type data, and the first set is used to characterize the distribution of the sensitive data in the R-type data; Determine a second set according to the proportion of the data to be detected in each type of data in the R type data, and the second set is used to characterize the distribution of the data to be detected in the R type data; The Euclidean distance between the first set and the second set is determined to be the distribution difference between the sensitive data and the data to be detected in the R-type data. 5. The method of claim 2, wherein before clustering the data in the third data set, the method further includes: Convert character data in the third data set into numerical data. 6. A sensitive data identification device, characterized by including: An acquisition module is used to acquire a first data set and a second data set. The first data set includes N fields of sensitive data, and the second data set includes M fields of data to be detected; wherein, the first data set includes N fields of sensitive data. The first data set is a pre-stored labeled sensitive data set, the second data set is a collected unlabeled data set, and M and N are both positive integers; A merging module, configured to merge the data of fields with the same character type in the first data set and the second data set to obtain a third data set, where the third data set includes data of S fields, where S is positive integer; A clustering module, used to cluster the data in the third data set to obtain R type data, where R is a positive integer; Determining module, used to determine the distribution difference between the sensitive data and the data to be detected in the R-type data of the third data set, and if the distribution difference is less than a first preset threshold, determine the third data Centralized data is sensitive data. 7. The device according to claim 6, wherein the merging module is specifically used to: The first field among the N fields of the first data set and the second field among the M fields of the second data set are respectively sampled to obtain multiple first sampled data and multiple second sampled data. ; Determine whether the character types of the plurality of first sampled data and the plurality of second sampled data are the same; If the character types are the same, the data in the first field and the data in the second field are merged, and the merged data is the data in any field in the third data set. 8. The device according to claim 7, wherein the merging module is specifically used to: Determine statistical parameters of the plurality of first sampling data and the plurality of second sampling data respectively, where the statistical parameters include mean and/or variance; Determine whether the character type of the plurality of first sampled data and the plurality of second sampled data is based on whether the difference in statistical parameters of the plurality of first sampled data and the plurality of second sampled data is less than a second preset threshold. same. 9. An electronic device, characterized by comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer execution instructions; The processor executes computer-executable instructions stored in the memory to implement the method according to any one of claims 1-5. 10. A computer-readable storage medium, characterized in that computer-executable instructions are stored in the computer-readable storage medium, and when executed by a processor, the computer-executable instructions are used to implement any one of claims 1-5. method described in the item.