CN116796170A - Data processing method, electronic device and medium - Google Patents

Abstract

The invention provides a data processing method, electronic equipment and media, and relates to technical fields such as big data and deep learning. The specific implementation plan includes: obtaining multiple object data; determining that the number of multiple object data is greater than a preset quantity threshold, and for any candidate object data in the multiple object data, based on the feature vector of the candidate object data and the first distance threshold, Determine neighboring object data associated with the candidate object data from a plurality of object data; in response to the presence of neighboring object data in the plurality of object data, generate first candidate representation data based on the candidate object data and the neighboring object data; based on the candidate object data, adjacent object data and first candidate representation data, process the plurality of object data to obtain a first representation data set; and determine that the number of first representation data contained in the first representation data set is less than or equal to a preset quantity threshold, The first representation data set is determined as the first sample data set.

Description

Data processing methods, electronic equipment and media

Technical field

The present invention relates to the technical field of data processing, specifically to the technical fields of big data, artificial intelligence, deep learning, etc., and in particular to a data processing method and device, a training method and device for a target object detection model, and a target object detection method. , devices, electronic equipment, storage media and computer program products.

Background technique

In fields such as deep learning and machine learning, it is usually necessary to use a large amount of sample data for model training. However, when the amount of sample data is too large, it will not only affect the training efficiency of the model, but also lead to the accumulation of local noise points, thus affecting the precision and accuracy of the model.

Contents of the invention

The invention provides a data processing method and device, a training method and device for a target object detection model, a target object detection method and device, electronic equipment, storage media and computer program products.

According to one aspect of the present invention, a data processing method is provided, including: acquiring multiple object data; determining that the number of multiple object data is greater than a preset quantity threshold, and targeting any candidate object data among the multiple object data, according to the feature vector of the candidate object data and the first distance threshold, determining neighboring object data associated with the candidate object data from the plurality of object data; in response to the presence of neighboring object data in the plurality of object data, determining based on the candidate object data and the neighboring object data to generate first candidate representation data; wherein the first candidate representation data is used to replace the candidate object data and neighboring object data; based on the candidate object data, neighboring object data and the first candidate representation data, multiple object data are processed, Obtain a first characterization data set; and determine that the number of first characterization data contained in the first characterization data set is less than or equal to a preset quantity threshold, and determine the first characterization data set as the first sample data set.

According to an embodiment of the present invention, processing the plurality of object data based on the candidate object data, the neighboring object data and the first candidate representation data to obtain the first representation data set includes: deleting the candidate object data and neighboring objects in the plurality of object data. object data to obtain a candidate object data set; and determining a first representation data set according to the candidate object data set and the first candidate representation data.

According to an embodiment of the present invention, the first characterization data set includes a plurality of first characterization data; in response to detecting that the candidate object data set does not include object data and determining that the number of the plurality of first characterization data is greater than a preset quantity threshold, the data The processing method further includes: determining the point density of each first representation data; determining the first representation data with the largest point density among the plurality of first representation data as the candidate first representation data; based on the point density of the candidate first representation data, Determine the number of second candidate representation data associated with the candidate first representation data; based on the number of second candidate representation data, perform representation data extraction on the plurality of first representation data to obtain a second representation data set; and determine the second candidate representation data If the number of second characterization data contained in the characterization data set is less than or equal to the preset quantity threshold, the second characterization data set is determined as the second sample data set.

According to an embodiment of the present invention, based on the number of second candidate characterization data, characterization data is extracted for a plurality of first characterization data, and the second characterization data set obtained includes: according to the feature vector of the candidate first characterization data and the second distance threshold , determine the neighboring representation data associated with the candidate first representation data from the plurality of first representation data; based on the number of the second candidate representation data, cluster the candidate first representation data and the neighboring representation data to obtain a number of second candidate representation data; deleting candidate first representation data and adjacent representation data among the plurality of first representation data to obtain a candidate representation data set; and determining the second representation according to the candidate representation data set and the number of second candidate representation data data set.

According to an embodiment of the present invention, in response to determining that the number of second characterization data contained in the second characterization data set is greater than the preset quantity threshold, the data processing method further includes: determining that the plurality of first characterization data are related to the candidate first characterization data The affected first representation data associated with each other; respectively determine the point density of the affected first representation data and the point density of the number of second candidate representation data; according to the point density of the affected first representation data, the number of second candidate representation data The point density of the candidate representation data and the point density of other representation data in the candidate representation data set except the affected first representation data are determined to determine the second representation data with the largest point density in the second representation data set as the candidate second representation data ; and updating the point density of the candidate first representation data based on the point density of the candidate second representation data, and repeatedly performing the operation of determining whether the number of second representation data contained in the second representation data set is less than or equal to a preset quantity threshold.

According to an embodiment of the present invention, determining the first characterization data with the largest point density among the plurality of first characterization data as the candidate first characterization data includes: based on the corresponding point densities of the plurality of first characterization data, determining the plurality of first characterization data. The characterization data is sorted to obtain a characterization data sequence; based on computing resources, the characterization data sequence is split into multiple first characterization data sub-sequences; wherein each first characterization data sub-sequence includes a preset number of first characterization data; For each first characterization data subsequence, determine the first characterization data with the highest point density in a preset number of first characterization data as the first subsequence characterization data; and determine the midpoint densities of multiple first subsequence characterization data. The largest first subsequence representation data is used as the candidate first representation data.

According to an embodiment of the present invention, in response to determining that the number of second characterization data contained in the second characterization data set is greater than the preset quantity threshold, the data processing method further includes: determining that the plurality of first characterization data are related to the candidate first characterization data associated affected first representation data; store the affected first representation data and a number of second candidate representation data into a sequence of representation data to be sorted; for each first representation data subsequence, delete the first representation data The characterization data related to the candidate first characterization data and adjacent characterization data in the sub-sequence and the affected first characterization data are obtained to obtain the second characterization data sub-sequence; and the values of the affected first characterization data in the to-be-sorted characterization data sequence are determined respectively. Point density and the point density of the number of second candidate representation data; according to the point density of the affected first representation data and the point density of the number of second candidate representation data in the representation data sequence to be sorted, and a plurality of second representation data According to the point density of the characterization data in the sub-sequence, the affected first characterization data and the number of second candidate characterization data in the characterization data sequence to be sorted are allocated to multiple second characterization data sub-sequences to obtain multiple third characterization data. subsequence; for each third representation data subsequence, determine the representation data with the highest point density in the third representation data subsequence as the second subsequence representation data; determine the representation data with the highest point density among multiple second subsequence representation data the second subsequence characterization data as the candidate second characterization data; and updating the point density of the candidate first characterization data based on the point density of the candidate second characterization data, repeatedly performing the step of determining the second characterization data contained in the second characterization data set. Operation to determine whether the quantity is less than or equal to the preset quantity threshold.

According to an embodiment of the present invention, each object data includes an object data identifier; based on the number of second candidate characterization data, performing characterization data extraction on a plurality of first characterization data to obtain the second characterization data set further includes: for each third candidate characterization data two candidate representation data, determining the candidate first representation data and adjacent representation data associated with the second candidate representation data; determining the object data identifier and adjacent representation of the candidate first representation data according to the object data identifiers corresponding to the plurality of object data the object data identifier of the data; determining the object data identifier of the second candidate representation data according to the object data identifier of the candidate first representation data and the object data identifier of the adjacent representation data; and combining the object data identifier of the second candidate representation data with the second Candidate representation data is associated.

According to an embodiment of the present invention, determining the object data identifier of the second candidate representation data based on the object data identifier of the candidate first representation data and the object data identifier of adjacent representation data includes: determining the object data identifier of the candidate first representation data and the adjacent object data identifier. The number of object data identifiers belonging to the same data identifier category among the object data identifiers representing the data; and using the object data identifier with the largest number as the object data identifier of the second candidate representation data.

According to an embodiment of the present invention, the number of second candidate representation data is positively correlated with the point density of candidate first representation data.

According to an embodiment of the present invention, determining the point density of each first representation data includes: for each first representation data, determining from a plurality of first representation data based on the second distance threshold and the feature vector of the first representation data. the number of neighboring characterization data associated with the first characterization data; and determining the number of neighboring characterization data as a point density.

According to an embodiment of the present invention, the data processing method further includes: in response to determining that adjacent object data associated with the candidate object data does not exist in the plurality of object data, determining the candidate object data as the first candidate representation data.

According to an embodiment of the present invention, each object data includes an object data identifier; generating the first candidate representation data according to the candidate object data and adjacent object data further includes: determining the candidate object data according to the object data identifiers corresponding to the plurality of object data. the object data identifier of the candidate object data and the object data identifier of the adjacent object data; determine the object data identifier of the first candidate representation data according to the object data identifier of the candidate object data and the object data identifier of the adjacent object data; and assign the object of the first candidate representation data The data identifier is associated with the first candidate representation data.

According to an embodiment of the present invention, determining the object data identifier of the first candidate representation data according to the object data identifier of the candidate object data and the object data identifier of the adjacent object data includes: determining the object data identifier of the candidate object data and the object of the adjacent object data. The number of object data identifiers belonging to the same data identifier category in the data identifiers; and the object data identifier with the largest number as the first candidate object data identifier.

According to another aspect of the present invention, a training method for a target object detection model is provided, including: obtaining a sample data set; using the sample data set to iteratively train a deep learning model until the output result of the deep learning model meets the iteration stop condition Or the cumulative number of iterative training reaches a preset threshold, and the target object detection model is obtained; where the sample data set is obtained using the data processing method described above.

According to another aspect of the present invention, a target object detection method is provided, including: inputting data to be processed into a target object detection model to obtain detection results for the data to be processed; wherein the target object detection model uses the target object described above The object detection model is trained using the training method.

According to another aspect of the present invention, a data processing device is provided, including: a first acquisition module for acquiring multiple object data; a first determination module for determining that the number of multiple object data is greater than a preset quantity threshold , for any candidate object data among the plurality of object data, determine the adjacent object data associated with the candidate object data from the plurality of object data according to the feature vector of the candidate object data and the first distance threshold; the generation module uses In response to the presence of adjacent object data in the plurality of object data, generating first candidate representation data according to the candidate object data and the adjacent object data; wherein the first candidate representation data is used to replace the candidate object data and the adjacent object data; the processing module, for processing multiple object data based on the candidate object data, neighboring object data and the first candidate representation data to obtain a first representation data set; and a second determination module for determining the first representation data contained in the first representation data set If the number of characterization data is less than or equal to the preset quantity threshold, the first characterization data set is determined as the first sample data set.

According to another aspect of the present invention, a training device for a target object detection model is provided, including: a second acquisition module for acquiring a sample data set; and a training module for iteratively training a deep learning model using the sample data set. , until the output result of the deep learning model meets the iteration stop condition or the cumulative number of iterative training reaches the preset number threshold, the target object detection model is obtained; wherein, the sample data set is obtained by using the data processing device described above.

According to another aspect of the present invention, a target object detection device is provided, including: an input module for inputting data to be processed into a target object detection model to obtain detection results for the data to be processed; wherein the target object detection model is It is trained using the training device of the target object detection model described above.

According to another aspect of the present invention, an electronic device is provided, including: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are processed by the one or more When multiple processors execute, one or more processors are caused to execute the method described above.

According to another aspect of the present invention, a computer-readable storage medium is provided, on which executable instructions are stored. When the instructions are executed by a processor, they cause the processor to perform the method described above.

According to another aspect of the present invention, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method as described above.

Description of the drawings

In order to further illustrate the technical content of the present invention, the following will be described in detail with reference to examples and drawings, wherein:

Figure 1 is a schematic diagram of an exemplary system architecture in which data processing methods and devices, target object detection model training methods and devices, and target object detection methods and devices can be applied according to an embodiment of the present invention;

Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention;

Figure 3 is a schematic diagram of the process of generating a first sample data set according to an embodiment of the present invention;

Figure 4 is a schematic diagram of a process of generating a second representation data set according to an embodiment of the present invention;

Figure 5 is a schematic diagram of reordering the point density of each second characterization data in the second characterization data set according to an embodiment of the present invention;

Figure 6A is a schematic diagram of the effect of data classification using related technologies;

Figure 6B is a schematic diagram of the effect of data classification based on the technical solution of the present invention;

Figure 7 is a flow chart of a training method for a target object detection model according to an embodiment of the present invention;

Figure 8 is a flow chart of a target object detection method according to an embodiment of the present invention;

Figure 9 is a block diagram of a data processing device according to an embodiment of the present invention;

Figure 10 is a block diagram of a training device for a target object detection model according to an embodiment of the present invention;

Figure 11 is a block diagram of a target object detection device according to an embodiment of the present invention; and

Figure 12 is a block diagram of an electronic device used to implement the data processing method, the training method of the target object detection model, and the target object detection method according to the embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments and the drawings in the embodiments. Obviously, the described embodiments are only some of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

1 is a schematic diagram of an exemplary system architecture in which data processing methods and devices, target object detection model training methods and devices, and target object detection methods and devices can be applied according to embodiments of the present invention. It should be noted that Figure 1 is only an example of a system architecture to which embodiments of the present invention can be applied, to help those skilled in the art understand the technical content of the present invention, but does not mean that the embodiments of the present invention cannot be used in other applications. Device, system, environment or scenario.

As shown in Figure 1, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium used to provide communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired and/or wireless communication links, and the like.

Users can use terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, and 103. For example, target detection applications, web browser applications, search applications, instant messaging tools, email clients or social platform software, etc. (only examples).

The terminal devices 101, 102, and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

The server 105 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud computing, network services, and middleware services.

The server 105 may be a server that provides various services, such as a backend management server that provides support for websites browsed by users using the terminal devices 101, 102, and 103 (example only). The background management server can analyze and process the received user request and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device.

For example, the server 105 can obtain multiple object data from the terminal devices 101, 102, and 103 through the network 104. When it is determined that the number of multiple object data is greater than the preset quantity threshold, the server 105 may process the multiple object data to generate a sample data set. The sample data contained in the sample data set can replace multiple object data.

In some embodiments, the server 105 can also use the above sample data set to train the deep learning model. After completing the training of the deep learning model, the server 105 may also use the trained deep learning model (eg, target object detection model) to detect target objects in the data to be processed. In some examples, the server 105 can also send the trained deep learning model (eg, target object detection model) to the terminal devices 101, 102, 103. In this way, the user can apply the target object detection model in the terminal device to perform target object detection.

It should be noted that the data processing method provided by the embodiment of the present invention can generally be executed by the server 105. Correspondingly, the data processing device provided by the embodiment of the present invention can generally be installed in the server 105. The data processing method provided by the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and/or the server 105. Correspondingly, the data processing apparatus provided by the embodiment of the present invention can also be arranged in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and/or the server 105.

It should be noted that the training method of the target object detection model provided by the embodiment of the present invention can generally be executed by the server 105 . Correspondingly, the training device for the target object detection model provided by the embodiment of the present invention can generally be installed in the server 105 . The training method of the target object detection model provided by the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and/or the server 105. Correspondingly, the training device for the target object detection model provided by the embodiment of the present invention can also be set up in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and/or the server 105.

It should be noted that the target object detection method provided by the embodiment of the present invention can generally be executed by the server 105. Correspondingly, the target object detection device provided by the embodiment of the present invention can generally be installed in the server 105 . The target object detection method provided by the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and/or the server 105. Correspondingly, the target object detection apparatus provided by the embodiment of the present invention can also be arranged in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and/or the server 105.

Alternatively, the target object detection method provided by the embodiment of the present invention can generally also be executed by the terminal device 101, 102, or 103. Correspondingly, the target object detection device provided by the embodiment of the present invention can also be provided in the terminal device 101, 102, or 103.

It should be understood that the number of terminal devices, networks and servers in Figure 1 is only illustrative. Depending on implementation needs, there can be any number of end devices, networks, and servers.

It should be noted that the sequence number of each operation in the following method is only used as a representation of the operation for the purpose of description, and should not be regarded as indicating the execution order of the respective operations. Unless explicitly stated, the methods need not be performed in exactly the order shown.

Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention.

As shown in FIG. 2 , the data processing method 200 includes operations S210 to S250.

In operation S210, a plurality of object data are obtained.

In operation S220, it is determined that the number of the plurality of object data is greater than the preset quantity threshold, and for any candidate object data in the plurality of object data, a determination is made from the plurality of object data according to the feature vector of the candidate object data and the first distance threshold. Neighboring object data associated with the candidate object data.

In operation S230, in response to the presence of neighboring object data in the plurality of object data, first candidate representation data is generated according to the candidate object data and the neighboring object data.

In operation S240, a plurality of object data are processed based on the candidate object data, neighboring object data and the first candidate representation data to obtain a first representation data set.

In operation S250, it is determined that the number of first characterization data contained in the first characterization data set is less than or equal to the preset quantity threshold, and the first characterization data set is determined as the first sample data set.

According to the embodiment of the present invention, the object data includes, but is not limited to, text, image, video, audio, or location information carrying identification data (where the identification data is associated with the location information) and other types of data, which can be selected according to actual needs. , no limitation is made here. In one example, the location information carrying identification data may include, for example, latitude and longitude information carrying administrative area code information. Of course, the present invention is not limited to this.

Multiple object data can be used, for example, to construct a sample data set for model training. Of course, the uses of multiple object data are not limited to this. For simplicity of explanation, the following uses multiple object data to construct a sample data set as an example.

According to an embodiment of the present invention, the preset quantity threshold is used to indicate an expected data quantity of sample data in the sample data set to be generated. The preset quantity threshold can be set according to actual conditions, and the present invention does not limit this.

In embodiments of the present invention, the number of multiple object data can be compared with a preset quantity threshold to determine whether further processing of the multiple object data is required to obtain a sample data set with a desired amount of data.

For example, if it is determined that the number of multiple object data is greater than the preset quantity threshold, that is, the number of multiple object data is greater than the expected data amount, it means that there is redundant object data in the multiple object data. At this time, multiple object data need to be processed to obtain a sample data set with the expected data volume. On the contrary, if it is determined that the number of multiple object data is less than or equal to the preset quantity threshold, it means that the number of multiple object data is appropriate. At this time, the multiple object data can not be processed, and the sample data can be obtained directly based on the multiple object data. set.

In the embodiment of the present invention, when it is determined that the number of multiple object data is greater than the preset quantity threshold, for any candidate object data among the multiple object data, it is determined according to the feature vector of the candidate object data and the first distance threshold. Whether there is adjacent object data associated with the candidate object data among the plurality of object data. The adjacent object data refers to the object data among multiple object data that have similar characteristics to the candidate object data.

It can be understood that for any two object data, if the characteristics of the two object data are more similar, the distance between them will be closer; and vice versa. Therefore, distances between object data may be utilized to determine neighboring object data associated with candidate object data.

For example, for any candidate object data, the distance between the feature vector of the candidate object data and the feature vector of another candidate object data is determined. The other candidate object data refers to any object data among the plurality of object data that is different from the candidate object data. If it is determined that the distance between the feature vector of the candidate object data and the feature vector of another candidate object data is less than or equal to the first distance threshold, the other candidate object data is determined as the neighboring object data associated with the candidate object data . On the contrary, if it is determined that the distance between the feature vector of the candidate object data and the feature vector of another candidate object data is greater than the first distance threshold, it is considered that the other candidate object data is not the neighboring object data associated with the candidate object data. Based on the above method, it can be determined whether there is adjacent object data associated with the candidate object data in the plurality of object data, and the number of adjacent object data associated with the candidate object data.

It should be noted that the first distance threshold described above may be one or more preset distance values, or one or more preset distance ranges. The specific distance threshold may be set according to the actual situation, and is not limited here.

In addition, when determining the distance between the feature vector of the candidate object data and the feature vector of another candidate object data, any one or more distance calculation methods in the related technology can be used, such as but not limited to Euclidean distance, Mahalanobis distance, Manhattan distance, etc. are not limited by the present invention.

In this embodiment of the present invention, if it is determined that at least one neighboring object data associated with the above-mentioned candidate object data exists in the plurality of object data, the first candidate representation data can be generated based on the candidate object data and the at least one neighboring object data.

Since each neighboring object data has similar characteristics to the candidate object data, at least one neighboring object data and the candidate object data may be regarded as object data of the same type. In this way, the first candidate representation data can be generated within a range including the candidate object data and at least one neighboring object data. The first candidate representation data is used to replace the candidate object data and the at least one neighboring object data. In other words, the first candidate representation data can be used to represent the candidate object data and at least one neighboring object data, which facilitates subsequent processing of multiple object data.

In one example, for example, a clustering center may be generated based on the candidate object data and at least one neighboring object data based on a clustering algorithm, and the clustering center may be determined as the first candidate representation data. In the embodiment of the present invention, an appropriate clustering algorithm can be selected according to actual needs to generate a clustering center, which is not limited here.

Next, the plurality of object data are processed based on the candidate object data, at least one neighboring object data associated with the candidate object data, and the first candidate representation data to obtain a first representation data set.

In this embodiment of the present invention, the first candidate representation data may represent candidate object data and at least one neighboring object data. Therefore, when processing multiple object data, the first candidate representation data can be used to replace the candidate object data and at least one neighboring object data in the multiple object data to obtain the first representation data set. The first characterization data set includes a plurality of first characterization data, and the plurality of first characterization data may represent data for a plurality of objects. Processing multiple object data based on the above method can achieve simplified processing of redundant data in multiple object data, thereby facilitating the acquisition of sample data with a desired amount of data.

Afterwards, it is determined whether the quantity of the first characterization data contained in the first characterization data set is less than or equal to the preset quantity threshold, that is, it is determined whether the quantity of the first characterization data is less than or equal to the expected data amount. If it is determined that the number of first characterization data contained in the first characterization data set is less than or equal to the preset quantity threshold, the first characterization data set is determined to be the first sample data set.

If it is determined that the number of first characterization data contained in the first characterization data set is greater than the preset quantity threshold, another candidate object data may be determined from the plurality of object data, and the above candidate object data may be updated using another candidate object data, Repeat the above operation of determining that the number of first characterization data contained in the first characterization data set is less than or equal to the preset quantity threshold until it is determined that the number of first characterization data contained in the first characterization data set reaches the preset quantity threshold; or in After processing all the object data, when it is determined that the number of first characterization data included in each acquired first characterization data set does not reach the preset quantity threshold, the execution of the above operations S220 to S250 is stopped.

According to an embodiment of the present invention, when it is determined that the number of multiple object data exceeds the expected data amount, first candidate representation data is generated by utilizing the candidate object data and adjacent object data associated with the candidate object data. Then, the first candidate representation data is used to replace the candidate object data and adjacent object data in the plurality of object data to obtain a first sample data set. As a result, it is possible to implement simplified processing of redundant data in multiple object data, thereby obtaining a first sample data set with a desired data amount.

Figure 3 is a schematic diagram of a process of generating a first sample data set according to an embodiment of the present invention. The process of generating the first sample data set is illustrated below with reference to FIG. 3 . In the embodiment of the present invention, each data can be shown in the form of data points. For example, the object data is video, picture, or audio, and the object data can be input into a trained neural network (such as Resnet, VGG, Transformer, etc.) Networks that can be used to process images (corresponding to BLSTM-RNN, WaveNet, GPT, etc. that can be used to process audio) are used to extract features to obtain the corresponding feature vectors. Data points or other forms (such as lines, surfaces, etc.) can be used to extract features. etc.) to represent the above feature vectors. It should be understood that the above representations are only for understanding the technical solutions described in the embodiments of the present invention, but the solutions of the present invention are not limited thereto.

As shown in 310 in Figure 3, the plurality of object data include, for example, object data _Pa , object data _Pb , object data _Pc , object data _Pd , object data _Pe , object data _Pf , object data _Pg , Object data _Ph , object data _Pj , object data _Pk , and object data _Pm .

When it is determined that the number of multiple object data is greater than the preset quantity threshold, for any candidate object data among the multiple object data, for example, the object data P _d can be determined as a candidate object data. According to the object data P _d With the feature vector and the first distance threshold, neighboring object data associated with the candidate object data may be determined from the plurality of object data. For example, all object data within the radius range with the object data P _d as the center and the first distance threshold as the radius can be determined as adjacent object data associated with the candidate object data (ie, the object data P _d ), that is, the object data P _a , object data _Pb , and object data _Pc are determined as adjacent object data associated with object data _Pd .

Afterwards, first candidate representation data is generated based on the candidate object data and all neighboring object data. As shown at 320 in Figure 3, for example, the object data _Pa , object data _Pb , object data _Pc , and object data _Pd can be clustered to generate a clustering center, and the clustering center can be The first candidate characterization data (i.e., ct ₁ shown in 320 ). The first candidate representation data ct ₁ may be used in place of the object data _Pa , object data _Pb , object data _Pc and object data _Pd .

Next, candidate object data and adjacent object data in multiple object data are deleted to obtain a candidate object data set. Then, a first representation data set is determined based on the candidate object data set and the first candidate representation data.

As shown in 320 and 330 in FIG. 3, for example, the object data _Pb and the adjacent object data associated with the object data _Pd (ie, the object data _Pa , the object data _Pb and the object data Pd) are deleted from the plurality of object data. _c ), obtain the candidate object data set. Afterwards, a first representation data set is obtained based on the candidate object data set and the first candidate representation data ct ₁ . The first representation data set includes first candidate representation data ct ₁ , object data Pe, object data P _f , object data P _g , object data _{Ph h ,} object data P _j , object data P _k , and object data P _m . Each data included in the first characterization data set is the first characterization data.

Next, it may be determined whether the number of first characterization data included in the first characterization data set is less than or equal to a preset quantity threshold. If so, the first representation data set is determined as the first sample data set. If not, another candidate object data is determined from the plurality of object data, and the process described above is used to continue processing the other candidate object data to obtain another first representation data set.

Assuming that the number of first characterization data contained in the above-mentioned first characterization data set is greater than the preset quantity threshold, as shown at 330 in Figure 3, the object data _Pg (example only) may also be determined as another candidate object data, And based on the feature vector of the object data _Pg and the first distance threshold, neighboring object data associated with another candidate object data (ie, object data _Pg ) is determined from the plurality of object data. For example, the adjacent object data associated with the object data _Pg includes the object data _Pf and the object data _Ph .

Next, another first candidate representation data is generated based on another candidate object data and all neighboring object data. As shown at 330 in Figure 3, for example, the object data _Pg , the object data _Pf and the object data _Ph can be clustered to generate another clustering center, and the other clustering center can be used as another third clustering center. A candidate characterization data (i.e., ct ₂ shown in 330). The first candidate representation data ct ₂ may be used in place of the object data P _g , the object data P _f and the object data Ph _h .

Next, another candidate object data and corresponding neighboring object data among the plurality of object data are deleted to obtain another candidate object data set. Then, another first representation data set is determined based on another candidate object data set and another first candidate representation data.

As shown at 330 and 340 in Figure 3, for example, the object data _Pg and the adjacent object data associated with the object data _Pg (ie, the object data _Pf and the object data _Ph ) are deleted from the plurality of object data to obtain another A candidate object data set. Afterwards, another first representation data set is obtained based on another candidate object data set and the first candidate representation data ct ₂ . Another first characterization data set includes first candidate characterization data ct ₁ , first candidate characterization data ct ₂ , object data Pe , object data P _j , object data P _k , and object data _{P m} . Each data included in another first characterization data set is the updated first characterization data.

Next, it may be determined whether the number of updated first representation data is less than or equal to a preset quantity threshold. If so, another first representation data set is determined as the first sample data set. If not, another candidate object data is determined from the plurality of object data, and the process described above is used to continue processing the other candidate object data to obtain another first representation data set.

Assuming that the quantity of the updated first representation data is still greater than the preset quantity threshold, another object data (including object data P _e , object data P _j , object data P _k , and object data P _m ) can also be determined from the remaining object data. Candidate data. For example, the object data _Pe may be determined as another candidate object data. Afterwards, neighboring object data associated with another candidate object data (ie, object data _Pe ) may be determined from the plurality of object data according to the feature vector of the object data _Pe and the first distance threshold. As shown at 340 in Figure 3, with the object data _Pe as the center and the first distance threshold as the radius, there is no adjacent object data associated with the object data _Pe . At this time, another candidate object data (ie, object data _Pe ) can be determined as another first candidate representation data, which is recorded as first candidate representation data ct ₃ .

Next, another candidate object data (ie, the object data P _e ) among the plurality of object data is deleted to obtain another candidate object data set. Afterwards, another first representation data set is determined based on another candidate object data set and the first candidate representation data ct ₃ . Another first representation data set includes first candidate representation data ct ₁ , first candidate representation data ct ₂ , first candidate representation data ct ₃ , and object data P _j , object data P _k , and object data P _m . Each data included in another first characterization data set is the updated first characterization data.

Next, it is determined whether the number of updated first characterization data contained in another first characterization data set is less than or equal to a preset quantity threshold. If so, another first representation data set is determined as the first sample data set. If not, determine another candidate object data from the remaining object data (including object data P _j , object data P _k , and object data P _m ), and use the process described above to continue processing the other candidate object data. ; Repeat the operation in this way until it is determined that the number of first characterization data contained in the first characterization data set reaches the preset quantity threshold, or when it is detected that another candidate object data set does not include object data, stop continuing to perform the above operations.

In the embodiment of the present invention, when it is determined that the number of multiple object data exceeds the expected data amount, first candidate representation data is generated by utilizing each candidate object data and corresponding neighboring object data. Then, the first candidate representation data is used to replace the candidate object data and adjacent object data in the plurality of object data to obtain a first sample data set. As a result, it is possible to implement simplified processing of redundant data in multiple object data, thereby obtaining a first sample data set with a desired data amount.

According to an embodiment of the present invention, each object data mentioned above may include an object data identifier. The object data identifier is used to indicate descriptive information associated with the object data. The description information includes, for example, but is not limited to the category to which the object data belongs, the target object contained in the object data and the category to which the target object belongs, or other information related to the object data, and so on.

In the above process of generating the first candidate representation data, the object data identifier corresponding to the first candidate representation data may also be determined according to the object data identifier of each object data.

In one example, for example, when it is determined that adjacent object data associated with the candidate object data exists in multiple object data, the object data identifier and adjacent object data of the candidate object data may be determined based on the object data identifiers corresponding to the multiple object data. The object data identifier of the object data.

Next, the object data identifier of the first candidate representation data is determined based on the object data identifier of the candidate object data and the object data identifier of the adjacent object data. For example, the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate object data and the object data identifiers of the adjacent object data may be determined. Then, the object data identifier with the largest number is used as the object data identifier of the first candidate representation data.

For example, the object data identifier of the candidate object data _Pd is object data identifier 1. The neighboring object data associated with the candidate object data _Pd includes object data _Pa , object data _Pb and object data _Pc . The object data identifiers corresponding to the object data P _a , object data P _b and object data P _c are respectively object data identifier 1 , object data identifier 2 and object data identifier 1 . The object data identifiers of the candidate object data P _d and the object data identifiers of adjacent object data are summarized to obtain one object data identifier 2 and three object data identifiers 1 . The object data identifier with the largest number, namely object data identifier 1, is used as the object data identifier of the first candidate representation data.

Afterwards, the object data identifier of the first candidate representation data is associated with the first candidate representation data. For example, the first candidate representation data can be stored in association with the object data identifier 1, and subsequently the object data identifier 1 can be used to describe the relevant information contained in the first candidate representation data.

In the embodiment of the present invention, the object data identifier of the first candidate representation data is determined by using the object data identifier of the candidate object data and the object data identifier of the adjacent object data. In this way, the first candidate representation data is used to replace the candidate object data and the adjacent object data. When selecting object data, the object data identifier of the first candidate representation data may also be used to represent the object data identifier corresponding to the candidate object data and adjacent object data.

In another example, for example, when it is determined that adjacent object data associated with the candidate object data does not exist in the plurality of object data, the object data identification of the candidate object data may be determined as the object data identification of the first candidate representation data.

For example, the object data identifier of the candidate object data P _e is object data identifier 3. If it is determined that there is no adjacent object data associated with the candidate object data _Pe in the plurality of object data, the candidate object data _Pe may be determined as the first candidate representation data, and the object data identification 3 may be determined as the first candidate representation data. The object data identifier.

In some embodiments, if it is detected that the candidate object data set does not include object data, and it is determined that the number of first characterization data contained in the first characterization data set is greater than the preset quantity threshold, the first characterization data can also be classified based on a clustering method. A plurality of first characterization data in the data set are processed to obtain a second sample data set with a desired data amount. This is illustrated below with reference to specific embodiments.

First, when it is detected that the candidate object data set does not include object data, and it is determined that the number of first characterization data included in the first characterization data set is greater than a preset quantity threshold, the point density of each first characterization data may be determined.

Exemplarily, for each first representation data, the number of adjacent representation data associated with the first representation data is determined from the plurality of first representation data based on the second distance threshold and the feature vector of the first representation data. Then, the number of adjacent characterization data is determined as the point density of the first characterization data.

In this embodiment of the present invention, adjacent characterization data is used to indicate first characterization data in the first characterization data set that has similar characteristics to the first characterization data. Based on the second distance threshold and the feature vector of the first representation data, the process of determining the neighboring representation data associated with the first representation data is similar to the above process of determining the neighboring object data, and will not be described again here.

It should be noted that the second distance threshold is different from the first distance threshold. The second distance threshold may be one or more preset distance values, or may be one or more preset distance ranges. The specific distance threshold may be set according to the actual situation, and is not limited here.

Next, the first characterization data with the highest point density among the plurality of first characterization data is determined as the candidate first characterization data.

For example, after determining the point densities corresponding to the plurality of first characterization data, the point densities corresponding to the plurality of first characterization data are sorted. Then, based on the point density sorting result, the first representation data with the largest point density among the plurality of first representation data is determined as the candidate first representation data.

Next, based on the point density of the candidate first representation data, the number of second candidate representation data associated with the candidate first representation data is determined.

In embodiments of the present invention, the second candidate representation data may be used to replace the candidate first representation data and adjacent representation data associated with the candidate first representation data. Subsequently, the second candidate representation data can be used to replace the candidate first representation data and the corresponding adjacent representation data to simplify the processing of redundant data in multiple first representation data, thereby obtaining second sample data with the desired data amount. set.

In order to enable the second candidate representation data to better replace the candidate first representation data and the corresponding adjacent representation data, in the embodiment of the present invention, the correlation with the candidate first representation data can be determined based on the point density of the candidate first representation data. The number of associated second candidate representation data.

It can be understood that if the point density of a certain first representation data is larger, it means that the number of adjacent representation data associated with it is greater. Correspondingly, more second candidate representation data are needed to characterize the candidate first representation data and corresponding adjacent representation data. On the contrary, if the point density of the first representation data is smaller, it means that the number of adjacent representation data associated with it is smaller. Correspondingly, less second candidate representation data may be used to characterize the candidate first representation data and the corresponding neighboring representation data. In other words, the number of second candidate representation data is positively correlated with the point density of candidate first representation data.

In one example, for example, the following formula (1) may be used to determine the number of second candidate characterization data.

In formula (1), N _cluster represents the number of second candidate representation data, N _density represents the point density of candidate first representation data, N _e represents the expected data volume of sample data, and N _retain represents the number of multiple first representation data. quantity.

Next, based on the number of second candidate characterization data, characterization data is extracted from the plurality of first characterization data to obtain a second characterization data set.

For example, adjacent characterization data associated with the candidate first characterization data may be determined from the plurality of first characterization data according to the feature vector of the candidate first characterization data and the second distance threshold. This process is similar to the above process of determining neighboring object data, and will not be described again here.

Afterwards, based on the number of second candidate representation data, clustering processing is performed on the candidate first representation data and adjacent representation data to obtain a number of second candidate representation data. For example, clustering algorithms in related technologies, such as but not limited to k-means clustering algorithm, can be used to perform clustering processing on the candidate first representation data and adjacent representation data to obtain a number of second candidate representation data. .

Afterwards, a number of second candidate representation data are used to replace the candidate first representation data and the corresponding adjacent representation data in the plurality of first representation data, so as to achieve simplified processing of redundant data in the plurality of first representation data. For example, candidate first representation data and adjacent representation data among the plurality of first representation data can be deleted to obtain a candidate representation data set. Then, a second representation data set is determined based on the candidate representation data set and the number of second candidate representation data. The second characterization data set includes first characterization data and a number of second candidate characterization data in addition to the candidate first characterization data and adjacent characterization data. Each data in the second characterization data set is the second characterization data.

Next, it is determined whether the number of second characterization data contained in the second characterization data set is less than or equal to a preset quantity threshold, that is, it is determined whether the number of second characterization data is less than or equal to the expected data amount. If it is determined that the amount of second characterization data contained in the second characterization data set is less than or equal to the preset quantity threshold, the second characterization data set is determined as the second sample data set.

If it is determined that the number of second characterization data contained in the second characterization data set is still greater than the preset quantity threshold, another candidate first characterization data may also be determined from the second characterization data set as the candidate second characterization data, and based on the candidate second Use the point density of the characterization data to update the point density of the candidate first characterization data, and repeat the above-mentioned operation of determining whether the number of second characterization data contained in the second characterization data set is less than or equal to the preset quantity threshold until the second characterization data is determined. The number of second representation data contained in the set is less than or equal to the preset quantity threshold, or the number of times of repeated operations reaches the predetermined number.

An example process of determining candidate second characterization data will be described below with reference to specific embodiments.

It can be understood that the second characterization data set is obtained by using a number of second candidate characterization data to replace candidate first characterization data and adjacent characterization data in a plurality of first characterization data. Therefore, the point density of each second representation data in the second representation data set will change accordingly, and the point density of each second representation data needs to be recalculated in order to determine the second representation data with the highest point density in the second representation data set, that is, Candidate second characterization data.

Figure 4 is a schematic diagram of a process of generating a second representation data set according to an embodiment of the present invention. The following is an example of the change in point density of each second characterization data in the second characterization data set with reference to FIG. 4 .

As shown in 440 in Figure 4 , the plurality of first characterization data include, for example, first characterization data ct ₁ , first characterization data ct ₂ , first characterization data ct ₃ , first characterization data ct ₄ , and first characterization data ct _5. First characterization data ct ₆ .......

For example, let the second distance threshold be lc. For each first characterization data, a number of adjacent characterization data associated with the first characterization data is determined from the plurality of first characterization data based on the second distance threshold lc and the feature vector of the first characterization data. Then, the number of adjacent characterization data is determined as the point density of the first characterization data. Take the first representation data ct ₁ as an example. For example, based on the second distance threshold lc and the feature vector of the first characterization data, it is determined that the number of adjacent characterization data associated with the first characterization data ct ₁ is nine. These nine neighboring characterization data are located in a density area with the first characterization data ct ₁ as the center and the second distance threshold lc as the radius, and include, for example, the first characterization data ct ₂ , the first characterization data ct ₆ , etc.

Afterwards, the corresponding point densities of the plurality of first representation data are sorted, and based on the point density sorting result, the first representation data with the largest point density among the plurality of first representation data is determined as the candidate first representation data. For example, the first representation data with the highest point density among the plurality of first representation data is the first representation data ct ₁ , that is, the candidate first representation data is the first representation data ct ₁ .

Next, based on the point density of the candidate first representation data, the number of second candidate representation data associated with the candidate first representation data is determined. For example, based on the above formula (1), it may be determined that the number of second candidate characterization data associated with the candidate first characterization data is, for example, 4.

Next, based on the number of second candidate representation data, clustering processing is performed on the candidate first representation data and adjacent representation data to obtain a number of second candidate representation data.

As shown in 450 in Figure 4, for example, based on the number of second candidate representation data, clustering processing is performed on the first representation data ct ₁ and the adjacent representation data of the first representation data ct ₁ to generate 4 second candidate representations. Data, such as second candidate representation data cp ₁ , second candidate representation data cp ₂ , second candidate representation data cp ₃ and second candidate representation data cp ₄ .

Next, a number of second candidate representation data are used to replace the candidate first representation data and corresponding adjacent representation data in the plurality of first representation data to obtain a second representation data set.

As shown at 460 in Figure 4, for example, the second candidate representation data cp ₁ to the second candidate representation data cp ₄ can be used to replace the first representation data ct 1 and the adjacent representation data of the first representation data ct ₁ to obtain the first representation data ct 1 and the adjacent representation data of the first representation data ct ₁ . 2. Characterization data set. The second characterization data set includes 4 second candidate characterization data and other first characterization data except the first characterization data ct ₁ and adjacent characterization data. Each representation data included in the second representation data set is the second representation data.

In the above process of generating the second representation data set, for the first representation data whose distance from the candidate first representation data (ie, the first representation data ct ₁ ) is less than or equal to the second distance threshold lc, that is, the first representation The neighboring representation data of data ct ₁ (for example, the first representation data ct ₂ shown in 440 ), these neighboring representation data and the candidate first representation data will be replaced. That is to say, these neighboring representation data and candidate first representation data will not exist in the second representation data set.

In addition, for the first characterization data whose distance from the candidate first characterization data is greater than one time of the second distance threshold lc and less than or equal to two times of the second distance threshold lc, for example, the first characterization data ct ₃ shown in 440 , since the density area of the candidate first characterization data overlaps with the density area of the first characterization data ct ₃ , in other words, the candidate first characterization data and the first characterization data ct ₃ have the same adjacent characterization data (for example, the first characterization data ct 3 ). 1. Characterization data ct ₆ ). After using a number of second candidate representation data to replace the candidate first representation data and adjacent representation data in the plurality of first representation data, the point density in the second representation data set such as the first representation data ct ₃ will be affected by the candidate first representation data The data (that is, the first representation data ct ₁ ) is changed. Therefore, the point density of these affected first representation data needs to be recalculated.

In addition, for the first characterization data whose distance from the candidate first characterization data is greater than twice the second distance threshold lc, such as the first characterization data within the density area of the first characterization data ct4 shown in 440, due to the candidate The first representation data and the adjacent representation data are independent of each other. Therefore, these data will not be affected by the candidate first representation data and the adjacent representation data, and there is no need to calculate the point density of these first representation data later.

In addition, the second representation data set also includes a newly generated number of second candidate representation data, and the distances between these second candidate representation data, and the distance between these second candidate representation data and the first candidate in the second representation data set are The distance between the characterization data and other first characterization data other than the adjacent characterization data will also change accordingly (as shown at 470 in Figure 4). Therefore, it is also necessary to calculate the point density of the newly generated number of second candidate representation data.

Based on the above introduction, it can be seen that when determining candidate second representation data, point density calculation can be performed only for the affected first representation data and a number of second candidate representation data, without having to calculate all the second representation data in the second representation data set. The point density is calculated. As a result, data processing efficiency can be improved and computing resources can be saved.

According to embodiments of the present invention, the candidate second characterization data may be determined in the following manner.

First, the affected first characterization data associated with the candidate first characterization data among the plurality of first characterization data is determined. For example, the first characterization data among the plurality of first characterization data whose distance from the candidate first characterization data is greater than one time of the second distance threshold and less than or equal to two times of the second distance threshold can be determined as the affected first characterization data. 1. Characterization data.

Next, the point density of the affected first representation data and the point density of the number of second candidate representation data are determined respectively.

It should be noted that in the embodiment of the present invention, when determining the point density of the affected first representation data and the point density of the number of second candidate representation data, for each representation data, based on the second distance threshold and the A feature vector of the representation data, and a number of neighboring representation data associated with the representation data is determined from the second set of representation data. Then, the number of adjacent representation data is determined as the point density of the representation data.

Next, according to the point density of the affected first representation data, the point density of the number of second candidate representation data, and the point density of other representation data in the candidate representation data set except the affected first representation data, the third point density is determined. The second representation data with the largest point density in the second representation data set is used as the candidate second representation data.

In some embodiments, reordering of point densities of respective second representation data in the second representation data set may be implemented based on an ordered queue to obtain candidate second representation data. The point density reordering process is illustrated below with reference to Figure 5.

FIG. 5 is a schematic diagram of reordering the point density of each second characterization data in the second characterization data set according to an embodiment of the present invention.

As shown in 501 in Figure 5, for a plurality of first characterization data, for example, the first characterization data ct ₁ to the first characterization data ct _n , n is an integer greater than 1, when determining the first characterization data ct ₁ to the first After the point density of the characterization data ct _n is determined, the first characterization data can be sorted according to the size of the point density to obtain the characterization data sequence 51 . Afterwards, according to the characterization data sequence 51, the first characterization data with the highest point density among the plurality of first characterization data, that is, the first characterization number ct ₁ can be obtained, and the first characterization number ct ₁ is used as the candidate first characterization data.

Next, based on the point density of the candidate first representation data, the number of second candidate representation data associated with the candidate first representation data is determined. For example, it may be determined that the number of second candidate representation data associated with the first representation number ct ₁ is m, and m is a positive integer.

Next, based on the number of second candidate representation data, clustering processing is performed on the candidate first representation data and the corresponding adjacent representation data to obtain a number of second candidate representation data. For example, by performing clustering processing on the first representation data ct ₁ and the corresponding adjacent representation data (for example, including the first representation data ct _g , the first representation data ct _j , etc., where g and j are positive integers less than n), m second candidate representation data are obtained, such as second candidate representation data cp ₁ , second candidate representation data cp ₁ , ..., second candidate representation data cp _m .

Next, affected first characterization data associated with the candidate first characterization data among the plurality of first characterization data is determined. For example, it is determined that the affected first characterization data associated with the first characterization data ct ₁ includes the first characterization data ct ₂ .

Next, as shown in 502 in FIG. 5 , the first characterization data ct ₁ and adjacent characterization data corresponding to the first characterization data ct ₁ (for example, including the first characterization data ct _g , The first characterization data ct _j , etc.).

In addition, as shown at 503 in FIG. 5 , the affected first characterization data, such as the first characterization data ct ₂ , in the characterization data sequence 51 can be moved into the characterization data sequence 52 to be sorted. And move m second candidate representation data (for example, second candidate representation data cp ₁ to second candidate representation data cp _m ) into the sequence of representation data to be sorted 52 . This facilitates reinserting each representation data in the representation data sequence 52 to be sorted into the representation data sequence 51 after determining the point density of each representation data in the representation data sequence 52 to be sorted.

Please continue to refer to 504 to 506 in Figure 5. After determining the point density of each representation data in the representation data sequence 52 to be sorted, each representation data can be inserted back into the representation data sequence 51 in sequence to obtain the reordered Characterize data sequences.

Take inserting the first characterization data ct ₂ back into the characterization data sequence 51 as an example. For example, the insertion position of the first characterization data ct ₂ can be determined by comparing the point density of the first characterization data ct ₂ with the remaining characterization data in the characterization data sequence 51 .

If the point density of the first characterization data ct ₂ is greater than or equal to the maximum point density of the remaining characterization data in the characterization data sequence 51 (corresponding to the point density of the first characterization data ct ₃ in the characterization data sequence 51 ), then the first characterization data ct ₂ is inserted to the left of the first representation data ct ₃ .

If the point density of the first characterization data ct ₂ is less than or equal to the minimum point density of the remaining characterization data in the characterization data sequence 51 (corresponding to the point density of the first characterization data ct _n in the characterization data sequence 51 ), then the first characterization data ct ₂ is inserted to the right of the first representation data ct _n .

If the point density of the first characterization data ct ₂ is greater than the minimum point density and less than the maximum point density, starting from the position of the maximum point density in the characterization data sequence 51 , for example, the first representation data ct can be compared by sliding comparison. ₂ Insert back the characterizing data sequence 51.

When inserting each representation data back into the representation data sequence 51 through sliding comparison, the point density of the representation data to be inserted can satisfy the following conditions.

N _{density_i-1} ≥N _{density_current} ≥N _{density_i} (2)

In formula (2), N _{density_i-1} and N _{density_i} represent the point density of two adjacent representation data in the representation data sequence 51, and N _{density_current} represents the point density of the representation data to be inserted. Where i=2, 3,...,n.

For example, when it is determined that the point density of the first characterization data ct ₂ is between the point densities of two adjacent characterization data (for example, the first characterization data ct ₄ and the first characterization data ct _k ), the first characterization data may be ct ₂ is inserted between the two adjacent characterization data, thereby completing the insertion of the first characterization data ct ₂ back into the characterization data sequence 51 .

It should be noted that in the process of inserting each representation data back into the representation data sequence 51, in addition to the sliding comparison method, other suitable methods can also be used, such as the binary search method. You can choose according to actual needs, and there are no restrictions here.

Based on the above method, the point density of each second representation data in the second representation data set can be reordered, so that candidate second representation data can be obtained according to the reordered ordered queue.

According to embodiments of the present invention, in the above process of determining candidate first representation data, after obtaining the representation data sequence, the representation data sequence can also be split into multiple first representation data sub-sequences for parallel processing based on computing resources. , to speed up the efficiency of obtaining candidate first characterization data. This method is especially suitable for situations where the amount of data in the first characterization data set is large.

For example, after the plurality of first representation data are sorted according to their respective point densities corresponding to the plurality of first representation data, a sequence of representation data is obtained. The characterization data sequence may be split into a plurality of first characterization data sub-sequences based on computing resources. Wherein, each first characterization data subsequence includes a preset number of first characterization data. In one example, each first characterization data sub-sequence may include the same amount of first characterization data, which can ensure that the processing speed of each first characterization data sub-sequence is relatively consistent during parallel computation.

Next, for each first characterization data subsequence, the first characterization data with the highest point density among the preset number of first characterization data is determined as the first subsequence characterization data.

Next, the first sub-sequence representation data with the highest point density among the plurality of first sub-sequence representation data is determined as the candidate first representation data.

According to embodiments of the present invention, in the above process of using an ordered queue to reorder the point density of each second representation data in the second representation data set to obtain candidate second representation data, it is also possible to use computing resources to The characterization data sequence is split into multiple first characterization data sub-sequences for reordering and parallel processing, thereby speeding up the efficiency of obtaining candidate second characterization data. Wherein, each first characterization data subsequence includes a preset number of first characterization data.

For example, after determining the affected first characterization data associated with the candidate first characterization data among the plurality of first characterization data, the affected first characterization data and the number of second candidate characterization data are stored in the to-be-sorted representation in the data sequence.

Next, for each first characterization data subsequence, delete the characterization data related to the candidate first characterization data and adjacent characterization data in the first characterization data subsequence as well as the affected first characterization data, to obtain the second characterization data subsequence. sequence.

Next, the point density of the affected first representation data and the point density of the number of second candidate representation data in the sequence of representation data to be sorted are determined respectively. It should be noted that when determining the point density of the affected first representation data and the point density of the number of second candidate representation data in the sequence of representation data to be sorted, for each representation data, based on the second distance threshold and the representation A feature vector of the data, and a number of neighboring representation data associated with the representation data is determined from the second representation data set. Then, the number of adjacent representation data is determined as the point density of the representation data.

Next, according to the point density of the affected first representation data and the point density of the number of second candidate representation data in the representation data sequence to be sorted, and the point density of the representation data in the plurality of second representation data sub-sequences, respectively The affected first characterization data and the number of second candidate characterization data in the characterization data sequence to be sorted are allocated to a plurality of second characterization data sub-sequences to obtain a plurality of third characterization data sub-sequences.

In the embodiment of the present invention, the process of allocating the affected first characterization data and the number of second candidate characterization data in the characterization data sequence to be sorted to multiple second characterization data subsequences is the same as the above-described interpolation of each characterization data. The process of characterizing data sequences is similar and will not be described again here.

Next, for each third characterization data subsequence, the characterization data with the highest point density in the third characterization data subsequence is determined as the second subsequence characterization data. Afterwards, the second sub-sequence representation data with the highest point density among the plurality of second sub-sequence representation data is determined as the candidate second representation data.

According to an embodiment of the present invention, in the above process of generating a number of second candidate representation data, the object data identifier corresponding to each second candidate representation data may also be determined based on the object data identifier of each object data.

For example, for each second candidate representation data, candidate first representation data and adjacent representation data associated with the second candidate representation data are determined. Then, based on the object data identifiers corresponding to the plurality of object data, the object data identifier of the candidate first representation data and the object data identifier of the adjacent representation data are determined.

Afterwards, the object data identification of the second candidate characterization data is determined based on the object data identification of the first characterization data candidate and the object data identification of adjacent characterization data. and associate the object data identifier of the second candidate representation data with the second candidate representation data.

For example, the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate first representation data and the object data identifiers of adjacent representation data may be determined. Then, the object data identifier with the largest number is used as the object data identifier of the second candidate representation data. Here, the process of determining the object data identifier of the second candidate representation data is similar to the above-mentioned process of determining the object data identifier of the first candidate representation data, and will not be described again.

FIG. 6A is a schematic diagram of the effect of data classification using related technologies, and FIG. 6B is a schematic diagram of the effect of data classification based on the technical solution of the present invention. The advantages of the technical solution of the present invention will be described below with reference to Figures 6A and 6B.

The plurality of object data include, for example, object data _Pa1 , object data _Pa2 , object data _Pa3 , . . . . Among them, the object data identifier of the object data _Pa1 is object data identifier 1, the object data identifier of the object data _Pa2 is object data identifier 2, and the object data identifier of the object data _Pa3 is object data identifier 1.

The object data identifier of each object data is, for example, used to indicate the category to which the object data belongs. For example, the category of the object data _Pa1 and the object data _Pa3 is category 1, and the category of the object data _Pa2 is category 2, for example. It should be noted that, in order to simply and clearly illustrate the data classification effect, in this embodiment, the object data with the same object data identifier in FIG. 6A and FIG. 6B are represented by the same symbols.

Take the object data P _a3 as the object data to be classified as an example. Relevant technologies such as k-Nearest Neighbor classification algorithm (k-Nearest Neighbor, KNN) and the combination of the technical solution of the present invention and the k-Nearest Neighbor classification algorithm are respectively used to classify the object data P _a3 .

Figure 6A is a schematic diagram of the effect of data classification based on the k-nearest neighbor classification algorithm. As shown in 610 and 611 in Figure 6A, when the k-nearest neighbor classification algorithm is used to classify the object data P _a3 , with the nearest neighbor value k=5 (only an example), the five closest neighbors to the object data P _a3 can be determined. The object data includes 3 object data with object data identifier 2 and 2 object data with object data identifier 1. Therefore, it can be determined that the classification result of the object data _Pa3 is category 2, that is, the object data identifier of the object data _Pa3 is object data identifier 2. Obviously, this is inconsistent with the actual object data identification (ie, object data identification 1) of the object data _Pa3 . It can be seen that if the object data P _a3 is classified directly based on the k nearest neighbor classification algorithm, because the object data is too dense, the classification results will be affected by local noise, causing the classification results to be biased.

Figure 6B is a schematic diagram of the effect of data classification based on the technical solution of the present invention. As shown at 610 to 630 in Figure 6B, after processing multiple object data based on the solution of the present invention, the first sample data set can be obtained. As shown in 630, the first sample data set includes object data _Pa3 , a plurality of first candidate representation data _ct11 , and first candidate representation data _ct21 . Among them, the first candidate representation data ct ₁₁ is generated from the object data _Pa1 and the corresponding neighboring object data. The first candidate representation data ct ₂₁ is generated from the object data _Pa2 and corresponding neighboring object data.

Based on the first sample data set, the k-nearest neighbor classification algorithm is used to classify the object data P _a3 . With the nearest neighbor value k=5 (only an example), the five object data closest to the object data P _a3 can be determined. It includes 4 object data with object data identification 1 and 1 object data with object data identification 2. Therefore, it can be determined that the classification result of the object data P _a3 is category 1, that is, the object data identifier of the object data P _a3 is the object data identifier 1. At this time, the classification result of the object data _Pa3 is consistent with the actual object data identification of the object data _Pa3 (ie, the object data identification 1).

It can be understood that in the process of processing multiple object data using the solution of the present invention, the first sample data set is obtained by using the first candidate representation data to replace the corresponding object data and adjacent object data. As a result, the simplified processing of redundant data in multiple object data can be achieved, thereby eliminating the influence of local noise to a certain extent, which is beneficial to improving the accuracy of classification results. The results shown in Figure 6B also illustrate this point.

Figure 7 is a flow chart of a training method for a target object detection model according to an embodiment of the present invention.

As shown in FIG. 7 , the training method 700 of the target object detection model includes operations S710 to S720.

In operation S710, a sample data set is obtained. In the embodiment of the present invention, the sample data set is obtained using the data processing method described in the above embodiment. The sample data set contains sample data of the desired amount of data.

In operation S720, the deep learning model is iteratively trained using the sample data set until the output result of the deep learning model meets the iteration stop condition or the cumulative number of iterative trainings reaches a preset number threshold, and the target object detection model is obtained.

In the embodiment of the present invention, the above sample data set can be used to perform multiple iterative trainings on the deep learning model until the output result of the deep learning model satisfies the iteration stop condition or the cumulative number of iterative training reaches a preset threshold, and the target object is obtained. Detection model. The iteration stop condition may include, for example, that the difference value between the output result of the deep learning model and the label information of the sample data satisfies a preset convergence condition.

In this embodiment of the present invention, the sample data includes, for example, at least one of text, image, and video to be detected. The target object detection model is, for example, used to process sample data to obtain detection results for the sample data.

In another example, the sample data includes, for example, text, images, videos, audios, or location information carrying identification data to be classified. The sample data can also be used to train a deep learning model, so that the trained deep learning model can be used to classify the sample data.

In another example, the sample data includes text to be converted, for example. The sample data can also be used to train a deep learning model, so that the trained deep learning model can be used to process the text to be converted and obtain conversion data for the text to be converted. The text to be converted includes, for example, the text to be translated, and the conversion data includes, for example, translated text. In other words, a trained deep learning model is used to translate text.

It should be noted that the type of sample data and the use of the trained deep learning model in the embodiment of the present invention are not limited to the above examples, and can also be determined according to the actual application scenario, which will not be described again here.

According to embodiments of the present invention, by using a sample data set with a desired amount of data to train a deep learning model, not only can the training efficiency of the model be improved, but computing resources can be saved. It can also avoid the accumulation of local noise caused by excessive sample data, thus improving the accuracy of model output.

Figure 8 is a flow chart of a target object detection method according to an embodiment of the present invention.

As shown in FIG. 8 , the target object detection method 800 includes operation S810.

In operation S810, the data to be processed is input into the target object detection model to obtain a detection result for the data to be processed. Among them, the target object detection model is trained using the training method of the target object detection model described above.

According to the embodiment of the present invention, the data to be processed includes, for example, at least one of text, image, and video to be detected. The target object detection model is, for example, used to process the data to be processed to obtain detection results for the target objects in the data to be processed. Of course, the type of data to be processed and the use of the target object detection model in the embodiment of the present invention are not limited to the above examples, and can also be determined according to the actual application scenario, which will not be described again here.

In the solution of the embodiment of the present invention, by using the target object detection model trained in the above manner to detect the target object for the data to be processed, the accuracy of target object detection can be improved.

Figure 9 is a block diagram of a data processing device according to an embodiment of the present invention.

As shown in FIG. 9 , the data processing device 900 includes: a first acquisition module 910 , a first determination module 920 , a generation module 930 , a processing module 940 and a second determination module 950 .

The first acquisition module 910 is used to acquire multiple object data.

The first determination module 920 is used to determine that the number of multiple object data is greater than the preset quantity threshold, and for any candidate object data in the multiple object data, based on the feature vector of the candidate object data and the first distance threshold, from the multiple objects Neighboring object data associated with the candidate object data is determined in the data.

The generation module 930 is configured to generate first candidate representation data according to the candidate object data and the adjacent object data in response to the presence of adjacent object data in the plurality of object data; wherein the first candidate representation data is used to replace the candidate object data and the adjacent object data. .

The processing module 940 is configured to process multiple object data based on the candidate object data, neighboring object data and the first candidate representation data to obtain a first representation data set.

The second determination module 950 is configured to determine that the number of first characterization data contained in the first characterization data set is less than or equal to the preset quantity threshold, and determine the first characterization data set as the first sample data set.

Figure 10 is a block diagram of a training device for a target object detection model according to an embodiment of the present invention.

As shown in Figure 10, the training device 1000 of the target object detection model includes: a second acquisition module 1010 and a training module 1020.

The second acquisition module 1010 is used to acquire a sample data set. Among them, the sample data set is obtained by using the data processing device 900 described in the above embodiment.

The training module 1020 is used to iteratively train the deep learning model using the sample data set until the output result of the deep learning model meets the iteration stop condition or the cumulative number of iterative trainings reaches a preset threshold to obtain the target object detection model.

Figure 11 is a block diagram of a target object detection device according to an embodiment of the present invention.

As shown in FIG. 11 , the target object detection device 1100 includes: an input module 1110 .

The input module 1110 is used to input the data to be processed into the target object detection model to obtain the target object detection model described in the above embodiments for the data to be processed, which is trained by the training device 1000 .

It should be noted that the implementation of each module/unit/subunit, etc., the technical problems solved, the functions implemented, and the technical effects achieved in the device embodiment are respectively the same as the implementation of the corresponding steps in the method embodiment. , the technical problems solved, the functions implemented, and the technical effects achieved are the same or similar, and will not be described again here.

In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (for example, including but not limited to user personal information) comply with the provisions of relevant laws and regulations, and are not Violate public order and good customs.

In the technical solution of the present invention, before obtaining or collecting relevant data, the authorization or consent of the data owner is obtained.

According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

According to an embodiment of the present invention, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are processed by at least one processor. processor execution, so that at least one processor can execute the method as described above.

According to an embodiment of the present invention, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform the method as described above.

According to an embodiment of the present invention, a computer program product includes a computer program, and the computer program implements the above method when executed by a processor.

FIG. 12 schematically shows a block diagram of an electronic device suitable for implementing a data processing method, a training method of a target object detection model, and a target object detection method according to an embodiment of the present invention.

As shown in Figure 12, an electronic device 1200 according to an embodiment of the present invention includes a processor 1201, which can be loaded into a random access memory (RAM) 1203 according to a program stored in a read-only memory (ROM) 1202 or from a storage part 1208. program to perform various appropriate actions and processes. Processor 1201 may include, for example, a general-purpose microprocessor (eg, a CPU), an instruction set processor and/or an associated chipset, and/or a special-purpose microprocessor (eg, an application specific integrated circuit (ASIC)), or the like. Processor 1201 may also include onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present invention.

In the RAM 1203, various programs and data required for the operation of the electronic device 1200 are stored. The processor 1201, ROM 1202, and RAM 1203 are connected to each other through a bus 1204. The processor 1201 performs various operations according to the method flow of the embodiment of the present invention by executing programs in the ROM 1202 and/or RAM 1203. It should be noted that the program may also be stored in one or more memories other than ROM 1202 and RAM 1203. The processor 1201 may also perform various operations according to the method flow of embodiments of the present invention by executing programs stored in the one or more memories.

According to an embodiment of the present invention, the electronic device 1200 may further include an input/output (I/O) interface 1205 that is also connected to the bus 1204. Electronic device 1200 may also include one or more of the following components connected to I/O interface 1205: an input portion 1206 including a keyboard, mouse, etc.; including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and an output section 1207 of a speaker and the like; a storage section 1208 including a hard disk and the like; and a communication section 1209 including a network interface card such as a LAN card, a modem and the like. The communication section 1209 performs communication processing via a network such as the Internet. Driver 1210 is also connected to I/O interface 1205 as needed. Removable media 1211, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage portion 1208 as needed.

The present invention also provides a computer-readable storage medium. The computer-readable storage medium can be included in the equipment/device/system described in the above embodiments; it can also exist independently without being assembled into the equipment/system. in the device/system. The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, but is not limited to, portable computer disks, hard disks, random access memory (RAM), and read-only memory (ROM). , erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, the computer-readable storage medium may include one or more memories other than ROM 1202 and/or RAM 1203 and/or ROM 1202 and RAM 1203 described above.

Embodiments of the invention also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flowchart. When the computer program product is run in the computer system, the program code is used to enable the computer system to implement the data processing method, the training method of the target object detection model and the target object detection method provided by the embodiments of the present invention.

When the computer program is executed by the processor 1201, the above functions defined in the system/device of the embodiment of the present invention are performed. According to embodiments of the present invention, the systems, devices, modules, units, etc. described above may be implemented by computer program modules.

In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program can also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 1209, and/or installed from the removable medium 1211. The program code contained in the computer program can be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

In such embodiments, the computer program may be downloaded and installed from the network via communication portion 1209, and/or installed from removable media 1211. When the computer program is executed by the processor 1201, the above-mentioned functions defined in the system of the embodiment of the present invention are performed. According to embodiments of the present invention, the systems, devices, devices, modules, units, etc. described above may be implemented by computer program modules.

According to the embodiments of the present invention, the program code for executing the computer program provided by the embodiments of the present invention may be written in any combination of one or more programming languages. Specifically, high-level procedures and/or object-oriented programming languages may be used. programming language, and/or assembly/machine language to implement these computational procedures. Programming languages include, but are not limited to, programming languages such as Java, C++, python, "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device, such as provided by an Internet service. (business comes via Internet connection).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block in the block diagram or flowchart illustration, and combinations of blocks in the block diagram or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or may be implemented by special purpose hardware-based systems that perform the specified functions or operations. Achieved by a combination of specialized hardware and computer instructions.

Those skilled in the art will understand that features described in various embodiments of the present invention may be combined and/or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the invention may be combined and/or combined in various ways without departing from the spirit and teachings of the invention. All such combinations and/or combinations fall within the scope of the invention.

The embodiments of the present invention have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the invention. Although each embodiment is described separately above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present invention.

Claims (18)

1. A data processing method, including: Get multiple object data; It is determined that the number of the plurality of object data is greater than the preset quantity threshold, and for any candidate object data in the plurality of object data, according to the feature vector of the candidate object data and the first distance threshold, from the plurality of object data Determine adjacent object data associated with the candidate object data in the object data; In response to the presence of the adjacent object data in the plurality of object data, generating first candidate representation data according to the candidate object data and the adjacent object data; wherein the first candidate representation data is used to replace the candidate object data and the adjacent object data; Based on the candidate object data, the adjacent object data and the first candidate representation data, the plurality of object data are processed to obtain a first representation data set; and It is determined that the number of first characterization data contained in the first characterization data set is less than or equal to the preset quantity threshold, and the first characterization data set is determined as a first sample data set. 2. The method according to claim 1, wherein the plurality of object data are processed based on the candidate object data, the adjacent object data and the first candidate representation data to obtain a first representation. Data sets include: Delete the candidate object data and the adjacent object data in the plurality of object data to obtain a candidate object data set; and The first characterization data set is determined based on the candidate object data set and the first candidate characterization data. 3. The method of claim 2, the first representation data set comprising a plurality of first representation data; in response to detecting that the candidate object data set does not include object data, and determining the plurality of first representations The amount of data is greater than the preset amount threshold, and the method further includes: determining a point density for each of said first characterization data; Determine the first characterization data with the largest point density among the plurality of first characterization data as the candidate first characterization data; determining a number of second candidate characterization data associated with the candidate first characterization data based on the point density of the candidate first characterization data; Based on the number of the second candidate characterization data, perform characterization data extraction on the plurality of first characterization data to obtain a second characterization data set; and It is determined that the number of second characterization data contained in the second characterization data set is less than or equal to the preset quantity threshold, and the second characterization data set is determined as a second sample data set. 4. The method according to claim 3, wherein, based on the number of the second candidate characterization data, performing characterization data extraction on the plurality of first characterization data to obtain the second characterization data set includes: determining adjacent characterization data associated with the candidate first characterization data from the plurality of first characterization data based on a feature vector of the candidate first characterization data and a second distance threshold; Based on the number of the second candidate representation data, perform clustering processing on the candidate first representation data and the adjacent representation data to obtain the number of second candidate representation data; Delete the candidate first characterization data and the adjacent characterization data in the plurality of first characterization data to obtain a candidate characterization data set; and The second characterization data set is determined based on the candidate characterization data set and the number of second candidate characterization data. 5. The method of claim 4, wherein in response to determining that a quantity of second characterization data contained in the second characterization data set is greater than the preset quantity threshold, the method further comprises: determining affected first characterization data associated with the candidate first characterization data among the plurality of first characterization data; Determine the point density of the affected first representation data and the point density of the number of second candidate representation data respectively; According to the point density of the affected first representation data, the point density of the number of second candidate representation data, and other representation data in the candidate representation data set except the affected first representation data. Point density, determining the second representation data with the largest point density in the second representation data set as the candidate second representation data; and Update the point density of the candidate first characterization data based on the point density of the candidate second characterization data, and repeatedly determine whether the number of second characterization data contained in the second characterization data set is less than or equal to the preset Quantity threshold operations. 6. The method according to claim 4, wherein determining the first characterization data with the largest point density among the plurality of first characterization data as the candidate first characterization data includes: Sort the plurality of first representation data according to the corresponding point density of each of the plurality of first representation data to obtain a sequence of representation data; Based on computing resources, split the characterization data sequence into a plurality of first characterization data sub-sequences; wherein each of the first characterization data sub-sequences includes a preset number of first characterization data; For each first characterization data subsequence, determine the first characterization data with the highest point density among the preset number of first characterization data as the first subsequence characterization data; and The first sub-sequence representation data with the highest point density among the plurality of first sub-sequence representation data is determined as the candidate first representation data. 7. The method of claim 6, wherein in response to determining that a quantity of second characterization data contained in the second characterization data set is greater than the preset quantity threshold, the method further comprises: determining affected first characterization data associated with the candidate first characterization data among the plurality of first characterization data; Store the affected first characterization data and the number of second candidate characterization data into a sequence of characterization data to be sorted; For each first characterization data subsequence, delete the characterization data related to the candidate first characterization data and the adjacent characterization data and the affected first characterization data in the first characterization data subsequence, and obtain a second representation data subsequence; Determine respectively the point density of the affected first representation data and the point density of the number of second candidate representation data in the sequence of representation data to be sorted; According to the point density of the affected first representation data in the to-be-sorted representation data sequence and the point density of the number of second candidate representation data, and the point density of the representation data in a plurality of the second representation data sub-sequences. Point density, respectively assign the affected first characterization data and the number of second candidate characterization data in the to-be-sorted characterization data sequence to multiple second characterization data sub-sequences to obtain multiple third characterization data subsequence; For each third characterization data subsequence, determine the characterization data with the highest point density in the third characterization data subsequence as the second subsequence characterization data; Determine the second subsequence representation data with the highest point density among the plurality of second subsequence representation data as the candidate second representation data; and Update the point density of the candidate first characterization data based on the point density of the candidate second characterization data, and repeatedly determine whether the number of second characterization data contained in the second characterization data set is less than or equal to the preset Quantity threshold operations. 8. The method of claim 4, wherein each of the object data includes an object data identifier; and based on the number of the second candidate characterization data, characterization data extraction is performed on the plurality of first characterization data. , obtaining the second representation data set also includes: for each of the second candidate representation data, determining candidate first representation data and the adjacent representation data associated with the second candidate representation data; Determine the object data identifier of the candidate first representation data and the object data identifier of the adjacent representation data according to the object data identifiers corresponding to the plurality of object data; Determine the object data identifier of the second candidate representation data based on the object data identifier of the candidate first representation data and the object data identifier of the adjacent representation data; and The object data identifier of the second candidate representation data is associated with the second candidate representation data. 9. The method of claim 8, wherein the object data identifier of the second candidate representation data is determined based on the object data identifier of the candidate first representation data and the object data identifier of the adjacent representation data. include: Determine the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate first representation data and the object data identifiers of the adjacent representation data; and The object data identifier with the largest number is used as the object data identifier of the second candidate representation data. 10. The method according to any one of claims 3 to 9, wherein the number of the second candidate characterization data is positively correlated with the point density of the candidate first characterization data. 11. The method of any one of claims 3 to 9, wherein determining the point density of each of the first characterization data includes: For each of the first representation data, based on a second distance threshold and a feature vector of the first representation data, determine from the plurality of first representation data the number of adjacent representation data associated with the first representation data. quantity; and The number of adjacent characterization data is determined as the point density. 12. The method of any one of claims 1 to 9, further comprising: In response to determining that no adjacent object data associated with the candidate object data exists in the plurality of object data, the candidate object data is determined as the first candidate representation data. 13. The method according to any one of claims 1 to 9, wherein each of the object data includes an object data identifier; and generating a first candidate representation based on the candidate object data and the adjacent object data. The data also includes: Determine the object data identifier of the candidate object data and the object data identifier of the adjacent object data according to the object data identifiers corresponding to the plurality of object data; determining the object data identifier of the first candidate representation data according to the object data identifier of the candidate object data and the object data identifier of the adjacent object data; and The object data identifier of the first candidate representation data is associated with the first candidate representation data. 14. The method of claim 13, wherein determining the object data identifier of the first candidate representation data based on the object data identifier of the candidate object data and the object data identifier of the adjacent object data includes: Determining the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate object data and the object data identifiers of the adjacent object data; and The object data identifier with the largest number is used as the object data identifier of the first candidate representation data. 15. A training method for a target object detection model, including: Get sample data set; Use the sample data set to iteratively train the deep learning model until the output result of the deep learning model satisfies the iteration stop condition or the cumulative number of iterative trainings reaches a preset threshold to obtain the target object detection model; Wherein, the sample data set is obtained using the method described in any one of claims 1 to 14. 16. A target object detection method, including: Input the data to be processed into the target object detection model, and obtain the detection results for the data to be processed; Wherein, the target object detection model is trained using the method described in claim 15. 17. An electronic device, including: one or more processors; memory for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the method according to any one of claims 1 to 16. 18. A computer-readable storage medium having executable instructions stored thereon, which when executed by a processor causes the processor to perform the method according to any one of claims 1 to 16.