CN117036873A - Multisource three-dimensional data fusion system - Google Patents

Abstract

The invention relates to a multi-source three-dimensional data fusion method, which includes a discrete grid input module, a point cloud input module, and a data fusion module; the discrete grid input module and the point cloud input module are parallel, and the outputs of the two modules are jointly input to the data fusion module. ;The output of the data fusion module is the point cloud data and discrete grid data after data fusion. The entire system realizes the alignment and fusion of point cloud data and grid data through the calculation cooperation between various modules, so that downstream tasks can obtain Higher quality fused data. Through the fusion, alignment and optimization of multi-source data, the present invention achieves accurate description of three-dimensional entities, improves model calculation accuracy and saves computing resources.

Description

A multi-source three-dimensional data fusion system

Technical field

The invention relates to the field of big data technology, and in particular to a multi-source three-dimensional data fusion system.

Background technique

As the structure of the analysis object becomes increasingly complex and the scale expands, the role of CAE in simulation becomes increasingly important. A single data source can often only describe certain characteristics of the modeling target in a one-sided manner, which creates a bottleneck in 3D modeling and analysis from the source. , however, the industry's current requirements for CAE simulation are increasing day by day, and simulation plays an increasingly important role in the entire design process. Simulation with a single data source is difficult to meet the requirements in terms of accuracy or functionality.

The concept of data fusion was proposed by American scholars around 1970. It was born under the background of "data fusion" and is often used in military applications. Initially, it refers to the analysis, extraction and synthesis of image information from multiple sensors, multiple phases, and multiple data sources; now it has further developed to combine data sources with different characteristics through the organic fusion of data, learn from each other's strengths, and achieve a more comprehensive and objective reflect the characteristics of the target object. When researchers studied data fusion and related algorithms, they found that by improving the fusion algorithm, not only can information be fully fused to obtain a more valuable three-dimensional model, but also the same model can be simulated with different functions through parallel computing. , improve the efficiency of simulation analysis.

In the existing technology, point cloud is a commonly used data format in three-dimensional modeling. Most of the data collected by a three-dimensional laser scanner is point cloud data. The specific data of point cloud data includes important information such as position, normal vector, and color. Among them, distance Information and three-dimensional coordinate information are the main information. Information such as the target's reflectivity, reflection intensity, distance from the point to the scanner, horizontal angle, and vertical angle are all derived information from the laser point cloud and are often ignored during the processing. Although point cloud data contains a lot of information, it also contains a lot of noise and errors. It requires multiple operations such as feature matching and denoising to be used effectively. Discrete grid is also a common three-dimensional geometric data. The grid is derived from the fusion of geometric topology and mechanics. It is a complex interdisciplinary subject. The idea of grid originates from the idea of discretization solution. By combining the needs The continuous solution area is discretized and decomposed into several continuous sub-areas, and the physical variables of each sub-area are solved respectively. Each sub-area is adjacent to each other, continuous and coordinated, thereby achieving coordination and continuity of the entire variable field. However, mesh division is a serious challenge to engineers' geometric topology planning capabilities. Often, area division will take nearly 80% of engineers' entire analysis work process. At the same time, due to structural requirements, the quality of the mesh is sometimes difficult to control. At the same time, the quality of the grid directly affects the final simulation accuracy.

Contents of the invention

The purpose of the present invention is to provide a multi-source three-dimensional data fusion system to at least solve one of the shortcomings of the existing technology.

In order to achieve the above objects, the present invention adopts the following technical solutions:

Specifically, a multi-source 3D data fusion system is proposed, including:

The point cloud input module is used to extract features from the point cloud data, obtain the feature vector V of the point cloud data, obtain a generated point cloud based on the feature vector V, and output a binary classification based on the generated point cloud as the point cloud feature P ;

The grid input module is used to re-grid the input grid data, construct a subdivision structure, obtain a multi-resolution representation of the general grid, and perform zigzag expansion convolution to output grid features M;

Data fusion module, the input end is connected in parallel to the output end of the point cloud input module and the output end of the grid input module, encoding is performed according to the point cloud feature P and the grid feature M, and then normalized fusion is performed;

The training module is used to train the system. The strategy adopted is that point cloud data and grid data are input in turn during training, that is, the point cloud data is input separately at a certain time, but at the same time the point cloud data output by the data fusion module is input. Perform loss calculations on point clouds and grids, update the data fusion module parameters; then use the grid data alone as input, and simultaneously perform loss calculations on the point clouds and grids output by the data fusion module, update the data fusion module parameters, and repeat multiple times. The above steps finally achieve the alignment and completion of point cloud and grid data.

Further, specifically, the point cloud input module includes,

The multi-scale feature extractor E includes the feature embedding layer FEL, the attention layer and the multi-layer perceptron MLP in series. The function process of the feature embedding layer FEL is,

The point cloud data [B, Pnum, 3] passes through two multi-layer perceptron MLPs to increase the point-by-point features from 3 dimensions to 64 dimensions, that is, [B, Pnum, 64], and then the network sequentially performs sampling and K nearest neighbor aggregation operations. , The setting of the number of sampling points and the number of neighborhood points is adapted to the number of points of the point cloud data of this scale. At the same time, the difference between the point cloud features after neighborhood aggregation and the neighborhood center point features is used as the initial feature, and then through the multi-layer perceptron And perform a series operation with the point cloud features before neighborhood aggregation. Finally, after maximum pooling, the aggregate feature VT=[B,pnum/4,128] is output. The specific calculation is as shown in the formula,

In the formula, V _P represents the characteristics of the input point cloud, Represents the point cloud features that have been dimensionally increased by MLP, V _E represents the point cloud aggregation features output by FEL, SG() represents the sampling aggregation operation, RP(x,k) represents the operation of copying x k times, C represents the concatenation operation, MP Represents the maximum pooling operation,

The function process of the attention layer is,

The offset attention method is optimized. First, linear transformation is performed on the input VE to obtain the Que-ry (Q), Key (K) and Value (V) matrices, that is

In the formula, W _q , W _k , W _v represent the learnable linear transformation weight, d _a is the dimension of Que-ry (Q) and Key (K) matrix, d _e is the dimension of Value (V) matrix, and then use Que-ry (Q matrix and Key (K) matrix calculate attention weight through matrix dot product And add the elements/> Perform softmax and L1 norm normalization, and the intermediate feature V _sa is the weighted sum of the Value vector using the corresponding attention weight. The calculation formula is as follows,

V _sa =A·V;

The pyramid generator G is used to take the final feature vector V as input, and the output is the generated point cloud corresponding to the previous one in 3 scales;

The attention discriminator D is used to improve the performance of point cloud completion; the attention discriminator D is also an autoencoder structure. The encoder side uses a single-scale feature extractor composed of a feature embedding layer and an attention layer. The generated point cloud P1 and its corresponding real point cloud are sent to the discriminator D. After calculation by the feature extractor, a 640-dimensional feature vector is output, and then passed through continuous fully connected layers, and finally a binary output of false or true.

Further, specifically, the specific action process of the grid input module includes:

For a given triangular mesh, learn a global representation of the three-dimensional shape, or feature vectors on each face, to obtain local geometric features, for

Further, specifically, the data fusion module includes:

The point cloud encoder block P _enc and the mesh encoder block M _enc are used to encode the point cloud features P input by the point cloud input module and the mesh features M input by the mesh input module. The formula is as follows,

P _enc (P)＝φ _P and M _enc (M)＝φ _M

In the formula φ _P , The encoder block consists of fully connected layers f ₁ ^m and f ₂ ^m respectively, where m∈{P, M}, and/> f ₁ ^m and f ₂ ^m are followed by batch normalization, ReLU and dropout layers respectively;

The cross-attention block, used to share information between the point cloud representation φ _P and the mesh’s representation φ _M , consists of a multi-head self-attention layer, followed by a fully connected layer, using residual links for both layers, Then normalize,

The residual link calculation method of the cross attention block P _proj and the subsequent output module M _proj is as shown in the formula:

in The output module consists of two linear layers/> and/> The sequence consists of , where/> After that are batch normalization, ReLU activation function and dropout layer.

The beneficial effects of the present invention are:

The present invention proposes a multi-source three-dimensional data fusion method, which includes a discrete grid input module, a point cloud input module, and a data fusion module; the discrete grid input module and the point cloud input module are parallel, and the outputs of the two modules are jointly input to the data fusion module. ;The output of the data fusion module is the point cloud data and discrete grid data after data fusion. The entire system realizes the alignment and fusion of point cloud data and grid data through the calculation cooperation between various modules, so that downstream tasks can obtain Higher quality fused data.

Description of the drawings

The above and other features of the present disclosure will be more apparent from the detailed description of the embodiments illustrated in the accompanying drawings, in which the same reference numerals designate the same or similar elements. It will be apparent that the appended drawings in the following description The drawings are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts. In the drawings:

Figure 1 shows a schematic structural diagram of a multi-source three-dimensional data fusion system of the present invention;

Figure 2 shows the structure diagram of the feature embedding layer FEL of a multi-source three-dimensional data fusion system of the present invention;

Figure 3 shows the structure diagram of the attention layer of a multi-source three-dimensional data fusion system of the present invention;

Figure 4 shows a schematic diagram of grid approximation fusion of a multi-source three-dimensional data fusion system of the present invention;

Figure 5 shows a schematic diagram of grid convolution of a multi-source three-dimensional data fusion system of the present invention;

Figure 6 shows the structure diagram of the cross-attention block of a multi-source three-dimensional data fusion system of the present invention.

Detailed ways

The following will give a clear and complete description of the concept, specific structure and technical effects of the present invention in conjunction with the embodiments and drawings, so as to fully understand the purpose, solutions and effects of the present invention. It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of this application can be combined with each other. The same reference numbers used throughout the drawings indicate the same or similar parts.

Point cloud is a commonly used data format in 3D modeling. Most of the data collected by 3D laser scanners is point cloud data. The specific data of point cloud data includes important information such as position, normal vector, color, etc. Among them, distance information and 3D coordinate information is the main information. Information such as the target's reflectivity, reflection intensity, distance from the point to the scanner, horizontal angle, and vertical angle are all derived information from the laser point cloud and are often ignored during the processing. Although point cloud data contains a lot of information, it also contains a lot of noise and errors. It requires multiple operations such as feature matching and denoising to be used effectively. Discrete grid is also a common three-dimensional geometric data. The grid is derived from the fusion of geometric topology and mechanics. It is a complex interdisciplinary subject. The idea of grid originates from the idea of discretization solution. By combining the needs The continuous solution area is discretized and decomposed into several continuous sub-areas, and the physical variables of each sub-area are solved respectively. Each sub-area is adjacent to each other, continuous and coordinated, thereby achieving coordination and continuity of the entire variable field. However, mesh division is a serious challenge to engineers' geometric topology planning capabilities. Often, area division will take nearly 80% of engineers' entire analysis work process. At the same time, due to structural requirements, the quality of the mesh is sometimes difficult to control. At the same time, the quality of the grid directly affects the final simulation accuracy. Based on the above problems, a point cloud and grid fusion method is proposed. Through the fusion, alignment, and optimization of multi-source data, it can achieve accurate description of three-dimensional entities, improve model calculation accuracy, and save computing resources.

Referring to Figure 1, Embodiment 1, the present invention proposes a multi-source three-dimensional data fusion method, including:

The point cloud input module is used to extract features from the point cloud data, obtain the feature vector V of the point cloud data, obtain a generated point cloud based on the feature vector V, and output a binary classification based on the generated point cloud as the point cloud feature P ;

The grid input module is used to re-grid the input grid data, construct a subdivision structure, obtain a multi-resolution representation of the general grid, and perform zigzag expansion convolution to output grid features M;

Data fusion module, the input end is connected in parallel to the output end of the point cloud input module and the output end of the grid input module, encoding is performed according to the point cloud feature P and the grid feature M, and then normalized fusion is performed;

The training module is used to train the system. The strategy adopted is that point cloud data and grid data are input in turn during training, that is, the point cloud data is input separately at a certain time, but at the same time the point cloud data output by the data fusion module is input. Perform loss calculations on point clouds and grids, update the data fusion module parameters; then use the grid data alone as input, and simultaneously perform loss calculations on the point clouds and grids output by the data fusion module, update the data fusion module parameters, and repeat multiple times. The above steps finally achieve the alignment and completion of point cloud and grid data.

As a preferred embodiment of the present invention,

Specifically, the point cloud input module is a multi-scale point cloud completion network model based on Transformer. The entire network adopts an encoder-decoder structure and consists of a multi-scale feature extractor E, a pyramid point generator G and an attention discriminator. D composition. The input of the model is three original point clouds of different scales, which are then sent to the feature extractor E of the corresponding scale to generate the feature vector V corresponding to the original point cloud. Then the feature vector V is used as the input of the pyramid point generator G to output the generated point cloud in 3 scales. The loss function mainly consists of two parts: generation loss and adversarial loss. The generation loss is obtained by calculating the CD value (Chamfer distance) between the generated point clouds of 3 scales and the corresponding real point clouds. The adversarial loss is obtained by the attention discriminator. D is calculated. The attention discriminator D draws on the idea of GAN. Its input is the first scale generated point cloud, and the output is a binary classification value of true or false. The attention discriminator D is introduced to jointly train with the pyramid point generator G, thereby indirectly improving the point cloud completion performance of the point generator.

Combined with Figure 2, the multi-scale feature extractor E includes the feature embedding layer FEL, the attention layer and the multi-layer perceptron MLP in series. The function process of the feature embedding layer FEL is,

The point cloud data [B, Pnum, 3] passes through two multi-layer perceptron MLPs to increase the point-by-point features from 3 dimensions to 64 dimensions, that is, [B, Pnum, 64], and then the network sequentially performs sampling and K nearest neighbor aggregation operations. , The setting of the number of sampling points and the number of neighborhood points is adapted to the number of points of the point cloud data of this scale. The purpose is to make the network more capable of extracting local feature information of the object. At the same time, in order to avoid the negative impact of the absolute coordinate information of points on the neighborhood aggregation operation, the difference between the point cloud features after neighborhood aggregation and the neighborhood center point features is used as the initial feature, and then combined with the neighborhood through the multi-layer perceptron. The point cloud features before aggregation are operated in series, and finally after maximum pooling, the aggregated feature VT=[B,pnum/4,128] is output. The specific calculation is as shown in the formula,

In the formula, V _P represents the characteristics of the input point cloud, V _P represents the point cloud features that have been dimensionally increased by MLP, V _E represents the point cloud aggregation features output by FEL, SG() represents the sampling aggregation operation, and RP(x,k) represents The operation of copying x k times, C represents the concatenation operation, MP represents the maximum pooling operation,

Combined with Figure 3, the function process of the attention layer is,

The offset attention method is optimized. First, linear transformation is performed on the input VE to obtain the Que-ry (Q), Key (K) and Value (V) matrices, that is

In the formula, W _q , W _k , W _v represent the learnable linear transformation weight, d _a is the dimension of Que-ry (Q) and Key (K) matrix, d _e is the dimension of Value (V) matrix, and then use Que-ry (Q matrix and Key (K) matrix calculate attention weight through matrix dot product And add the elements/> Perform softmax and L1 norm normalization, and the intermediate feature V _sa is the weighted sum of the Value vector using the corresponding attention weight. The calculation formula is as follows,

V _sa =A·V;

The pyramid generator G is used to take the final feature vector V as input, and the output is the generated point cloud corresponding to the previous three scales.

The pyramid point generator G takes the final feature vector V as input, and the output is the generated point cloud corresponding to the previous three scales. Its network architecture mainly consists of a fully connected layer and a Reshape operation. In commonly used algorithms, the fully connected decoder is good at predicting the overall geometric structure of the point cloud, but because it only uses the last layer of features for completion, it is difficult to extract the local structural features of the point cloud. Therefore, we draw on the idea of layer-by-layer feature extraction from the feature pyramid network, and gradually complete the point cloud completion operation according to the process from rough to fine;

Attention discriminator D is used to improve the performance of point cloud completion; the point cloud completion task belongs to the generative task. Drawing on the idea of mutually promoting training of the generative network and the discriminant network in the generative adversarial network, the present invention also introduces attention The discriminator D is used to improve the performance of point cloud completion. The attention discriminator D is also the structure of an autoencoder. The encoder uses a single-scale feature extractor composed of a feature embedding layer and an attention layer to generate point clouds. P1 and its corresponding real point cloud are sent to the discriminator D. After calculation by the feature extractor, a 640-dimensional feature vector is output, and then passes through continuous fully connected layers, and finally is a binary output of false or true.

In conjunction with Figure 4 and Figure 5, as a preferred embodiment of the present invention, specifically, the specific action process of the grid input module includes:

Through the three-dimensional grid convolution operation, unlike different two-dimensional convolution operations, the convolution is only performed on the same plane. The three-dimensional grid convolution performs a three-dimensional convolution operation on the entire three-dimensional grid data, resulting in refinement and higher The high-resolution three-dimensional grid can achieve similar effects to the down-sampling and up-sampling operations in 2-dimensional convolution through forward or reverse operations, so that grid data can also be operated using mature neural network structures.

Previous grid deep learning methods store features at points or edges, but the degrees of points are not fixed and the convolution of edges is inflexible. Through the grid convolution method on the patches, the regular nature of each patch being adjacent to three patches is fully utilized. By extending this property, a variety of different convolution modes are designed based on the distance between patches.

As can be seen from Figure 5, this grid convolution method on the patch is intuitive, flexible, and regular, very similar to the image situation. In the figure, a) is the triangular patch convolution, b) is the repeated access that may occur in the convolution, and c) is a more complex convolution example.

Since the order of patches in the three-dimensional data format is not fixed, when calculating the convolution results, the calculation results are independent of the patch order by taking the neighborhood mean, difference mean, etc., and satisfy the arrangement invariance. The following formula gives the definition of convolution and the meaning of each term.

w ₀ e _i represents the central feature, Represents the sum of neighborhood features,/> Represents the sum of neighborhood differences,/> Represents the sum of the differences between center and field.

With reference to Figure 6, as a preferred embodiment of the present invention, specifically, the data fusion module includes:

The point cloud encoder block P _enc and the mesh encoder block M _enc are used to encode the point cloud features P input by the point cloud input module and the mesh features M input by the mesh input module. The formula is as follows,

P _enc (P)＝φ _P and M _enc (M)＝φ _M

In the formula φ _P , The encoder block consists of fully connected layers f ₁ ^m and f ₂ ^m respectively, where m∈{P, M}, and/> f ₁ ^m and f ₂ ^m are followed by batch normalization, ReLU and dropout layers respectively;

Cross-attention block, used to share information between point cloud and mesh representations φ _P and φ _M , consists of a multi-head self-attention layer, followed by a fully connected layer, using residual links for both layers, Then normalize,

The feedforward blocks of the point cloud encoder and mesh encoder branches each consist of a fully connected layer m∈{P,M}, then GELU activation function, dropout layer, and another fully connected layer/> m∈{P,M}, the last connection is the dropout layer. The output of the cross attention block is/>

The residual link calculation method of the cross attention block and subsequent output modules P _proj and M _proj is as shown in the formula:

in The output module consists of two linear layers/> and/> The sequence consists of , where/> This is followed by batch normalization, ReLU activation function, and dropout layer with dropout rate r _proj .

The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in various embodiments of the present invention can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software function modules.

If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments can be implemented. Wherein, the computer program includes computer program code, which may be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may include: any entity or system capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , random access memory (RAM, RandomAccess Memory), electrical carrier signals, telecommunications signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium Excluded are electrical carrier signals and telecommunications signals.

Although the present invention has been described in considerable detail and in particular to several of the described embodiments, it is not intended to be limited to any such details or embodiments or to any particular embodiment, but rather is to be considered by reference The appended claims are intended to provide the broadest possible interpretation of these claims, taking into account the prior art, to effectively cover the intended scope of the invention. In addition, the above description of the present invention is based on embodiments foreseeable by the inventor for the purpose of providing a useful description, and those non-substantive changes to the present invention that are not yet foreseen can still represent equivalent changes of the present invention.

The above are only preferred embodiments of the present invention. The present invention is not limited to the above-mentioned embodiments. As long as the technical effects of the present invention are achieved by the same means, they shall fall within the protection scope of the present invention. Various modifications and changes may be made to the technical solutions and/or implementations within the scope of the present invention.

Claims (4)

1. A multi-source three-dimensional data fusion system, which is characterized by including: The point cloud input module is used to extract features from the point cloud data, obtain the feature vector V of the point cloud data, obtain a generated point cloud based on the feature vector V, and output a binary classification based on the generated point cloud as the point cloud feature P ; The grid input module is used to re-grid the input grid data, construct a subdivision structure, obtain a multi-resolution representation of the general grid, and perform zigzag expansion convolution to output grid features M; Data fusion module, the input end is connected in parallel to the output end of the point cloud input module and the output end of the grid input module, encoding is performed according to the point cloud feature P and the grid feature M, and then normalized fusion is performed; The training module is used to train the system. The strategy adopted is that point cloud data and grid data are input in turn during training, that is, the point cloud data is input separately at a certain time, but at the same time the point cloud data output by the data fusion module is input. Perform loss calculations on point clouds and grids, update the data fusion module parameters; then use the grid data alone as input, and simultaneously perform loss calculations on the point clouds and grids output by the data fusion module, update the data fusion module parameters, and repeat multiple times. The above steps finally achieve the alignment and completion of point cloud and grid data. 2. A multi-source three-dimensional data fusion system according to claim 1, characterized in that, specifically, the point cloud input module includes: Multi-scale feature extractor E, including feature embedding layer FEL, attention layer and multi-layer concatenated multi-layer perceptron MLP, The function process of the feature embedding layer FEL is, The point cloud data [B, Pnum, 3] passes through two multi-layer perceptron MLPs to increase the point-by-point features from 3 dimensions to 64 dimensions, that is, [B, Pnum, 64], and then the network sequentially performs sampling and K nearest neighbor aggregation operations. , The setting of the number of sampling points and the number of neighborhood points is adapted to the number of points of the point cloud data of this scale. At the same time, the difference between the point cloud features after neighborhood aggregation and the neighborhood center point features is used as the initial feature, and then through the multi-layer perceptron And perform a series operation with the point cloud features before neighborhood aggregation. Finally, after maximum pooling, the aggregate feature VT=[B,pnum/4,128] is output. The specific calculation is as shown in the formula, In the formula, V _P represents the characteristics of the input point cloud, Represents the point cloud features that have been dimensionally increased by MLP, V _E represents the point cloud aggregation features output by FEL, SG() represents the sampling aggregation operation, RP(x,k) represents the operation of copying x k times, C represents the concatenation operation, MP Represents the maximum pooling operation, The function process of the attention layer is, The offset attention method is optimized. First, linear transformation is performed on the input VE to obtain the Que-ry (Q), Key (K) and Value (V) matrices, that is In the formula, W _q , W _k , W _v represent the learnable linear transformation weight, d _a is the dimension of Que-ry (Q) and Key (K) matrix, d _e is the dimension of Value (V) matrix, and then use Que-ry (Q matrix and Key (K) matrix calculate attention weight through matrix dot product And add the elements/> Perform softmax and L1 norm normalization, and the intermediate feature V _sa is the weighted sum of the Value vector using the corresponding attention weight. The calculation formula is as follows, V _sa =A·V; The pyramid generator G is used to take the final feature vector V as input, and the output is the generated point cloud corresponding to the previous one in 3 scales; The attention discriminator D is used to improve the performance of point cloud completion; the attention discriminator D is also an autoencoder structure. The encoder side uses a single-scale feature extractor composed of a feature embedding layer and an attention layer. The generated point cloud P1 and its corresponding real point cloud are sent to the discriminator D. After calculation by the feature extractor, a 640-dimensional feature vector is output, and then passed through continuous fully connected layers, and finally a binary output of false or true. 3. A multi-source three-dimensional data fusion system according to claim 1, characterized in that, specifically, the specific action process of the grid input module includes: The input data of the grid input module is a three-dimensional grid, and a three-dimensional convolution operation is performed on the three-dimensional grid to obtain a refined, higher-resolution three-dimensional grid. Since the order of patches in the three-dimensional data format is not fixed, when calculating the convolution results, the neighborhood mean and difference mean are used to make the calculation results independent of the patch order and satisfy the arrangement invariance. The convolution process is as follows: , w ₀ e _i represents the central feature, Represents the sum of neighborhood features,/> represents the sum of neighborhood differences, Represents the sum of the differences between center and field. 4. A multi-source three-dimensional data fusion system according to claim 1, characterized in that, specifically, the data fusion module includes: The point cloud encoder block P _enc and the mesh encoder block M _enc are used to encode the point cloud features P input by the point cloud input module and the mesh features M input by the mesh input module. The formula is as follows, P _enc (P)＝φ _P and M _enc (M)＝φ _M In the formula φ _P , The encoder blocks are respectively composed of fully connected layers f ₁ ^m ,/> Composition, where m∈{P, M}, and/> f ₁ ^m and f ₂ ^m are followed by batch normalization, ReLU and dropout layers respectively; The cross-attention block, used to share information between the point cloud representation φ _P and the mesh’s representation φ _M , consists of a multi-head self-attention layer, followed by a fully connected layer, using residual links for both layers, Then normalize, The residual link calculation method of the cross attention block P _proj and the subsequent output module M _proj is as shown in the formula: where θ _P , The output module consists of two linear layers/> and/> The sequence consists of , where/> After that are batch normalization, ReLU activation function and dropout layer.