WO2023151170A1 - Point cloud compression method and point cloud decompression method - Google Patents

Abstract

A point cloud compression method and a point cloud decompression method. The point cloud compression method may comprise: performing binary coding on the coordinates of each point of a point cloud by means of a tree structure, converting a binary code into a decimal code, and taking the decimal code as a node occupation code which corresponds to each node in the tree structure; moving a preset window for the nodes in the tree structure, and determining an uncompressed node in the preset window as the current node to be compressed; inputting context information of said current node into a pre-trained attention neural network model, and predicting a probability distribution of information to be compressed that corresponds to said current node; and inputting, into an arithmetic coder for entropy coding, said information and the probability distribution of said information that corresponds to said current node, so as to obtain a point cloud code stream, and taking the point cloud code stream as a compression result of the point cloud. Thus, the technical problems in the relevant art of a large calculation amount and a low compression efficiency are solved, and the technical effects of improving the compression effect and reducing the calculation amount are achieved.

Description

A Point Cloud Compression and Decompression Method

Cross References to Related Applications

This disclosure claims the priority of the Chinese patent application with application number 202210127272.2 and titled "A Method for Compressing and Decompressing Point Clouds" filed with the State Intellectual Property Office of China on February 11, 2022, the entire contents of which are hereby incorporated by reference In this disclosure.

technical field

The present disclosure relates to the technical field of point cloud data compression and decompression, in particular to a point cloud compression and decompression method.

Background technique

Point cloud data is an important data structure for 3D representation, so effective compression technology is very necessary for the storage and transmission of 3D point cloud. At present, the point cloud is first voxelized and an octree structure is established, and the context information of the voxel structure corresponding to each node of the octree structure is used to learn the features of the binary code using a 3D convolutional neural network, and then The occupancy states of the eight child nodes of the octree node are obtained, thereby realizing lossless compression of the octree.

However, using the context information of the voxel structure is likely to cause a technical problem of excessive calculation. In order to reduce the calculation amount, less context information of the voxel structure is necessary, and the use of less context information of the voxel structure is easy to produce Technical problems with inaccurate predictions and poor compression.

Contents of the invention

In view of this, the present disclosure at least provides a point cloud compression and decompression method, which can perform compression and decompression processing through the node occupancy code and attribute information of the node to be compressed, and when performing compression and decompression, the corresponding node to be compressed can be The node occupancy code and location information of the same layer node and parent node are used as the context information of the node to be compressed, which solves the technical problems of large amount of calculation and low compression efficiency in related technologies, and achieves the technology of improving the compression effect and reducing the amount of calculation Effect.

Some embodiments of the present disclosure provide a method for compressing a point cloud, wherein the compression method may include: performing binary encoding on point coordinates of the point cloud through a tree structure, converting the binary encoding into decimal encoding, as each point in the tree structure The node occupancy code corresponding to each node; move the preset window to the nodes in the tree structure, and determine the uncompressed node in the preset window as the current node to be compressed; input the context information of the current node to be compressed into the pre-trained In the attention neural network model, the probability distribution of the information to be compressed corresponding to the current node to be compressed is predicted; according to the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed, input to the arithmetic encoder for entropy encoding to obtain a point cloud Code stream, the point cloud code stream is used as the compression result of the point cloud.

Optionally, the tree structure may be an octree structure.

Optionally, the preset window can be configured to move according to the breadth-first principle; when moving the preset window, arrange the nodes of the tree structure into a breadth-first sequence according to the breadth-first principle, and use the front nodes in the breadth-first sequence as the nodes to be compressed Nodes corresponding to nodes at the same level until the number of nodes at the same level in the preset window meets the preset number; when the number of nodes at the same level in the preset window and/or the number of multi-layer parent nodes of nodes at the same level cannot meet, you can Supplement the nodes in the preset window with default nodes.

Optionally, the training step of the attention neural network model may include: building a tree structure from the sample point cloud, and determining the context information of the sample node and the information to be compressed of the node, wherein the sample point cloud is configured to be consistent with the test point cloud are different but similar; the context information of the sample node is input into the first layer of attention operation network of the attention neural network model, and the first weighted context matrix of the sample node is obtained; the first weighted context matrix is compared with the context information Adding, inputting the result of addition into the first layer of multi-layer perceptron network to obtain the second weighted context matrix; inputting the second weighted context matrix into the second layer of attention operation network to obtain the third weighted context matrix; Adding the third weighted context matrix to the second weighted context matrix, inputting the result of the addition into the second layer of multi-layer perceptron network, and obtaining the probability distribution of the information to be compressed corresponding to the predicted node to be compressed; multiple The probability distribution of the information to be compressed corresponding to the sample node and the information to be compressed are brought into the loss function to calculate the loss value; the optimization algorithm based on deep learning is used for backpropagation to optimize the loss value and update the weight of the attention neural network model ; Execute the above steps multiple times, and when the rate of change of the loss value output by the loss function reaches a preset threshold, a trained attention neural network model is obtained.

Optionally, the context information of the current node to be compressed may include: the multi-layer parent node of the same layer node corresponding to the current node to be compressed, the node occupancy code and location information of the previous node at the same layer, and the location information of the node to be compressed ;Using the multi-layer parent node of the same layer node corresponding to the current node to be compressed, the node occupancy code and location information of the previous node of the same layer, and the current location information of the node to be compressed, as the context information corresponding to the node to be compressed, which can include : Use the node occupancy code and location information of the previous node of the current node to be compressed and the multi-layer parent node corresponding to the previous node of the same layer as the first context information corresponding to the current node to be compressed; the current node to be compressed The location information, the node occupancy code of the previous compressed node at the same layer of the current node to be compressed, and the node occupancy code and location information of the multi-layer parent node corresponding to the current node to be compressed are used as the second context corresponding to the current node to be compressed Information, the first context information and the second context information are used as the context information corresponding to the current node to be compressed; the to-be-compressed information corresponding to the current node to be compressed may include: the node occupation code of the current node to be compressed.

Optionally, the context information of the current node to be compressed may also include: the node occupancy code and location information of the multi-layer parent node of the node at the same layer corresponding to the current node to be compressed, the node occupancy code and location information of the node at the same layer, the previous The node attributes and/or node attribute residuals corresponding to nodes of the same order; the to-be-compressed information corresponding to the current to-be-compressed node may also include: the node attributes and/or node attribute residuals of the current to-be-compressed node.

Optionally, the location information may include: node index, and/or node depth, and/or bounding box coordinates of the node.

Optionally, inputting the context information of the current node to be compressed into the pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed may include: inputting the context information of the current node to be compressed Input to the first layer of attention operation network of the attention neural network model to obtain the first weighted context matrix of the current node to be compressed; add the first weighted context matrix and context information, and input the result of the addition to the first In the layer multi-layer perceptron network, the second weighted context matrix is obtained; the second weighted context matrix is input into the second layer of attention operation network to obtain the third weighted context matrix; the third weighted context matrix is combined with the second weighted context Matrix addition, the result of the addition is input into the second-layer multi-layer perceptron network to obtain the fourth weighted context matrix; the fourth weighted context matrix is input into the third-layer multi-layer perceptron network to obtain the fourth Five weighted context matrices; the fifth weighted context matrix is passed through the softmax function to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

Optionally, the context information of the current node to be compressed is input into the first layer of attention operation network of the attention neural network model to obtain the first weighted context matrix of the current node to be compressed, which may include: the current node to be compressed The context information is input to the first multi-layer perceptron to obtain the first output matrix; the context information of the current node to be compressed is input to the second multi-layer perceptron to obtain the second output matrix; the context information of the current node to be compressed is obtained Input to the third multilayer perceptron to obtain the third output matrix; multiply the transpose matrix of the second output matrix with the first output matrix to obtain the inner product of the matrix; add the inner product of the matrix to the mask matrix, The added result is input to the softmax function to obtain the attention matrix; the attention matrix is multiplied by the third output matrix to obtain the first weighted context matrix of the current node to be compressed.

Optionally, each element in the attention matrix can be an attention value; the attention value of the attention matrix can be calculated by the following formula:

指的是第j个节点与第k个节点上下文之间的注意力值，第j个节点是同层节点中的第j个节点并且是当前待压缩节点，f _j是第j个节点的上下文信息，f _k是第k个节点的上下文信息，第k个节点是第j个节点的前序同层节点；分子

代表第j个节点与第k个节点上下文之间的相似值，分母

指的是第j个节点与第1个节点至第j个节点上下文之间的相似值的和；MLP ₂(f _j)指的是将第j个节点的上下文信息输入至第二多层感知机，得出第j个节点对应的第二输出矩阵；

指的是将第k个节点的上下文信息输入至第一多层感知机，得出第k个节点对应的第一输出矩阵的转置矩阵。 In formula (1),

Refers to the attention value between the jth node and the kth node context, the jth node is the jth node in the nodes of the same layer and is the current node to be compressed, f _j is the context of the jth node information, f _k is the context information of the kth node, and the kth node is the preorder node of the jth node;

Represents the similarity value between the jth node and the kth node context, denominator

Refers to the sum of the similarity values between the jth node and the context of the first node to the jth node; MLP ₂ (f _j ) refers to inputting the context information of the jth node into the second multi-layer perception machine to obtain the second output matrix corresponding to the jth node;

Refers to inputting the context information of the kth node into the first multi-layer perceptron to obtain the transpose matrix of the first output matrix corresponding to the kth node.

Optionally, inputting the context information of the current node to be compressed into the pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed may include: inputting the context information of the current node to be compressed Carry out dimension expansion to generate expanded dimension context information; input the expanded dimension context information into the pre-trained attention neural network model, and predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

Optionally, expanding the context information of the current node to be compressed to generate the expanded context information may include: first expanding the dimension of the context information of the current node to be compressed through a one-hot code operation to generate a one-hot code of the context information , and the one-hot code of the context information is generated through the embedding operation to expand the dimension context information.

Optionally, the point cloud compression method may include providing a point cloud compression device, which may include a first quantization module, a first determination module, a first prediction module, and a first compression module, wherein: the first quantization The module can be configured to binary code the coordinates of the point cloud through the tree structure, and convert the binary code into a decimal code as the node occupation code corresponding to each node in the tree structure; the first determination module can be configured as a tree structure The nodes in the structure move the preset window, and determine the uncompressed node in the preset window as the current node to be compressed; the first prediction module can be configured to input the context information of the current node to be compressed into the pre-trained attention In the neural network model, the probability distribution of the information to be compressed corresponding to the current node to be compressed is predicted; and the first compression module may be configured to input the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed to the arithmetic encoder Entropy encoding is performed to obtain the point cloud code stream, and the point cloud code stream is used as the compression result of the point cloud.

Other embodiments of the present disclosure also provide a method for decompressing a point cloud, which may include: moving a preset window for the tree structure, and determining the undecompressed nodes in the preset window each time as being currently to be decompressed Compress the node; input the context information of the current node to be decompressed into the pre-trained attention neural network model, predict the probability distribution of the information to be compressed corresponding to the current node to be decompressed; The probability distribution of the decompressed information and the point cloud code stream are input to the arithmetic decoder for entropy decoding, and the information to be decompressed corresponding to the node to be decompressed is obtained; the information to be decompressed is constructed into a tree structure, and the point cloud is obtained from the tree structure. Through inverse quantization, the decompressed point cloud is obtained.

Optionally, the point cloud decompression method may include providing a point cloud decompression device, the decompression device may include: a second determination module, a second prediction module, a first decompression module and a second quantization module, Wherein: the second determination module can be configured to move the preset window to the tree structure, and determine the uncompressed node in the preset window of each movement as the current node to be decompressed; the second prediction module can be configured to use the current The context information of the node to be decompressed is input into the pre-trained attention neural network model to predict the probability distribution of the information to be decompressed corresponding to the current node to be decompressed; the first decompression module can be configured to The probability distribution of the information to be decompressed corresponding to the compressed node and the point cloud code stream are input to the arithmetic decoder for entropy decoding to obtain the information to be decompressed corresponding to the current node to be decompressed; the second quantization module can be configured to convert the information to be decompressed The compressed information builds a tree structure, obtains the point cloud from the tree structure, and obtains the decompressed point cloud through dequantization.

Some other embodiments of the present disclosure also provide an electronic device, which may include: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory pass through The bus communicates, and the machine-readable instructions are executed by the processor to execute the steps of the point cloud compression method in any one of the above possible implementations, and/or the steps of the point cloud decompression method.

Some other embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program can be stored, and when the computer program is run by a processor, it can perform the points in any of the above possible implementation manners. The steps of the cloud compression method, and/or, the steps of the point cloud decompression method.

Optionally, the computer-readable storage medium may be a general-purpose storage medium capable of storing program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

Still other embodiments of the present disclosure provide a point cloud compression and decompression method. The point cloud compression method may be: binary code the point cloud through a tree structure to point coordinates, convert the binary code into a decimal code, As the node occupancy code corresponding to each node in the tree structure; move the preset window for the nodes in the tree structure, and determine the uncompressed node in the preset window as the current node to be compressed; input the context information of the current node to be compressed To the pre-trained attention neural network model, predict the probability distribution of the information to be compressed corresponding to the current node to be compressed; according to the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed, input to the arithmetic encoder for entropy Encode to get the point cloud code stream, and use the point cloud code stream as the compression result of the point cloud. This disclosure abandons the context information of the voxel structure, and compresses and decompresses the node to be compressed by using the node occupancy code and location information of the same layer node and the parent node corresponding to the current node to be compressed as the context information of the current node to be compressed At the same time, the peer nodes and parent nodes of the current node to be compressed are considered, which solves the technical problems of large calculation and low compression efficiency in related technologies, and achieves the technical effect of improving the compression effect and reducing the calculation.

In order to make the above-mentioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the accompanying drawings used in the embodiments will be briefly introduced below. It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

Fig. 1 shows a flowchart of a point cloud compression method provided by an embodiment of the present disclosure.

Fig. 2 shows a flowchart of a point cloud geometric compression method provided by an embodiment of the present disclosure.

Fig. 3 shows a flowchart of a point cloud attribute compression method provided by an embodiment of the present disclosure.

Fig. 4 shows a flow chart of a point cloud decompression method provided by an embodiment of the present disclosure.

Fig. 5 shows a flowchart of a method for geometrically decompressing a point cloud provided by an embodiment of the present disclosure.

Fig. 6 shows a flowchart of a point cloud attribute decompression method provided by an embodiment of the present disclosure.

FIG. 7 shows a schematic diagram of attribute completion provided by an embodiment of the present disclosure.

Fig. 8 shows a schematic diagram of a preset window provided by an embodiment of the present disclosure.

FIG. 9 shows a schematic diagram of an attention neural network model provided by an embodiment of the present disclosure.

Fig. 10 shows a schematic diagram of point cloud compression and decompression provided by an embodiment of the present disclosure.

Fig. 11 shows a schematic diagram of a point cloud compression device provided by an embodiment of the present disclosure.

Fig. 12 shows a schematic diagram of an apparatus for decompressing a point cloud provided by an embodiment of the present disclosure.

Fig. 13 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. It should be understood that the technical solutions in the embodiments of the present disclosure The accompanying drawings are for illustration and description purposes only, and are not intended to limit the protection scope of the present disclosure. Additionally, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this disclosure illustrate operations implemented in accordance with some embodiments of the disclosure. It should be understood that the operations of the flowcharts may be performed out of order, and steps that do not have a logical context may be performed in reverse order or simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart, or remove one or more operations from the flowchart under the guidance of the present disclosure.

In addition, the described embodiments are only some of the embodiments of the present disclosure, not all of them. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the present disclosure.

In related technologies, when compressing and decompressing point cloud data, the voxel structure corresponding to the node to be compressed is used as context information, resulting in technical problems such as excessive calculation and inaccurate prediction results, or, the same voxel structure of the node to be compressed is not Layer nodes are used as the context information of the nodes to be compressed, resulting in inaccurate prediction results, which in turn leads to technical problems of poor compression effect.

Based on this, an embodiment of the present disclosure provides a method for compressing and decompressing a point cloud. When compressing and decompressing a node to be compressed, the node occupancy code and location information of the node at the same layer and the parent node corresponding to the node to be compressed are used as The context information of the node to be compressed solves the technical problems of large amount of calculation and low compression efficiency in related technologies, and achieves the technical effect of improving the compression effect and reducing the amount of calculation. The optional methods are as follows:

Please refer to FIG. 1 . FIG. 1 is a flow chart of a point cloud compression method provided by an embodiment of the present disclosure. As shown in Figure 1, a method for compressing a point cloud provided by an embodiment of the present disclosure may include the following steps:

S101. Perform binary encoding on the point coordinates of the point cloud through the tree structure, convert the binary encoding into decimal encoding, and use it as a node occupation code corresponding to each node in the tree structure.

Exemplarily, before the point cloud is binary-coded on the coordinates of the point through the tree structure, and the binary code is converted into a decimal code as the node occupancy code corresponding to each node in the tree structure, the method may include quantifying the point cloud . Quantifying the point cloud can be understood as approximating the discrete values of a large number of possible coordinates or attributes of the point cloud to fewer discrete values. In order to build a tree structure for the quantized point cloud in the future. In lossless compressed embodiments, quantization may not be performed. In an embodiment provided by the present disclosure, the tree structure may be an octree structure.

When quantifying the point cloud, it is necessary to perform geometric quantization and attribute quantization on the point cloud. When performing geometric quantization (that is, coordinate quantization), the minimum three-dimensional coordinate value (that is, the three-dimensional coordinate value of the point in the lower right corner, referring to accompanying drawing 7) in the point cloud can be determined, and the three-dimensional coordinates of all points in the point cloud The difference between the value and the minimum three-dimensional coordinate value (that is, the point cloud data is shifted so that the point in the lower right corner of the shifted point cloud data is at the origin), and then the difference is compared with the quantization step and rounded, and the rounded The adjusted ratio is used as the three-dimensional coordinate value of each point in the geometrically quantized point cloud data. When performing attribute quantization (that is, color (RGB value or YUV value) and/or reflectance quantization), the original point cloud attribute can be compared with the quantization step size and then rounded, and the rounded ratio can be used as attribute quantization The node attributes of each point of the point cloud.

Among them, the quantization step size (qs) is a user-adjustable parameter, and its value depends on the desired compression degree and acceptable distortion. The larger the quantization step size, the larger the distortion of the point cloud data, but the smaller the bit rate. This also means that our model can be applied to the compression of variable bit rates (the same model, achieving different compression sizes and distortions). Wherein, the quantization step size of geometric quantization and attribute quantization may be different.

The node occupancy code refers to the arrangement position of each point in the point cloud, and the eight-bit binary code of the arrangement position of each point is converted into a decimal code, and the decimal code is used as the node occupancy code of this node.

For example, please refer to FIG. 7 , which shows a schematic diagram of attribute completion provided by an embodiment of the present disclosure. In FIG. 7 , the voxel indicated by the node corresponding to the node occupancy code of 73 includes eight sub-voxels, and the sub-voxels are numbered, and the number ranges from number 0 to number 7. Among them, the sub-voxel existence points where No. 1, No. 4 and No. 7 are located have a corresponding binary code of 01001001 (corresponding to 0a00b00c in Figure 7), which is converted to 73 in decimal, and 73 is the node occupancy code of this node.

S102. Move the preset window to the nodes in the tree structure, and determine the uncompressed node in the preset window as the current node to be compressed.

The compression of point clouds can be divided into geometry compression and attribute compression. When performing geometry compression, it corresponds to the geometry preset window.

For example, please refer to FIG. 8 , which shows a schematic diagram of a preset window provided by an embodiment of the present disclosure. In Figure 8, the " _xi " node refers to the current node to be compressed, N ₁ refers to the number of nodes in the same layer corresponding to the node to be compressed in the geometry preset window; N ₂ refers to the number of nodes to be compressed in the attribute preset window The number of nodes on the same layer corresponding to the compressed node; W1 refers to the geometric preset window with the number of nodes on the same layer (N ₁ ) of the current node to be compressed is 8 and the layer height is 4; W2 refers to the same layer of the node to be compressed currently The attribute preset window with the number of nodes (N ₂ ) being 7 and the layer height being 1.

S103. Input the context information of the current node to be compressed into the pre-trained attention neural network model, and predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

Wherein, the preset window is configured to move according to the breadth-first principle.

When moving the preset window, the nodes of the tree structure can be arranged in a breadth-first sequence according to the breadth-first principle, and the nodes in the front of the breadth-first sequence are used as the same-layer nodes corresponding to the nodes to be compressed until the number of same-layer nodes in the preset window satisfies Preset quantity (N1 or N2). When the number of same-level nodes in the preset window and/or the number of multi-layer parent nodes of the same-level nodes cannot be satisfied (that is, there are no more nodes in the tree structure), default nodes can be used to supplement the preset window node, which occurs in some layers of the tree structure near the root.

The peer nodes corresponding to the current node to be compressed may include the current node to be compressed, the previous peer node of the current node to be compressed, and/or the subsequent peer node of the current node to be compressed.

In a preferred embodiment, when performing geometric compression, the context information of the current node to be compressed (that is, the geometric context information of the current compressed node) may include: the multi-layer parent node of the same layer node corresponding to the current node to be compressed, the preamble The node occupancy code and location information of nodes in the same layer, and the location information of the node to be compressed.

The context information corresponding to the current node to be compressed does not include the node occupancy code of the current node to be compressed, because the purpose of decompression is to decode the node occupancy code of the current node to be compressed, that is, the node occupancy of the current node to be compressed during decompression The code is unknown, so it does not consider its own node occupation code when compressing.

The node occupancy code and location information of the multi-layer parent node corresponding to the previous node of the current node to be compressed and the corresponding multi-layer parent node can be used as the first context information corresponding to the node to be compressed; the location information of the node to be compressed , the node occupancy code of the previous compressed node at the same layer of the node to be compressed, and the node occupancy code and location information of the multi-layer parent node corresponding to the node to be compressed, as the second context information corresponding to the node to be compressed, the first The context information and the second context information serve as context information corresponding to the node to be compressed.

Wherein, the location information may include: node index, and/or node depth, and/or bounding box coordinates of the node. The node index refers to the position of the node in the sibling nodes of the same parent node, and the range of the node index is 0 to 7; the node depth refers to the depth of the octree structure of the node, and the root node in the octree structure The depth is 1. The bounding box refers to the cubic space of the voxel corresponding to the node, and the bounding box coordinates refer to the coordinates of the two vertices of the diagonal diagonal of the cubic space. The diagonal directions of multiple bounding boxes corresponding to the same point cloud are the same , as shown in FIG. 8 , (x ₀ , y ₀ , z ₀ , x ₁ , y ₁ , z ₁ ) are the bounding box coordinates of the voxel corresponding to the node occupation code 73.

Exemplarily, returning to FIG. 8 , in FIG. 8 , a node whose node occupation code is 9 has a node depth of 1, a node index of 7, and node bounding box coordinates of (0,0,0,7,7,7).

When performing attribute compression, the context information of the current node to be compressed can also include: the node occupancy code and location information of the multi-layer parent node of the same layer node corresponding to the current node to be compressed, and the node occupancy code and location information of the same layer node , the node attribute and/or the node attribute residual corresponding to the previous node at the same layer.

In a preferred embodiment, when attribute compression is performed, the context information of the current node to be compressed (that is, the node attribute (node attribute residual) context information of the current compressed node) may include: the node at the same layer corresponding to the current node to be compressed Node occupancy codes and location information of multi-layer parent nodes, previous nodes of the same layer, subsequent nodes of the same layer, nodes to be compressed, and node attributes and/or node attribute residuals corresponding to the previous nodes of the same layer.

Optionally, the node occupancy code and location information of the previous node of the current node to be compressed and the multi-layer parent node corresponding to the previous node of the same layer are used as the first context information corresponding to the node to be compressed; the current node to be compressed The node occupancy code and location information of the multi-layer parent node corresponding to the subsequent node of the same layer and the subsequent node of the same layer are used as the second context information corresponding to the node to be compressed; the node occupancy code and location information of the node to be compressed, And the node occupancy code and location information of the multi-layer parent node corresponding to the node to be compressed, as the third context information corresponding to the node to be compressed; The difference is used as the fourth context information corresponding to the node to be compressed; the first context information, the second context information, the third context information and the fourth context information are used as the context information corresponding to the node to be compressed.

Wherein, when attribute compression is performed, the node at the same layer corresponding to the current node to be compressed may be the deepest leaf node in the tree structure, that is, a node without child nodes in the tree structure. Only leaf nodes have node attributes and/or node attribute residuals.

The node attribute can be the 3-bit RGB value or YUV value or reflectivity or other attributes of the node, and the node attribute residual can be the attribute coding difference between this node and the previous node after the attributes of the node are completed. Wherein, the value range of the three primary color channels of the 3-bit RGB value can be 0-255, that is, the node attribute can be a 3-bit RGB value. Attribute completion can be for the convenience of constructing context and calculation of neural network, and the supplementary attribute value (residual value of attribute value) does not enter into entropy coding.

For example, the attribute completion method of a node, please refer to FIG. 7 , and FIG. 7 shows a schematic diagram of attribute completion provided by an embodiment of the present disclosure. As shown in Figure 7, for example, in terms of the R attribute in the color, among the nodes corresponding to the node occupancy code 73, the node attribute value corresponding to the sub-voxel numbered 1 is a, and the node attribute value corresponding to the sub-voxel numbered 4 The value is b, the attribute value of the node corresponding to the sub-voxel numbered 7 is c, then the attribute code corresponding to the node corresponding to the node occupancy code 73 is 0a00b00c, and 0 means that there is no point cloud point in the sub-voxel. Circular left-shift completion of the attribute code, the first circular left-shift completion is aa0bb0cc, the attribute code after the first attribute completion still has an attribute code of zero, then the second circular left-shift completion is aabbbccc , the attribute code after the second attribute completion has no zero code, then the attribute completion is completed, and the attribute code after attribute completion is aabbbccc. If the attribute code of the preceding node attribute completion is dddeeedd, the node occupation code is 73, and the corresponding node attribute residual is: (d-a)(d-a)(d-b)(e-b)(e-b)(e-c)( d-c)(d-c).

When performing geometric compression, the information to be compressed corresponding to the current node to be compressed may include: the node occupancy code of the current node to be compressed; when performing attribute compression, the information to be compressed corresponding to the current node to be compressed may include: the current node to be compressed Node attributes and/or node attribute residuals for nodes.

When performing geometric compression, the i-th node to be compressed can be determined as the current node to be compressed, and the context information of the i-th node to be compressed is input into the pre-trained attention neural network model to predict the i-th node to be compressed Probability distribution q _i (x) of node occupancy codes corresponding to compressed nodes, that is, q _i (x) is the probability distribution of node occupancy codes of the i-th node to be compressed being 1-255. For example, the probability distribution corresponding to the current node to be compressed is (0.01, 0.2, 0.056, ..., 0.36), which means that the probability of the node occupancy code of the current node to be compressed is 1 is 0.01, and the probability of being 2 is 0.2, which is 3 The probability of is 0.056, ..., the probability of being 255 is 0.36), and the sum of the probability distributions is 1.

即，

为第i个待压缩节点的第n _A个节点属性和/或节点属性残差的概率分布。 When attribute compression is performed, the node attribute and/or node attribute residual probability distribution corresponding to the current node to be compressed can be predicted

Right now,

is the probability distribution of the nth node attribute and/or node attribute residual of the i- _th node to be compressed.

The steps of training the attention neural network model can be: Step 1, construct a tree structure from the sample point cloud, and determine the context information of the sample node and the information to be compressed of the node (wherein, the sample point cloud is different from the point cloud of the test , but the sample point cloud is similar to the test point cloud, for example, it is a different frame of the same sequence point cloud or a similar point cloud); Step 2, input the context information of the sample node into the first node of the attention neural network model In the one-layer attention operation network, the first weighted context matrix of the sample node is obtained; Step 3, the first weighted context matrix is added to the context information, and the result of the addition is input into the first layer of multi-layer perceptron network, Obtain the second weighted context matrix; Step 4, input the second weighted context matrix into the second layer attention operation network to obtain the third weighted context matrix; Step 5, combine the third weighted context matrix with the second weighted context matrix Add, input the result of addition to the second layer of multi-layer perceptron network, and obtain the probability distribution of the information to be compressed corresponding to the predicted node to be compressed; Step 6: the probability of the information to be compressed corresponding to the multiple sample nodes The distribution and information to be compressed are brought into the loss function to calculate the loss value; among them, the loss function of geometric compression can be:

loss ₁ ＝-∑ _i log ₂ q _i (x _i |f; w ₁ ) (2)

的属性残差(即，本公开实施中属性补全之前的属性0a00b00c的J＝{1,4,7})，属性压缩的损失函数为： In formula (2), loss ₁ refers to the loss function of geometric compression, x _i refers to the current node to be compressed (i-th node to be compressed), and f refers to the i-th node to be compressed after dimension expansion context information, w ₁ refers to the weight of the geometric compression attention neural network, q _i refers to the probability distribution of the node occupancy code corresponding to the ith node to be compressed, i refers to the range of all nodes to be compressed, ∑ _i log ₂ q _i (xi | _f ; w ₁ ) refers to the sum of the estimated code rates corresponding to all nodes to be compressed; the loss function of attribute compression is designed to only calculate point cloud points where attributes exist

Attribute residual (that is, J={1,4,7} of attribute 0a00b00c before attribute completion in the implementation of the present disclosure), the loss function of attribute compression is:

是每个属性在属性码流中的权重，n _A指的是第n _A个属性(即，RGB值中的第几个属性，例如R是RGB值中的第1个属性)，N _A指的是节点属性的总个数(例如RGB值为3个属性)，i指代的范围是全部的待压缩节点(即，属性压缩时，是全部的叶节点)。步骤七：使用基于深度学习的优化算法进行反向传播，来优化损失值，更新注意力神经网络模型的权重w ₁和/或w ₂。可以多次执行步骤一至步骤七，当损失函数输出的损失值的变化率达到预置阈值时(即，损失函数的输出值收敛)，则得到训练好的注意力神经网络模型。 Among them, in formula (3), loss ₂ refers to the loss function of attribute compression, w ₂ refers to the weight of attribute compression attention neural network,

is the weight of each attribute in the attribute code stream, n _A refers to the n _Ath attribute (that is, which attribute in the RGB value, for example, R is the first attribute in the RGB value), N _A refers to is the total number of node attributes (for example, the RGB value is 3 attributes), and the range referred to by i is all the nodes to be compressed (that is, when the attribute is compressed, it is all the leaf nodes). Step 7: Use an optimization algorithm based on deep learning to perform backpropagation to optimize the loss value and update the weight w ₁ and/or w ₂ of the attention neural network model. Steps 1 to 7 can be performed multiple times. When the rate of change of the loss value output by the loss function reaches a preset threshold (that is, the output value of the loss function converges), a trained attention neural network model is obtained.

Input the context information of the current node to be compressed into the pre-trained attention neural network model, and predict the probability distribution of the information to be compressed corresponding to the current node to be compressed, which may include: expanding the context information of the current node to be compressed, Generate the expanded dimension context information; input the expanded dimension context information into the pre-trained attention neural network model, and predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

In the present disclosure, the context information of the current node to be compressed can be expanded through one-hot code operation to generate the one-hot code of the context information, and then the one-hot code of the context information can be embedded to generate the expanded context information.

One-hot code operation (one-hot code) is a code system in which there are as many bits as there are states, and only one bit is 1, and the others are all 0. For example, if the node index of a certain node is 1, its one-hot code is (0,1,0,0,0,0,0,0), which expands the one-dimensional data into eight dimensions. The embedding operation (Embedding) is to linearly map the result of the one-hot encoding operation to a preset dimension.

Optionally, inputting the context information of the current node to be compressed into the pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed may include:

Input the context information of the current node to be compressed into the first layer of attention operation network of the attention neural network model to obtain the first weighted context matrix of the current node to be compressed; add the first weighted context matrix and the context information, and The result of the addition is input to the first layer of multi-layer perceptron network to obtain the second weighted context matrix; then the second weighted context matrix is input to the second layer of attention operation network to obtain the third weighted context matrix; the second weighted context matrix is obtained The three-weighted context matrix is added to the second weighted context matrix, and the result of the addition is input into the second layer multi-layer perceptron network to obtain the third weighted context matrix; the third weighted context matrix is input to the third layer multi-layer perceptron network In the layer perceptron network, the fourth weighted context matrix is obtained; the fourth weighted context matrix is input into the third layer multi-layer perceptron network, and the fifth weighted context matrix is obtained; the fifth weighted context matrix is passed through the softmax function, The probability distribution of the information to be compressed corresponding to the current node to be compressed is predicted.

For a preferred embodiment, please refer to FIG. 9 which shows a schematic diagram of an attention neural network model provided by an embodiment of the present disclosure. In Figure 9, N refers to the number of nodes on the same layer corresponding to the current node to be compressed (the geometry preset window is N ₁ , the attribute preset window is N ₂ , and N ₁ =N ₂ =N by default when performing attribute compression) ; H refers to the layer height of the preset window (the layer height of the geometry preset window is K, and the layer height of the attribute preset window is 1), and C refers to the dimension of the context information (the dimension corresponding to the geometry compression is C _G , the dimension corresponding to attribute compression is C _A +K×C _G ); H×C refers to the dimension of the context information in the preset window; N ₀ refers to the number of nodes to be compressed; C ₀ refers to the nodes to be compressed The dimension of the corresponding probability distribution (the dimension corresponding to geometric compression is 255, and the dimension of attribute compression such as RGB is 3×256×8 (the attribute code of each leaf node has 8 bits)).

Optionally, input the context information (N, H, C) of the current node to be compressed into the first layer of attention operation network to obtain the first weighted context matrix (N, H×C); then the first The weighted context matrix is added to the context information, and the added result is input to the first layer of multi-layer perceptron to obtain the second weighted context matrix (N, H×C); then the second weighted context matrix is input to the second In the two-layer attention operation network, the third weighted context matrix (N, H×C) is obtained; the third weighted context matrix is added to the second weighted context matrix, and the result of the addition is input to the second layer of multi-layer perception In the computer network, the fourth weighted context matrix (N, H×C) is obtained, and the fourth weighted context matrix is input into the third layer multi-layer perceptron network, and the fifth weighted context matrix (N, C ₀ ) is obtained; The fifth weighted context matrix is passed through the softmax function to predict the probability distribution (N ₀ , C ₀ ) of the information to be compressed corresponding to the current node to be compressed.

Input the context information of the current node to be compressed into the first layer of attention operation network of the attention neural network model to obtain the first weighted context matrix of the current node to be compressed, which may include:

Input the context information of the current node to be compressed to the first multi-layer perceptron to obtain the first output matrix; input the context information of the current node to be compressed to the second multi-layer perceptron to obtain the second output matrix; the current The context information of the node to be compressed is input to the third multi-layer perceptron to obtain the third output matrix; the transpose matrix of the second output matrix is multiplied by the first output matrix to obtain the inner product of the matrix; the inner product of the matrix and The mask matrix is added, and the result of the addition is input to the softmax function to obtain the attention matrix; the attention matrix is multiplied by the third output matrix to obtain the first weighted context matrix of the current node to be compressed.

求和的长度不同。例如，当第j个节点为当前待压缩节点时，第j+1以后的节点的上下文信息 将不存在于第j个节点的结果中。因此，在一次计算中便可以根据输出的N个结果，通过截断得出后N ₀个结果，作为后N ₀个待压缩节点的概率分布，以减少网络传播次数，加快压缩速度。 Among them, the mask matrix (matrix with -∞ in the upper right corner and 0 in the lower left corner) can be used to predict the probability distribution corresponding to the current node to be compressed, only consider the compressed nodes in front of the current node to be compressed, and then lead to the attention value denominator

The sums are of different lengths. For example, when the jth node is the current node to be compressed, the context information of the j+1th node will not exist in the result of the jth node. Therefore, in one calculation, according to the output N results, the last N ₀ results can be obtained by truncation as the probability distribution of the last N ₀ nodes to be compressed, so as to reduce the number of network propagation and speed up the compression speed.

Among them, each element in the attention matrix is an attention value, and the attention value of the attention matrix can be calculated by the following formula:

指的是第j个节点与第k个节点上下文之间的注意力值，第j个节点是同层节点中的第j个节点并且是当前待压缩节点，f _j是第j个节点的上下文信息，f _k是第k个节点的上下文信息，第k个节点是第j个节点的前序同层节点；分子

代表第j个节点与第k个节点上下文之间的相似值，分母

指的是第j个节点与第1个节点至第j个节点上下文之间的相似值的和；MLP ₂(f _j)指的是将第j个节点的上下文信息输入至第二多层感知机，得出第j个节点对应的第二输出矩阵；

指的是将第k个节点的上下文信息输入至第一多层感知机，得出第k个节点对应的第一输出矩阵的转置矩阵。 In formula (1),

Refers to the attention value between the jth node and the kth node context, the jth node is the jth node in the nodes of the same layer and is the current node to be compressed, f _j is the context of the jth node information, f _k is the context information of the kth node, and the kth node is the preorder node of the jth node;

Represents the similarity value between the jth node and the kth node context, denominator

Refers to the sum of the similarity values between the jth node and the context of the first node to the jth node; MLP ₂ (f _j ) refers to inputting the context information of the jth node into the second multi-layer perception machine to obtain the second output matrix corresponding to the jth node;

Refers to inputting the context information of the kth node into the first multi-layer perceptron to obtain the transpose matrix of the first output matrix corresponding to the kth node.

That is, the kth node is located between the 1st node to the jth node.

S104. According to the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed, input to an arithmetic encoder for entropy encoding to obtain a point cloud code stream, and use the point cloud code stream as a point cloud compression result.

That is, the probability distribution of information to be compressed and the information to be compressed corresponding to all nodes to be compressed are input to the arithmetic encoder for entropy encoding to obtain a point cloud code stream, and the point cloud code stream is used as the compression result of the point cloud. Among them, according to the object of point cloud compression, the point cloud code stream can be divided into geometric code stream and attribute code stream, that is, when geometric compression is performed, the compressed geometric code stream is obtained; when attribute compression is performed, the compressed object is attribute stream.

The arithmetic encoder converts the probability distribution corresponding to the input node to be compressed and the information to be compressed into a decimal less than 1, and the storage method of this decimal is binary, which is the generated point cloud code stream. The point cloud code stream is a series of compressed binary bit streams. For example, a point cloud file with an original size of 1.94MB (2038480 bytes) can be compressed into a file with a size of 55.5KB (56890 bytes), saving 97.2% of storage space.

For a preferred embodiment, please refer to FIG. 2 . FIG. 2 shows a flow chart of a point cloud geometric compression method provided by an embodiment of the present disclosure. Optionally, the steps of a point cloud geometric compression method are as follows:

S201. Obtain geometric context information corresponding to the current node to be compressed, and obtain N ₁ ×K×3-dimensional geometric context information.

Geometric context information corresponding to the current node to be compressed can be acquired to obtain N ₁ ×K×3-dimensional geometric context information. Among them, N ₁ nodes refer to the number of nodes on the same layer as the current node to be compressed in the geometry preset window, K refers to the height of the geometry preset window, and 3 refers to the node occupancy code, node depth, and node index. That is to say, the _N1 nodes include the current node to be compressed.

S202. Dimensionally expand the N ₁ ×K×3 dimensional geometric context information through one-hot encoding and embedding operations to generate expanded dimension geometric context information, which is N ₁ ×K×C _G dimensional expanded geometric context information.

In the embodiment of the present disclosure, the range of the node occupancy code can be 1-255, and then the node occupancy code can be 255 dimensions after the one-hot code operation; the range of the node depth can be 1-16, and then the node index can be one-hot The code operation can be 16-dimensional; the range of node index can be 0-7, and the node depth can be 8-dimensional after one-hot code operation. Then the 255-dimensional node occupancy code is converted to 128-dimensional by embedding operation, the 16-dimensional node depth is converted to 6-dimensional by embedding operation, and the 8-dimensional node index is converted to 4-dimensional by embedding operation. Therefore, after dimension expansion through one-hot encoding operation and embedding operation, the 3-dimensional data of geometric context information can be converted into _CG- dimensional (138 dimensions in total, which is the sum of 128-, 6-, and 4-dimensional).

S203. Input the N ₁ ×K×C _G dimension expanded geometric context information into the trained attention neural network model, and predict the probability distribution of the node occupancy code corresponding to the current node to be compressed.

S204 , judging whether the probability distributions of node occupation codes corresponding to all the nodes to be compressed have been obtained.

If the probability distribution of the node occupancy codes corresponding to all the nodes to be compressed has not been obtained, it can return to step S201; if the probability distribution of the node occupancy codes corresponding to all the nodes to be compressed is obtained, the node The probability distribution of the occupancy code and the node occupancy code of the node to be compressed are input to the arithmetic encoder for entropy encoding to obtain the geometric code stream, which is used as the geometric compression result of the point cloud.

For a preferred embodiment, please refer to FIG. 3 . FIG. 3 shows a flow chart of a point cloud attribute compression method provided by an embodiment of the present disclosure. Optionally, the steps of a point cloud attribute compression method are as follows:

S301. Obtain context information of node attributes (node attribute residuals) corresponding to the current node to be compressed, and obtain N ₂ ×8×N _A- dimensional node attribute (node attribute residuals) context information.

The node attribute context information corresponding to the current to-be-compressed node can be obtained, and N ₂ ×8×N _A- dimensional attribute context information can be obtained. Among them, N ₂ nodes refer to the number of nodes in the same layer corresponding to the current node to be compressed in the attribute preset window, and 8 refers to the number of digits of the completed attribute code included in each node (that is, the above mentioned The node occupancy code is 73 and the corresponding attribute code is aabbbccc after completion), N _A refers to the number of node attributes (that is, 3-bit RGB value or 1-bit reflectance value).

S302. Expand the dimensionality of the _N2 ×8×N _A- dimensional node attribute (node attribute residual) context information through one-hot code operation and embedding operation, and generate the expanded dimension node attribute (node attribute residual) context information, which is N ₂ ×C _A -dimensional expanded dimension node attribute (node attribute residual) context information.

The node attributes in the embodiments of the present disclosure can use 3-bit RGB values, and the N _A dimension corresponds to red values, blue values, and green values. The range of red value, blue value and green value can be 0-255. Therefore, the red value is converted into 256 dimensions after one-hot encoding operation, and the blue value is converted into 256 dimensions after one-hot encoding operation. The green value is converted to 256 dimensions after one-hot encoding operation. Then the 256-dimensional red value is converted to 128 dimensions through the embedding operation, the 256-dimensional blue value is converted to 128 dimensions through the embedding operation, and the 256-dimensional green value is converted to 128 dimensions through the embedding operation. Therefore, after dimension expansion through one-hot encoding operation and embedding operation, the N _A dimension data of attribute context information can be converted into C _A dimension (384 dimensions in total, which are the sum of 128 dimensions, 128 dimensions, and 128 dimensions).

S303. Input the N ₂ ×C _A dimension expanded dimension node attribute (node attribute residual) context information and the N ₂ ×K×C _G dimension expanded dimension geometric context information into the trained attention neural network model, Predict the probability distribution of the node attributes (node attribute residuals) corresponding to the current node to be compressed.

That is to say, geometric context information is also required during attribute compression, and the geometric window corresponding to the geometric context information required during attribute compression may be different from the corresponding geometric window during geometry compression.

S304. Judging whether the probability distributions of node attributes (node attribute residuals) corresponding to all the nodes to be compressed are obtained.

If the probability distribution of the node attributes (node attribute residuals) corresponding to all the nodes to be compressed has not been obtained, it can return to step S301; if the probability distributions of the node attributes (node attribute residuals) corresponding to all the nodes to be compressed are obtained, Then the probability distribution of the node attributes (node attribute residuals) corresponding to all nodes to be compressed and the node attributes (node attribute residuals) of the nodes to be compressed can be input into the arithmetic encoder for entropy encoding to obtain the attribute code stream, and the attribute The code stream is the attribute compression result of the point cloud.

Based on the same application idea, the embodiment of the present disclosure also provides a point cloud decompression method corresponding to the point cloud compression method provided in the above embodiment, because the point cloud decompression method in the embodiment of the present disclosure solves the problem The principle is similar to the point cloud compression method in the above-mentioned embodiments of the present disclosure, so the implementation of the point cloud decompression can refer to the implementation of the point cloud compression method, and the repetition will not be repeated. Please refer to FIG. 10 , which shows a schematic diagram of point cloud compression and decompression provided by an embodiment of the present disclosure. The geometry prediction network and attribute prediction network in FIG. 10 are the attention neural network in the embodiment of the present disclosure.

Referring to FIG. 4 , FIG. 4 shows a flow chart of a point cloud decompression method provided by an embodiment of the present disclosure. The steps of a point cloud decompression method are as follows:

S401. Move the preset window for the tree structure, and determine the uncompressed node in each moved preset window as the current node to be decompressed.

Point cloud decompression can be divided into geometry decompression and attribute decompression.

S402. Input the context information of the current node to be decompressed into the pre-trained attention neural network model, and predict the probability distribution of the information to be decompressed corresponding to the current node to be decompressed.

When performing geometric decompression, the multi-layer parent node of the same layer node corresponding to the current node to be decompressed, the node occupancy code and location information of the previous node at the same layer, and the location information of the current node to be decompressed can be used as the current Context information corresponding to the node to be decompressed.

When performing attribute decompression, the node occupancy code and location information of the multi-layer parent node corresponding to the current node to be compressed, the node occupancy code and location information of the node at the same layer, and the node corresponding to the previous node at the same layer Attributes and/or node attribute residuals are used as context information corresponding to the nodes to be decompressed.

The attention neural network for point cloud decompression is the same as the attention neural network for the point cloud compression method, and the specific generation method of the attention neural network will not be repeated here.

Optionally, since the compressed and decompressed attention neural network models are the same, and the input context information is also the same, the probability distribution corresponding to the predicted node to be decompressed obtained during compression is the same as the predicted node to be decompressed obtained during decompression. The probability distribution corresponding to the decompression node is the same.

S403. Input the probability distribution of the information to be decompressed corresponding to the current node to be decompressed and the point cloud code stream to the arithmetic decoder for entropy decoding, and obtain the information to be decompressed corresponding to the current node to be decompressed.

When performing geometric decompression, the information to be decompressed refers to the node occupancy code, and the point cloud code stream refers to the geometric code stream; when performing attribute decompression, the information to be decompressed refers to node attributes and/or node attributes The residual, the attribute code stream refers to the attribute code stream.

S404. Build a tree structure for the information to be decompressed, obtain a point cloud from the tree structure, and then obtain a decompressed point cloud through dequantization.

The information to be decompressed can be gradually decompressed to construct a tree structure, and the point cloud can be obtained from the tree structure, and then decompressed to obtain the decompressed point cloud.

For a preferred embodiment, please refer to FIG. 5 . FIG. 5 shows a flow chart of a point cloud geometric decompression method provided by an embodiment of the present disclosure. Optionally, the steps of a geometric decompression method of a point cloud are as follows:

S501. Obtain geometric context information corresponding to the current node to be decompressed, and obtain N ₁ ×K×3-dimensional geometric context information.

S502. Dimensionally expand the N ₁ ×K×3-dimensional geometric context information through a one-hot encoding operation and an embedding operation to generate expanded geometric context information, which is N ₁ ×K×C _G -dimensional expanded geometric context information.

S503. Input the expanded geometric context information of N ₁ ×K×C _G dimensions into the trained attention neural network model, and predict the probability distribution of the node occupancy code of the current node to be decompressed.

S504. Input the probability distribution and the geometric code stream of the node occupancy code of the current node to be decompressed to the arithmetic decoder for entropy decoding, and decode the node occupancy code of the current node to be decompressed.

S505. Determine whether the node occupation codes of all nodes to be decompressed have been obtained.

If the node occupancy codes of all the nodes to be decompressed have not been obtained, it may return to step S501.

If the node occupancy codes of all the nodes to be decompressed are obtained (that is, all the nodes to be decompressed are decompressed), a tree structure may be constructed from the node occupancy codes obtained by decoding step by step. The quantized point cloud can be obtained from the complete tree structure constructed, and the quantized point cloud can be dequantized to obtain the coordinates of the decompressed point cloud.

For a preferred embodiment, please refer to FIG. 6 . FIG. 6 shows a flow chart of a method for decompressing attributes of a point cloud provided by an embodiment of the present disclosure. Optionally, the steps of a point cloud attribute decompression method are as follows:

S601. Obtain context information of node attributes (node attribute residuals) corresponding to the current node to be decompressed, and obtain N ₂ ×8×N _A- dimensional node attribute (node attribute residuals) context information.

S602. Dimensionally expand the N ₂ × 8 × N _A -dimensional node attribute (node attribute residual) context information through one-hot code operation and embedding operation, and generate the expanded dimension node attribute (node attribute residual) context information, which is N ₂ ×C _A -dimensional expanded dimension node attribute (node attribute residual) context information.

S603. Input the N ₂ ×C _A -dimensional expanded dimension node attribute (node attribute residual) context information and the N ₂ ×K×C _G- dimensional expanded dimension geometric context information into the trained attention neural network model, Predict the probability distribution of the node attributes (node attribute residuals) corresponding to the current node to be decompressed.

S604. Input the probability distribution and attribute code stream of the node attribute (node attribute residual) of the node to be decompressed to the arithmetic decoder for entropy decoding, and decode the node attribute (node attribute residual) of the node to be decompressed.

S605. Determine whether the node attributes (node attribute residuals) of all nodes to be decompressed have been acquired.

If the node attributes (node attribute residuals) of all the nodes to be decompressed have not been obtained, it may return to step S601.

If the acquisition of the node attributes (node attribute residuals) of all nodes to be decompressed is completed, the node attributes (node attribute residuals) of the nodes to be decompressed that are gradually decompressed can be used to construct the node attributes (node attribute residuals) of the leaf nodes of the tree structure. residuals). The node attributes (node attribute residuals) can be dequantized to reconstruct the point cloud attributes.

The attention neural network mentioned in this embodiment is a framework of deep learning, referring to the operation described in Figure 10 and formula (1). Similar networks named "Transformer" all belong to the category of attention neural network described in this disclosure.

The embodiment of the present disclosure also provides a point cloud compression device corresponding to the point cloud compression method provided in the above embodiment, because the principle of solving the problem of the point cloud compression device in the embodiment of the present disclosure is the same as that of the above embodiment of the present disclosure The point cloud compression method is similar, so the implementation of the point cloud compression device can refer to the implementation of the point cloud compression method, and the repetition will not be repeated.

Please refer to FIG. 11 , which shows a schematic diagram of a point cloud compression device provided by an embodiment of the present disclosure. The point cloud compression device 100 may include a first quantization module 101 , a first determination module 102 , a first prediction module 103 and a first compression module 104 . Wherein, the first quantization module 101 can be configured to perform binary coding on the point coordinates of the point cloud through a tree structure, and convert the binary coding into decimal coding as the node occupancy code corresponding to each node in the tree structure; the first determination Module 102 can be configured to move the preset window to the nodes in the tree structure, and determine the uncompressed node in the preset window as the current node to be compressed; the first prediction module 103 can be configured to move the current node to be compressed Input the context information into the pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed; the first compression module 104 can be configured to use the information to be compressed corresponding to the node to be compressed The probability distribution and the information to be compressed are input to the arithmetic encoder for entropy encoding to obtain the point cloud code stream, which is used as the point cloud compression result.

The embodiment of the present disclosure also provides a point cloud decompression device corresponding to the point cloud decompression method provided in the above embodiments, because the point cloud decompression device in the embodiment of the present disclosure has the same problem-solving principle as the present disclosure The point cloud decompression method in the above embodiments is similar, so the implementation of the point cloud decompression device can refer to the implementation of the point cloud decompression method, and the repetition will not be repeated.

Fig. 12 shows a schematic diagram of an apparatus for decompressing a point cloud provided by an embodiment of the present disclosure. See Figure 12. The point cloud decompression device 200 may include a second determination module 201 , a second prediction module 202 , a first decompression module 203 and a second quantization module 204 . Wherein, the second determination module 201 can be configured to move the preset window for the tree structure, and determine the uncompressed node in the preset window for each movement as the current node to be decompressed; the second prediction module 202 can be configured by It is configured to input the context information of the current node to be decompressed into the pre-trained attention neural network model, and predict the probability distribution of the information to be decompressed corresponding to the current node to be decompressed; the first decompression module 203 can be It is configured to input the probability distribution and point cloud code stream of the information to be decompressed corresponding to the current node to be decompressed to the arithmetic decoder for entropy decoding, and obtain the information to be decompressed corresponding to the current node to be decompressed; the second quantization module 204 , can be configured to construct a tree structure for the information to be decompressed, obtain the point cloud from the tree structure, and then obtain the decompressed point cloud through dequantization.

Please refer to FIG. 13. FIG. 13 shows a schematic structural diagram of an electronic device 300 provided by an embodiment of the present disclosure, which may include: a processor 310, a memory 320, and a bus 330. The memory 320 may be configured to store information that the processor 310 may The executed machine-readable instructions, when the electronic device 300 is running, the processor 310 and the memory 320 can communicate through the bus 330, and when the machine-readable instructions are executed by the processor 310, the point cloud in the above-mentioned embodiments can be executed. The steps of the compression method, and/or, the steps of the point cloud decompression method.

Based on the same application idea, embodiments of the present disclosure also provide a computer-readable storage medium, which can be configured to store a computer program, and when the computer program is run by a processor, it can execute the points provided by the above-mentioned embodiments. The steps of the cloud compression method, and/or, the steps of the point cloud decompression method. Optionally, the storage medium can be a general storage medium, such as a removable disk, a hard disk, etc., and when the computer program on the storage medium is run, it can be configured to perform the steps of the above-mentioned point cloud compression method, and/or, the point cloud The steps of the decompression method, by using the node occupancy code and location information of the same-level node and the parent node corresponding to the node to be compressed, as the context information of the node to be compressed, considering the same-level node and the parent node of the node to be compressed, solves the problem of The technical problems of large amount of calculation and low compression efficiency in the related art achieve the technical effect of improving the compression effect and reducing the amount of calculation.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the optional working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separated, and a component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the essence of the technical solution of the present disclosure or the part that contributes to the related technology or the part of the technical solution can be embodied in the form of software products, and the computer software products can be stored in a storage medium, including several The instructions are configured to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods of the various embodiments of the present disclosure. The aforementioned storage medium can include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc, etc., which can store program codes. medium.

The above are only optional implementations of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope of the present disclosure, and should cover all within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be determined by the protection scope of the claims.

Industrial Applicability

The present disclosure provides a point cloud compression and decompression method, which can perform compression and decompression processing through the node occupancy code and attribute information of the node to be compressed, and when performing compression and decompression, the same layer node corresponding to the node to be compressed and The node occupancy code and location information of the parent node are used as the context information of the node to be compressed, which solves the technical problems of large amount of calculation and low compression efficiency in related technologies, and achieves the technical effect of improving the compression effect and reducing the amount of calculation.

In addition, it can be understood that the point cloud compression and decompression method of the present disclosure is reproducible and can be used in various industrial applications, for example, the point cloud compression and decompression method of the present disclosure can be used in 3D laser mapping and other scenarios.

Claims (18)

A method for compressing point clouds, characterized in that the method for compressing comprises: Carrying out binary encoding to the point coordinates of the point cloud through the tree structure, converting the binary encoding into decimal encoding, as the node occupancy code corresponding to each node in the tree structure; moving the preset window to the nodes in the tree structure, and determining the uncompressed node in the preset window as the current node to be compressed; The context information of the current node to be compressed is input into the pre-trained attention neural network model, and the probability distribution of the information to be compressed corresponding to the current node to be compressed is predicted; According to the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed, input to an arithmetic encoder for entropy encoding to obtain a point cloud code stream, and use the point cloud code stream as a point cloud compression result. The compression method according to claim 1, wherein the tree structure is an octree structure. The compression method according to claim 1 or 2, wherein the preset window is configured to move according to the breadth-first principle; When moving the preset window, the nodes of the tree structure are arranged into a breadth-first sequence according to the breadth-first principle, and the previous nodes in the breadth-first sequence are used as the same layer nodes corresponding to the nodes to be compressed until the The number of peer nodes in the preset window meets the preset number; When the number of nodes at the same level in the preset window and/or the number of multi-layer parent nodes of nodes at the same level cannot meet the requirements, default nodes are used to supplement the nodes in the preset window. The compression method according to any one of the preceding claims, wherein the training step of the attention neural network model comprises: Construct a tree structure from the sample point cloud, and determine the context information of the sample node and the information to be compressed of the node, wherein the sample point cloud is different but similar to the test point cloud; The context information of the sample node is input into the first layer of attention operation network of the attention neural network model to obtain the first weighted context matrix of the sample node; adding the first weighted context matrix to the context information, and inputting the result of the addition into the first layer of multi-layer perceptron network to obtain a second weighted context matrix; Inputting the second weighted context matrix into the second layer of attention operation network to obtain a third weighted context matrix; adding the third weighted context matrix to the second weighted context matrix, and inputting the result of the addition into the second-layer multi-layer perceptron network to obtain the information to be compressed corresponding to the predicted node to be compressed Probability distributions; Bring the probability distribution of the information to be compressed corresponding to multiple sample nodes and the information to be compressed into the loss function, and calculate the loss value; Use an optimization algorithm based on deep learning to carry out backpropagation to optimize the loss value and update the weight of the attention neural network model; The above steps are executed multiple times, and when the rate of change of the loss value output by the loss function reaches a preset threshold, a trained attention neural network model is obtained. The compression method according to any one of claims 1 to 4, wherein the context information of the current node to be compressed includes: the multi-layer parent node of the same layer node corresponding to the current node to be compressed, the preorder same The node occupancy code and location information of the layer node, as well as the location information of the current node to be compressed; Use the multi-layer parent node of the same layer node corresponding to the current node to be compressed, the node occupancy code and location information of the previous node at the same layer, and the location information of the current node to be compressed as the node corresponding to the node to be compressed Contextual information, including: Using the node occupancy code and location information of the preceding peer node of the current node to be compressed and the multi-layer parent node corresponding to the preceding peer node as the first context information corresponding to the current node to be compressed; The location information of the current node to be compressed, the node occupancy code of the previous compressed node at the same layer of the current node to be compressed, and the node occupancy code and position of the multi-layer parent node corresponding to the current node to be compressed information, as the second context information corresponding to the current node to be compressed, Using the first context information and the second context information as context information corresponding to the current node to be compressed; The to-be-compressed information corresponding to the current to-be-compressed node includes: a node occupation code of the current to-be-compressed node. The compression method according to any one of claims 1 to 4, wherein the context information of the current node to be compressed further includes: the node occupancy of the multi-layer parent node of the same layer node corresponding to the current node to be compressed Code and location information, node occupancy code and location information of nodes at the same layer, node attributes and/or node attribute residuals corresponding to nodes at the same layer in the preceding sequence; The to-be-compressed information corresponding to the current to-be-compressed node further includes: node attributes and/or node attribute residuals of the current to-be-compressed node. The compression method according to claim 5 or 6, wherein the location information includes: node index, and/or node depth, and/or node bounding box coordinates. The compression method according to any one of claims 1 to 4, wherein the context information of the current node to be compressed is input into a pre-trained attention neural network model to predict the current node to be compressed The probability distribution of the corresponding information to be compressed includes: The context information of the current node to be compressed is input into the first layer of attention operation network of the attention neural network model to obtain the first weighted context matrix of the current node to be compressed; adding the first weighted context matrix and the context information, and inputting the result of the addition into the first layer of multi-layer perceptron network to obtain a second weighted context matrix; Inputting the second weighted context matrix into the second layer of attention operation network to obtain a third weighted context matrix; adding the third weighted context matrix to the second weighted context matrix, and inputting the result of the addition into the second-layer multilayer perceptron network to obtain a fourth weighted context matrix; The fourth weighted context matrix is input into the third layer multi-layer perceptron network to obtain the fifth weighted context matrix; Pass the fifth weighted context matrix through a softmax function to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed. The compression method according to claim 8, wherein the context information of the current node to be compressed is input into the first layer of attention operation network of the attention neural network model to obtain the first layer of the current node to be compressed A weighted context matrix comprising: Inputting the context information of the current node to be compressed into the first multi-layer perceptron to obtain a first output matrix; Inputting the context information of the current node to be compressed into the second multi-layer perceptron to obtain a second output matrix; Inputting the context information of the current node to be compressed into the third multi-layer perceptron to obtain a third output matrix; multiplying the transpose matrix of the second output matrix by the first output matrix to obtain a matrix inner product; Add the inner product of the matrix to the mask matrix, and input the added result to the softmax function to obtain the attention matrix; multiplying the attention matrix by the third output matrix to obtain the first weighted context matrix of the current node to be compressed. The compression method according to claim 9, wherein each element in the attention matrix is an attention value; Calculate the attention value of the attention matrix by the following formula:

In formula (1), the

Refers to the attention value between the jth node and the kth node context, the jth node is the jth node in the nodes of the same layer and is the current node to be compressed, f _j is the context of the jth node information, f _k is the context information of the kth node, and the kth node is the preorder node of the jth node; molecular

Represents the similarity value between the jth node and the kth node context, denominator

refers to the sum of the similarity values between the jth node and the context of the first node to the jth node; MLP ₂ (f _j ) refers to inputting the context information of the jth node into the second most A layer perceptron to obtain a second output matrix corresponding to the jth node;

It refers to inputting the context information of the kth node into the first multi-layer perceptron to obtain the transpose matrix of the first output matrix corresponding to the kth node. The compression method according to any one of claims 1 to 4, wherein the context information of the current node to be compressed is input into a pre-trained attention neural network model to predict the current node to be compressed The probability distribution of the corresponding information to be compressed includes: Dimensional expansion of the context information of the current node to be compressed to generate dimension expansion context information; The dimension expansion context information is input into the pre-trained attention neural network model, and the probability distribution of the information to be compressed corresponding to the current node to be compressed is predicted. The compression method according to claim 11, wherein the context information of the current node to be compressed is expanded to generate the expanded context information, including: Firstly, the context information of the current node to be compressed is expanded through a one-hot code operation to generate a one-hot code of the context information, and the one-hot code of the context information is generated through an embedding operation to generate the dimension-expanded context information. The compression method according to any one of the preceding claims, wherein the compression method includes providing a point cloud compression device, the compression device includes a first quantization module, a first determination module, a first prediction module and the first compression module, where: The first quantization module is configured to perform binary encoding on the point coordinates of the point cloud through the tree structure, and convert the binary encoding into decimal encoding as the node occupancy corresponding to each node in the tree structure. code; The first determining module is configured to move a preset window to the nodes in the tree structure, and determine an uncompressed node in the preset window as the current node to be compressed; The first prediction module is configured to input the context information of the current node to be compressed into the pre-trained attention neural network model, and predict the probability distribution of the information to be compressed corresponding to the current node to be compressed; and The first compression module is configured to input the probability distribution of the information to be compressed and the information to be compressed corresponding to the node to be compressed to an arithmetic encoder for entropy encoding to obtain a point cloud code stream, and convert the point cloud code stream As a result of the compression of the point cloud. A method for decompressing point clouds, characterized in that the method for decompressing comprises: Move the preset window to the tree structure, and determine the uncompressed node in the preset window as the current node to be decompressed each time; The context information of the current node to be decompressed is input into the pre-trained attention neural network model, and the probability distribution of the information to be decompressed corresponding to the current node to be decompressed is predicted; Inputting the probability distribution of the information to be decompressed corresponding to the current node to be decompressed and the point cloud code stream to an arithmetic decoder for entropy decoding to obtain the information to be decompressed corresponding to the current node to be decompressed; Constructing a tree structure for the information to be decompressed, obtaining a point cloud from the tree structure, and obtaining the decompressed point cloud through dequantization. The decompression method according to claim 14, characterized in that, the decompression method includes providing a point cloud decompression device, and the decompression device includes: a second determination module, a second prediction module, a first Decompression module and second quantization module, wherein: The second determination module is configured to move the preset window to the tree structure, and determine the uncompressed node within the preset window each time as the current node to be decompressed; The second prediction module is configured to input the context information of the current node to be decompressed into the pre-trained attention neural network model, and predict the probability distribution of the information to be decompressed corresponding to the current node to be decompressed; The first decompression module is configured to input the probability distribution of the information to be decompressed and the point cloud code stream corresponding to the current node to be decompressed to the arithmetic decoder for entropy decoding, and obtain the information corresponding to the current node to be decompressed. information to be decompressed; The second quantization module is configured to construct a tree structure for the information to be decompressed, obtain a point cloud from the tree structure, and obtain a decompressed point cloud through dequantization. An electronic device, characterized in that it includes: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the connection between the processor and the memory communicate with each other through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the point cloud compression method according to any one of claims 1 to 13, and/or, according to claim 14 Or the step of the decompression method of the point cloud described in 15. A computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the point cloud according to any one of claims 1 to 13 is executed. The steps of the compression method, and/or, the steps of the point cloud decompression method as claimed in claim 14 or 15. The computer-readable storage medium according to claim 17, wherein the computer-readable storage medium is a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., which can store program codes. Universal storage medium.

Images