WO2025000342A1

WO2025000342A1 - Encoding and decoding method, encoder, decoder, and storage medium

Info

Publication number: WO2025000342A1
Application number: PCT/CN2023/103834
Authority: WO
Inventors: 孙泽星
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2023-06-29
Filing date: 2023-06-29
Publication date: 2025-01-02
Anticipated expiration: 2025-12-29

Abstract

The present application discloses an encoding and decoding method, an encoder, a decoder, and a storage medium. At an encoding and decoding end, a basic mesh is subdivided, and initial geometric position information of vertexes comprised in a first vertex set in an initial mesh is determined; and on the basis of the initial geometric position information of the vertexes in the first vertex set and a mapping relationship corresponding to the first vertex set, first predicted geometric position information of the vertexes in the first vertex set is determined. In this way, when geometric position information of a first vertex set of a unit to be encoded and decoded is predicted, encoding and decoding of some vertex shift coefficients are adaptively skipped, and fitting is carried out by utilizing a mapping relationship representing the correlation between initial geometric position information and original geometric position information, to determine first predicted geometric position information corresponding to the initial geometric position information, thereby reducing the code rate of shift coefficients, ensuring the reconstruction quality of geometric position information of mesh vertexes, and improving the encoding and decoding efficiency of the geometric position information.

Description

Coding and decoding method, encoder, decoder and storage medium

Technical Field

本申请实施例涉及网格压缩编码技术领域，尤其涉及一种编解码方法、编码器、解码器以及存储介质。The embodiments of the present application relate to the technical field of grid compression coding, and in particular to a coding and decoding method, an encoder, a decoder, and a storage medium.

Background Art

在运动图像专家组(Moving Picture Experts Group，MPEG)提供的动态网格编码(Dynamic Mesh Coding)的标准参考软件中，在对网格的几何信息进行编解码时，主要包括对原始网格对应的移位系数的组织和压缩。In the standard reference software of Dynamic Mesh Coding provided by the Moving Picture Experts Group (MPEG), the encoding and decoding of the geometric information of the mesh mainly includes the organization and compression of the shift coefficients corresponding to the original mesh.

然而，目前常见的移位系数的组织方法并不是最佳的，进而会增大后续的无损编码移位系数的码率，降低了网格压缩性能。However, the currently common organization method of the shift coefficients is not optimal, which will increase the bit rate of the subsequent lossless coding of the shift coefficients and reduce the grid compression performance.

发明内容Summary of the invention

本申请实施例提供一种编解码方法、编码器、解码器以及存储介质，编解码移位系数时，能够提高移位系数的编码效率。The embodiments of the present application provide a coding and decoding method, an encoder, a decoder and a storage medium, which can improve the coding efficiency of shift coefficients when encoding and decoding shift coefficients.

本申请实施例的技术方案可以如下实现：The technical solution of the embodiment of the present application can be implemented as follows:

第一方面，本申请实施例提供了一种解码方法，应用于解码器，所述方法包括：In a first aspect, an embodiment of the present application provides a decoding method, which is applied to a decoder, and the method includes:

确定基础网格；Determine the base grid;

对所述基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；Subdividing the basic mesh to determine initial geometric position information of vertices included in a first vertex set in the initial mesh;

解码码流，确定所述第一顶点集合对应的映射关系；其中，所述映射关系用于指示所述第一顶点集合中顶点的初始几何位置信息和原始几何位置信息的相关性；Decoding the code stream to determine a mapping relationship corresponding to the first vertex set; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

根据所述第一顶点集合中顶点的所述初始几何位置信息和所述映射关系，确定所述第一顶点集合中所述顶点的第一预测几何位置信息。First predicted geometric position information of the vertices in the first vertex set is determined according to the initial geometric position information of the vertices in the first vertex set and the mapping relationship.

第二方面，本申请实施例提供了一种编码方法，应用于编码器，所述方法包括：In a second aspect, an embodiment of the present application provides an encoding method, which is applied to an encoder, and the method includes:

根据原始网格，确定基础网格；According to the original grid, determine the basic grid;

根据所述原始网格中所述第一顶点集合所包含顶点的原始几何位置信息和所述初始几何位置信息，确定所述第一顶点集合对应的映射关系；其中，所述映射关系用于指示所述第一顶点集合中顶点的初始几何位置信息和原始几何位置信息的相关性；Determine a mapping relationship corresponding to the first vertex set according to the original geometric position information of the vertices included in the first vertex set in the original mesh and the initial geometric position information; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

对所述映射关系进行代价计算，确定第一代价参考值；Performing cost calculation on the mapping relationship to determine a first cost reference value;

在根据所述第一代价参考值确定编码所述映射关系的情况下，对第一顶点集合对应的所述映射关系进行编码，将所得到的编码比特写入码流。In case of determining to encode the mapping relationship according to the first cost reference value, the mapping relationship corresponding to the first vertex set is encoded, and the obtained encoding bits are written into a bitstream.

第三方面，本申请实施例提供了一种码流，其中，所述码流是根据待编码信息进行比特编码生成的；其中，所述待编码信息包括下述至少一项：用于指示第一顶点集合对应的映射关系的映射关系指示信息、用于指示解码所述映射关系的第三指示信息、用于指示解码所述第一顶点集合中顶点的移位系数的第四指示信息。In a third aspect, an embodiment of the present application provides a code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: mapping relationship indication information for indicating a mapping relationship corresponding to a first vertex set, third indication information for indicating decoding of the mapping relationship, and fourth indication information for indicating decoding of shift coefficients of vertices in the first vertex set.

第四方面，本申请实施例提供了一种编码器，所述编码器包括：In a fourth aspect, an embodiment of the present application provides an encoder, the encoder comprising:

第一确定单元，配置为根据原始网格，确定基础网格；对所述基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；A first determining unit is configured to determine a basic mesh according to the original mesh; subdivide the basic mesh to determine initial geometric position information of vertices included in a first vertex set in the initial mesh;

拟合单元，配置为根据所述原始网格中所述第一顶点集合所包含顶点的原始几何位置信息和所述初始几何位置信息，确定所述第一顶点集合对应的映射关系；其中，所述映射关系用于指示所述第一顶点集合中顶点的初始几何位置信息和原始几何位置信息的相关性；A fitting unit, configured to determine a mapping relationship corresponding to the first vertex set according to the original geometric position information of the vertices included in the first vertex set in the original mesh and the initial geometric position information; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

第一预测单元，配置为对所述映射关系进行代价计算，确定第一代价参考值；A first prediction unit is configured to perform cost calculation on the mapping relationship to determine a first cost reference value;

编码单元，配置为在根据所述第一代价参考值确定编码所述映射关系的情况下，对第一顶点集合对应的所述映射关系进行编码，将所得到的编码比特写入码流。The encoding unit is configured to, when determining to encode the mapping relationship according to the first cost reference value, encode the mapping relationship corresponding to the first vertex set, and write the obtained encoding bits into a bitstream.

第五方面，本申请实施例提供了一种编码器，所述编码器包括：第一存储器和第一处理器；其中， In a fifth aspect, an embodiment of the present application provides an encoder, the encoder comprising: a first memory and a first processor; wherein,

所述第一存储器，用于存储能够在所述第一处理器上运行的计算机程序；The first memory is used to store a computer program that can be run on the first processor;

所述第一处理器，用于在运行所述计算机程序时，执行如第二方面所述的方法。The first processor is used to execute the method as described in the second aspect when running the computer program.

第六方面，本申请实施例提供了一种解码器，所述解码器包括：解码单元，第二确定单元；其中，In a sixth aspect, an embodiment of the present application provides a decoder, the decoder comprising: a decoding unit, a second determining unit; wherein,

第二确定单元，配置为确定基础网格；对所述基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；A second determining unit is configured to determine a basic mesh; subdivide the basic mesh to determine initial geometric position information of vertices included in a first vertex set in an initial mesh;

解码单元，配置为解码码流，确定所述第一顶点集合对应的映射关系；其中，所述映射关系用于指示所述第一顶点集合中顶点的初始几何位置信息和原始几何位置信息的相关性；A decoding unit configured to decode the code stream to determine a mapping relationship corresponding to the first vertex set; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

第二预测单元，配置为根据所述第一顶点集合中顶点的所述初始几何位置信息和所述映射关系，确定所述第一顶点集合中所述顶点的第一预测几何位置信息。The second prediction unit is configured to determine first predicted geometric position information of the vertices in the first vertex set according to the initial geometric position information of the vertices in the first vertex set and the mapping relationship.

第七方面，本申请实施例提供了一种解码器，第二存储器和第二处理器；其中，In a seventh aspect, an embodiment of the present application provides a decoder, a second memory, and a second processor; wherein:

所述第二存储器，用于存储能够在所述第二处理器上运行的计算机程序；The second memory is used to store a computer program that can be run on the second processor;

所述第二处理器，用于在运行所述计算机程序时，执行如第一方面所述的方法。The second processor is used to execute the method as described in the first aspect when running the computer program.

第九方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被执行时实现如第一方面所述的方法、或者实现如第二方面所述的方法。In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the method as described in the first aspect is implemented, or the method as described in the second aspect is implemented.

本申请实施例提供了一种编解码方法、编码器、解码器以及存储介质，在编解码端，对基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；根据所述第一顶点集合中顶点的所述初始几何位置信息和第一顶点集合对应的映射关系，确定所述第一顶点集合中所述顶点的第一预测几何位置信息。这样，在预测待编解码单元的第一顶点集合的几何位置信息时，自适应的跳过一些顶点移位系数的编解码，而是利用表征初始几何位置信息和原始几何位置信息相关性的映射关系进行拟合，确定初始几何位置信息对应的第一预测几何位置信息，降低移位系数的码率，在保证网格顶点的几何位置信息的重建质量的基础上，提高几何位置信息的编解码效率。The embodiment of the present application provides a coding and decoding method, an encoder, a decoder and a storage medium. At the coding and decoding end, the basic mesh is subdivided to determine the initial geometric position information of the vertices contained in the first vertex set in the initial mesh; according to the mapping relationship between the initial geometric position information of the vertices in the first vertex set and the first vertex set, the first predicted geometric position information of the vertices in the first vertex set is determined. In this way, when predicting the geometric position information of the first vertex set of the unit to be coded and decoded, the coding and decoding of some vertex shift coefficients are adaptively skipped, and the mapping relationship that characterizes the correlation between the initial geometric position information and the original geometric position information is used for fitting to determine the first predicted geometric position information corresponding to the initial geometric position information, reduce the code rate of the shift coefficient, and improve the coding and decoding efficiency of the geometric position information on the basis of ensuring the reconstruction quality of the geometric position information of the mesh vertices.

BRIEF DESCRIPTION OF THE DRAWINGS

图1A为三维网格图像的示意图一；FIG1A is a schematic diagram 1 of a three-dimensional grid image;

图1B为三维网格图像的局部放大图；FIG1B is a partial enlarged view of a three-dimensional grid image;

图2为三维网格的连接方式示意图；FIG2 is a schematic diagram of a connection method of a three-dimensional grid;

图3A为一幅三维网格图像的示意图二；FIG3A is a second schematic diagram of a three-dimensional grid image;

图3B为网格数据存储格式的示意图；FIG3B is a schematic diagram of a grid data storage format;

图3C为三维网格图像的属性图；FIG3C is a property diagram of a three-dimensional grid image;

图4为网格编码整体框架图；Figure 4 is a diagram of the overall framework of grid coding;

图5A为网格预处理流程的示意图；FIG5A is a schematic diagram of a grid preprocessing process;

图5B为移位系数的生成示意图；FIG5B is a schematic diagram of generating a shift coefficient;

图6A为网格几何位置信息的量化处理示意图一；FIG6A is a first schematic diagram of quantization processing of grid geometric position information;

图6B为网格几何位置信息的量化处理示意图二；FIG6B is a second schematic diagram of quantization processing of grid geometric position information;

图7A为三角面片的连接关系的编码示意图；FIG. 7A is a schematic diagram of coding of the connection relationship of triangular facets;

图7B为几何位置信息的编码示意图；FIG7B is a schematic diagram of encoding geometric position information;

图7C为纹理坐标的编码示意图；FIG7C is a schematic diagram of encoding of texture coordinates;

图8A为顶点移位系数的基本原理示意图；FIG8A is a schematic diagram of the basic principle of vertex shift coefficients;

图8B为移位系数映射到二维图像的编码示意图；FIG8B is a schematic diagram of encoding of mapping shift coefficients to a two-dimensional image;

图9为帧间几何位置信息的编码示意图；FIG9 is a schematic diagram of encoding geometric position information between frames;

图10A为帧内编码的示意图；FIG10A is a schematic diagram of intra-frame coding;

图10B为帧间编码的示意图；FIG10B is a schematic diagram of inter-frame coding;

图11A为帧内解码的示意图；FIG11A is a schematic diagram of intra-frame decoding;

图11B为帧间解码的示意图；FIG11B is a schematic diagram of inter-frame decoding;

图12为本申请实施例提供的一种编解码的网络架构示意图；FIG12 is a schematic diagram of a network architecture of a codec provided in an embodiment of the present application;

图13为本申请实施例提供的一种解码方法的流程示意图；FIG13 is a schematic diagram of a flow chart of a decoding method provided in an embodiment of the present application;

图14A为一种基础网格迭代细分示意图；FIG14A is a schematic diagram of iterative subdivision of a basic grid;

图14B为一种LOD空间结构示意图；FIG14B is a schematic diagram of a LOD space structure;

图15A为一种映射关系的示意图一；FIG15A is a schematic diagram of a mapping relationship;

图15B为一种映射关系的示意图二；FIG15B is a second schematic diagram of a mapping relationship;

图16为对量化之后的系数进行系数重置示意图； FIG16 is a schematic diagram of resetting coefficients after quantization;

图17为根据移位系数进行几何位置信息重建原理示意图；FIG17 is a schematic diagram showing the principle of geometric position information reconstruction based on the shift coefficient;

图18为本申请实施例提供的一种编码方法的流程示意图；FIG18 is a schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application;

图19为编码器的组成结构示意图一；FIG19 is a schematic diagram of the structure of the encoder;

图20为编码器的组成结构示意图二；FIG20 is a second schematic diagram of the structure of the encoder;

图21为解码器的组成结构示意图一；FIG21 is a schematic diagram of the structure of a decoder;

图22为解码器的组成结构示意图二。FIG. 22 is a second schematic diagram of the structure of the decoder.

DETAILED DESCRIPTION

为了能够更加详尽地了解本申请实施例的特点与技术内容，下面结合附图对本申请实施例的实现进行详细阐述，所附附图仅供参考说明之用，并非用来限定本申请实施例。In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述本申请实施例的目的，不是旨在限制本申请。Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

还需要指出，本申请实施例所涉及的术语“第一\第二\第三”仅是用于区别类似的对象，不代表针对对象的特定排序，可以理解地，“第一\第二\第三”在允许的情况下可以互换特定的顺序或先后次序，以使这里描述的本申请实施例能够以除了在这里图示或描述的以外的顺序实施。It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

需要说明的是，可以允许在同一视频场景中解码和合成不同的数据格式比特流。其中，至少可以包括图像格式、点云(Point Cloud)格式、网格(Mesh)格式。通过这种方式，可以为具有不同来源的多个数据格式(例如，网格、点云、图像等等)提供实时沉浸式视频交互服务。It should be noted that different data format bitstreams can be decoded and synthesized in the same video scene. Among them, at least image format, point cloud format, and mesh format can be included. In this way, real-time immersive video interaction services can be provided for multiple data formats (for example, mesh, point cloud, image, etc.) with different sources.

在本申请实施例中，基于数据格式的方法可以允许以数据格式的比特流级进行独立性处理。即与视频编码中的瓦片(tiles)或切片(slices)一样，该场景中的不同数据格式可以以独立的方式编码，从而可以基于数据格式进行独立的编码和解码。In the embodiment of the present application, the data format-based method can allow independent processing at the bitstream level of the data format. That is, like tiles or slices in video encoding, different data formats in the scene can be encoded in an independent manner, so that independent encoding and decoding can be performed based on the data format.

一般而言，三维动画内容采用基于关键帧的表示方法，即每帧是一个静态网格。不同时刻的静态网格具有相同的拓扑结构和不同的几何结构。但是，基于关键帧表示的三维动态网格的数据量特别大，因此如何能够有效的存储、传输和绘制成为三维动态网格发展所面临的问题。另外针对不同的用户终端(电脑、笔记本、便携式设备、手机)需要支持网格的空间可伸缩性；不同的网络带宽(宽带、窄带、无线)需要支持网格的质量可伸缩性。因此，三维动态网格压缩是一个非常关键的问题。Generally speaking, 3D animation content is represented based on keyframes, that is, each frame is a static mesh. Static meshes at different times have the same topological structure and different geometric structures. However, the amount of data of 3D dynamic meshes represented based on keyframes is extremely large, so how to effectively store, transmit and draw them has become a problem faced by the development of 3D dynamic meshes. In addition, the spatial scalability of the mesh needs to be supported for different user terminals (computers, notebooks, portable devices, mobile phones); different network bandwidths (broadband, narrowband, wireless) need to support the quality scalability of the mesh. Therefore, 3D dynamic mesh compression is a very critical issue.

三维网格是通过空间中的无数个多边形组成的三维物体表面，多边形由顶点和边组成，图1A展示了三维网格图像，图1B展示了三维网格图像的局部放大图，可以看到网格表面是由闭合多边形所组成的。A three-dimensional grid is a three-dimensional object surface composed of countless polygons in space. Polygons are composed of vertices and edges. FIG1A shows a three-dimensional grid image, and FIG1B shows a partial enlarged view of the three-dimensional grid image. It can be seen that the grid surface is composed of closed polygons.

二维图像在每一个像素点均有信息表达，分布规则，因此不需要额外记录其位置信息；然而网格中的顶点在三维空间中的分布具有随机性和不规则性，以及多边形的构成方式需要进行额外的规定，因此需要记录每一个顶点在空间中的位置，以及每个多边形的连接信息，才能完整地表达一幅网格图像，如图2所示，同样的顶点数目和顶点位置，由于连接方式的不同，所形成的表面也完全不同。A two-dimensional image has information expressed at each pixel point and is distributed regularly, so there is no need to record its position information additionally; however, the distribution of vertices in the mesh in three-dimensional space is random and irregular, and the way polygons are formed requires additional regulations. Therefore, it is necessary to record the position of each vertex in space and the connection information of each polygon in order to fully express a mesh image. As shown in Figure 2, the same number of vertices and vertex positions, due to different connection methods, form completely different surfaces.

除了以上信息之外，由于三维网格图像通常采用已有的二维图像/视频编码方式进行编码，因此需要将三维网格图像进行从三维空间到二维空间的转化，UV坐标就定义了这一转化过程。In addition to the above information, since the 3D grid image is usually encoded using an existing 2D image/video encoding method, the 3D grid image needs to be converted from 3D space to 2D space, and the UV coordinates define this conversion process.

与二维图像类似，采集过程中每一个位置可能会有对应的属性信息，通常为RGB颜色值，颜色值反映物体的色彩；对于三维网格图像来说，每一个顶点所对应的属性信息除了颜色以外，还有比较常见的是反射率(reflectance)值，反射率值反映物体的表面材质。三维网格图像的属性信息通过二维图像进行存储，其从二维到三维的映射由UV坐标规定。Similar to 2D images, each position in the acquisition process may have corresponding attribute information, usually RGB color values, which reflect the color of the object; for 3D mesh images, in addition to color, the attribute information corresponding to each vertex is also commonly reflectance values, which reflect the surface material of the object. The attribute information of 3D mesh images is stored in 2D images, and the mapping from 2D to 3D is specified by UV coordinates.

因此，三维网格数据通常包括三维几何位置信息(x，y，z)、几何位置信息三角面片的连接关系、纹理坐标(u，v)以及纹理坐标的连接关系，以及属性图。图3A为一幅三维网格图像，图3B网格数据存储格式包括三维几何位置信息、纹理坐标和连接信息，图3C为对应的属性图。Therefore, the 3D mesh data usually includes 3D geometric position information (x, y, z), the connection relationship of the geometric position information triangles, texture coordinates (u, v) and the connection relationship of the texture coordinates, and an attribute map. FIG3A is a 3D mesh image, FIG3B is a mesh data storage format including 3D geometric position information, texture coordinates and connection information, and FIG3C is a corresponding attribute map.

目前的三维动态网格压缩方法有基于空间-时间的预测方法，通过消除空间和时间相关性来提高压缩效率；基于主成分分析(Principal Components Analysis，PCA)的技术，在特征向量空间进行投影，使能量集中；基于小波的方法，支持空间可伸缩和质量可伸缩。Current 3D dynamic mesh compression methods include space-time prediction methods, which improve compression efficiency by eliminating spatial and temporal correlations; principal component analysis (PCA)-based technology, which projects in the eigenvector space to concentrate energy; and wavelet-based methods, which support spatial scalability and quality scalability.

需要说明的是，图4为网格编码整体框架图，图5A为二维曲线预处理流程的示意图，图5B为移位系数的生成示意图，三维网格的预处理过程可同理类比，在编码端主要分为预处理(Pre-processing)和编码器(Encoder)两部分。其中，首先通过预处理生成基础网格和移位系数。预处理流程包括：首先对原始网格(Original Mesh)进行下采样，生成顶点数量大幅减少的简化网格(Decimated Mesh)，或称基础网格/基本网格(Base Mesh)。然后对简化网格进行细分，在简化网格的边上插入新生成的顶点，得到细分网格(Subdivided Mesh)或称初始网格。最后对于细分网格中的每个顶点，寻找原始网格中与其距离最近的点，并计算这两个点的移位系数(Displacement)。在经过预处理后，将简化网格和移位系数输入编码器，生成码流。It should be noted that Figure 4 is a diagram of the overall framework of mesh coding, Figure 5A is a schematic diagram of the two-dimensional curve preprocessing process, and Figure 5B is a schematic diagram of the generation of the shift coefficient. The preprocessing process of the three-dimensional mesh can be analogous, and is mainly divided into two parts at the encoding end: preprocessing (Pre-processing) and encoder (Encoder). Among them, the basic mesh and shift coefficients are first generated through preprocessing. The preprocessing process includes: first, downsampling the original mesh (Original Mesh) to generate a simplified mesh (Decimated Mesh) with a greatly reduced number of vertices, Or called the base mesh. Then subdivide the simplified mesh, insert the newly generated vertices on the edges of the simplified mesh, and get the subdivided mesh or the initial mesh. Finally, for each vertex in the subdivided mesh, find the point in the original mesh that is closest to it, and calculate the displacement coefficient of the two points. After preprocessing, the simplified mesh and the displacement coefficient are input into the encoder to generate the bitstream.

V-DMC编码主要可以分为两大类：几何位置信息编码和属性信息编码。如图3B和图3C所示，序列basketball_player的每一帧文件中包括两个文件：basketball_player_fr0001_qp12_qt12.obj和basketball_player_fr0002.png。其中，basketball_player_fr0001_qp12_qt12.obj中包含四种信息分别为：几何位置信息(x,y,z)、几何位置信息三角面片的连接关系、纹理坐标(u,v)以及纹理坐标的连接关系。basketball_player_fr0002.png代表的是当前帧的纹理属性信息。在目前的V-DMC编码器中，几何位置信息采用DRACO以及Video Codec(AVC、HEVC或者VVC)联合编码,纹理信息编码直接利用Video Codec进行编码。因此，接下来对mesh的几何信息编码进行详细的介绍。V-DMC encoding can be mainly divided into two categories: geometric position information encoding and attribute information encoding. As shown in Figure 3B and Figure 3C, each frame file of the sequence basketball_player includes two files: basketball_player_fr0001_qp12_qt12.obj and basketball_player_fr0002.png. Among them, basketball_player_fr0001_qp12_qt12.obj contains four types of information: geometric position information (x, y, z), the connection relationship of geometric position information triangles, texture coordinates (u, v) and the connection relationship of texture coordinates. basketball_player_fr0002.png represents the texture attribute information of the current frame. In the current V-DMC encoder, the geometric position information is jointly encoded by DRACO and Video Codec (AVC, HEVC or VVC), and the texture information encoding is directly encoded by Video Codec. Therefore, the geometric information encoding of mesh is introduced in detail below.

几何信息可以分为：位置信息的编码(几何位置信息和纹理位置信息)与连接关系的编码(几何位置信息的三角面片连接关系、纹理位置信息的连接关系)。目前的V-DMC编码主要分为两种编码测试条件：帧内编码和帧间编码(低时延，目前没有RA的测试环境)。Geometric information can be divided into: encoding of position information (geometric position information and texture position information) and encoding of connection relationships (connection relationships of triangle patches of geometric position information and connection relationships of texture position information). The current V-DMC encoding is mainly divided into two encoding test conditions: intra-frame encoding and inter-frame encoding (low latency, currently there is no RA test environment).

帧内几何信息编码Intra-frame geometry coding

1.网格(mesh)预处理1. Mesh preprocessing

a)如图5A和图5B所示，以二维的连接关系为例子。原始网格的连接关系中含有大量的点，在对网格几何信息编码之前，首先进行网格几何信息的量化或者简单化，最终得到对应的简化网格(Decimated mesh)作为基础网格。a) As shown in Figures 5A and 5B, a two-dimensional connection relationship is used as an example. The connection relationship of the original mesh contains a large number of points. Before encoding the mesh geometry information, the mesh geometry information is first quantized or simplified, and finally the corresponding simplified mesh (Decimated mesh) is obtained as the basic mesh.

b)如图6A和图6B所示，基于三角面片坐标进行mesh的量化处理。按照量化点之间的连接关系，将量化处理可以划分为一下两种情况：b) As shown in FIG6A and FIG6B , mesh quantization is performed based on the coordinates of the triangular patch. According to the connection relationship between the quantization points, the quantization process can be divided into the following two cases:

当两个顶点量化之前属于一条边的两个顶点，则量化之后，需要将两个顶点连接的三角面片全部连接在一起，如图6A所示涉及到之前三角面片的消失化；否则，如果两个顶点不属于一条边的两个顶点，那么量化之后，仅仅需要将两个顶点的边界进行合并即可，如图6B所示对三角面片的数量不出产生影响。When two vertices belong to one edge before quantization, then after quantization, all the triangles connected to the two vertices need to be connected together, as shown in FIG6A , which involves the disappearance of the previous triangles; otherwise, if the two vertices do not belong to one edge, then after quantization, only the boundaries of the two vertices need to be merged, as shown in FIG6B , which does not affect the number of triangles.

c)基于三角面片坐标进行mesh的量化处理的整个过程中，最为核心的问题在于如何根据之前的顶点坐标得到最佳的顶点，目前的V-DMC中会在以下四种模式当中得到最佳的量化点，假设量化之前的顶点分布为V1和V2，量化之后的顶点坐标为V’，则存在以下：V1、V2、(V1+V2)/2以及Q-1(V1+V2)，其中Q为V1和V2顶点坐标对应的量化矩阵。最终基础量化前后的失真测度D来选取最佳的量化点。c) In the whole process of mesh quantization based on triangle patch coordinates, the core problem is how to get the best vertex based on the previous vertex coordinates. The current V-DMC will get the best quantization point in the following four modes. Assuming that the vertex distribution before quantization is V1 and V2, and the vertex coordinates after quantization are V', there are the following: V1, V2, (V1+V2)/2 and Q-1(V1+V2), where Q is the quantization matrix corresponding to the vertex coordinates of V1 and V2. Finally, the distortion measure D before and after basic quantization is used to select the best quantization point.

2.Base mesh编码2. Base mesh encoding

a)在得到Base mesh之后，会利用DRACO编码器来对Base mesh的几何信息编码。其中，几何信息主要包括：连接关系与几何位置信息的连接关系。整个DRACO编码的流程按照：首先完成连接关系的编码，其次基于几何位置的连接关系对点的几何位置信息进行编码，最后基于连接关系、几何位置信息来对纹理位置信息进行编码。a) After obtaining the Base mesh, the DRACO encoder is used to encode the geometric information of the Base mesh. The geometric information mainly includes: the connection relationship and the connection relationship of the geometric position information. The entire DRACO encoding process is as follows: first complete the encoding of the connection relationship, then encode the geometric position information of the point based on the connection relationship of the geometric position, and finally encode the texture position information based on the connection relationship and geometric position information.

b)连接关系的编码。DRACO对mesh的连接关系编码采用的是“EdgebreakerCoding”方案。具体的，如图7A所示：b) Coding of connection relationship. DRACO uses the "Edgebreaker Coding" scheme to encode the connection relationship of the mesh. Specifically, as shown in Figure 7A:

在对mesh的连接关系之前，将mesh的顶点划分为五种类型，CLRSE，其中每个符号表示的物理意义如下：Before connecting the mesh, the vertices of the mesh are divided into five types, CLRSE, where the physical meaning of each symbol is as follows:

i.C:与当前顶点连接的三角面片没有一个完成编码i.C: None of the triangles connected to the current vertex have completed encoding

ii.L与当前顶点连接的左边的三角面片完成编码ii.L completes the encoding of the triangle patch on the left connected to the current vertex

iii.R:与当前顶点连接的右边的三角面片完成编码iii.R: The triangle on the right connected to the current vertex completes the encoding

iv.S:与当前顶点连接的左边和右边的三角面片都没有完成编码iv.S: The left and right triangles connected to the current vertex have not been encoded.

v.E:与当前顶点连接的左边和右边的三角面片都已经完成编码v.E: The left and right triangles connected to the current vertex have been encoded.

最终按照一定的顺序，对每个顶点的类型以及顶点的处理顺序进行编码，解码端按照顶点的处理顺序以及顶点的类型来恢复mesh的几何连接关系。Finally, the type of each vertex and the processing order of the vertices are encoded in a certain order, and the decoding end restores the geometric connection relationship of the mesh according to the processing order and type of the vertices.

c)几何位置信息的编码。在完成顶点连接关系的编码之后，基于顶点的连接关系来对每个顶点的几何位置信息进行预测编码。预测编码采用的思想是“平行四边形算法(Parallelograms algorithm)”，具体的如图7B所示：c) Coding of geometric position information. After completing the coding of the vertex connection relationship, the geometric position information of each vertex is predictively coded based on the vertex connection relationship. The idea adopted in predictive coding is the "Parallelograms algorithm", as shown in Figure 7B:

利用与当前待编码点相邻的三个顶点：左边(left)顶点、右边(right)顶点以及对端(opposite)顶点简单的线性拟合：
pred_pos＝(left+right)-oppositeA simple linear fit is performed using the three vertices adjacent to the current point to be encoded: the left vertex, the right vertex, and the opposite vertex:
pred _pos =(left+right)-opposite

d)在完成点连接关系以及几何位置信息之后，基于这两者解码重建的基础，来对纹理坐标进行预测编码，具体如图7C所示。同样的，假设当前顶点为C，根据点的连接关系可以得到当前点的Left和Right顶点，其次利用left和right顶点的纹理坐标来对当前顶点C的纹理坐标进行预测编码。d) After completing the point connection relationship and geometric position information, the texture coordinates are decoded and reconstructed based on these two. Prediction coding is specifically shown in Figure 7C. Similarly, assuming that the current vertex is C, the Left and Right vertices of the current point can be obtained according to the connection relationship of the points, and then the texture coordinates of the left and right vertices are used to predict the texture coordinates of the current vertex C.

3.Displacement编码3. Displacement encoding

a)首先，在完成Base mesh的编码重建之后，会利用一定的划分算法，来对Base mesh进行划分得到初始重建mesh，具体的可以参考如图5A中的细分网格对应的曲线，简化网格对应的曲线可以利用简单的线性插值，得到细分网格对应的曲线。其中新插入的点坐标根据当前边界上的两个顶点进行线性插值得到：
a) First, after the encoding and reconstruction of the base mesh is completed, a certain partitioning algorithm will be used to partition the base mesh to obtain the initial reconstructed mesh. For details, please refer to the curve corresponding to the subdivided mesh in Figure 5A. The curve corresponding to the simplified mesh can be obtained by simple linear interpolation. The coordinates of the newly inserted point are obtained by linear interpolation based on the two vertices on the current boundary:

b)其次，计算划分之后的细分mesh与原始mesh之间点的误差Delta，误差Delta可以是一种世界坐标系下点的误差。最终，利用每个点之间的误差Delta以及每个点的法向量Norm计算得到每个点的Displacement值，具体的如图8A所示。b) Secondly, the error Delta between the points of the subdivided mesh and the original mesh is calculated. The error Delta can be a point error in the world coordinate system. Finally, the Displacement value of each point is calculated using the error Delta between each point and the normal vector Norm of each point, as shown in Figure 8A.

具体的计算方式如下：
Displacement＝Delta×NormThe specific calculation method is as follows:
Displacement = Delta × Norm

c)在计算得到每个点的Displacement之后，可以利用提升小波变换(Lifting Transform)来将空域残差系数变换到频域，得到对应的频域残差系数。c) After calculating the Displacement of each point, the lifting wavelet transform can be used to transform the spatial domain residual coefficients to the frequency domain to obtain the corresponding frequency domain residual coefficients.

d)最终，会利用Packing算法将每个点的频域残差系数按照一定顺序映射到二维的图像当中，目前的V-DMC按照莫顿码的顺序进行排列，具体的如图8B所示。d) Finally, the Packing algorithm is used to map the frequency domain residual coefficients of each point into a two-dimensional image in a certain order. The current V-DMC is arranged in the order of the Morton code, as shown in FIG8B .

e)最终可以利用传统的Video Codec来对二维图像进行编码。e) Finally, the traditional Video Codec can be used to encode the two-dimensional image.

4.重着色4. Recoloring

重着色属于编码端的算法，当完成编码端几何信息的重建之后。利用原始的几何信息、原始的纹理属性信息以及重建的mesh几何信息，来对重建之后的mesh的纹理属性信息进行重着色。Recoloring is an algorithm on the encoder side. After the reconstruction of the geometric information on the encoder side is completed, the texture attribute information of the reconstructed mesh is recolored using the original geometric information, the original texture attribute information, and the reconstructed mesh geometric information.

帧间几何信息编码Inter-frame geometry information coding

a)类似帧内几何信息编码，包括几何连接关系以及几何位置信息编码。但是这里需要注意的是，帧间的几何位置信息编码，仅仅需要编码当前Base mesh的几何位置信息(x,y,z)，不需要编码连接关系以及纹理位置信息(u,v)，具体的原因如下：如果当前帧可以采用帧间编码，那么在编码端会利用当前帧的参考帧的Base mesh来得到当前帧的mesh信息，因此当前帧和参考帧拥有相同的链接关系以及uv纹理坐标，唯有几何位置信息不相同。a) Similar to intra-frame geometric information coding, including geometric connection relationship and geometric position information coding. However, it should be noted that the inter-frame geometric position information coding only needs to encode the geometric position information (x, y, z) of the current Base mesh, and does not need to encode the connection relationship and texture position information (u, v). The specific reasons are as follows: If the current frame can be inter-frame coded, then the Base mesh of the reference frame of the current frame will be used at the encoding end to obtain the mesh information of the current frame. Therefore, the current frame and the reference frame have the same connection relationship and uv texture coordinates, but the geometric position information is different.

b)基于a)我们可以知道，当前帧和参考帧只有几何位置信息存在误差，因此当前的V-DMC对当前帧的几何位置信息进行预测编码。b) Based on a), we know that there is only an error in the geometric position information between the current frame and the reference frame, so the current V-DMC predicts and encodes the geometric position information of the current frame.

具体如图9所示，中间黑色顶点为待编码/解码的顶点，利用当前点在参考帧得到对应的预测点(类似视频编码当中的同位块)，其次利用当前点的邻域点(已经完成编码顶点的MV)来对当前点的MV进行预测编码。具体的如下所示，假设当前点的坐标为pos，对应的同位点的坐标为Pred_pos，则当前点的MV计算为：
MV＝Pos-Pred_pos As shown in Figure 9, the black vertex in the middle is the vertex to be encoded/decoded. The current point is used in the reference frame to obtain the corresponding prediction point (similar to the same-position block in video encoding). Then, the neighborhood point of the current point (the MV of the vertices that have been encoded) is used to predict the MV of the current point. As shown below, assuming that the coordinates of the current point are pos and the coordinates of the corresponding same-position point are Pred_pos, the MV of the current point is calculated as:
MV＝Pos-Pred _pos

在当前的V-DMC存在了两种预测编码模式：There are two prediction coding modes in the current V-DMC:

i.对当前点的MV直接进行编码i. Directly encode the MV of the current point

ii.对当前点的MV利用邻域进行预测编码ii. Use the neighborhood to predict the MV of the current point

在编码端利用率失真优化算法得到每个Coding Group(CG)的最佳编码模式，当前的V-DMC设定每个CG的点数最多为16个。At the encoding end, the rate-distortion optimization algorithm is used to obtain the optimal encoding mode for each Coding Group (CG). The current V-DMC sets the maximum number of points for each CG to 16.

纹理属性信息的编码：目前的V-DMC对纹理属性信息编码直接利用Video-Codec进行编码，例如：AVC、HEVC、VVC或者VV-Enc。Coding of texture attribute information: The current V-DMC encodes texture attribute information directly using Video-Codec, such as AVC, HEVC, VVC or VV-Enc.

图10A为帧内编码的示意图，如图10A所示，在帧内编码器中，可以采用常见的静态网格编码器(Static Mesh Encoder)对简化网格进行编码，生成对应的码流(Compressed base mesh bitstream)。接下来，用重构的简化网格更新移位系数(Update Displacements)。对更新后的移位系数进行小波变换(Wavelet Transform)和量化(Quantization)后得到移位系数。并进行图像打包(Image Packing)后采用高效率视频编码(High Efficiency Video Coding，H.265-HEVC)进行编码，生成移位系数的码流(Compressed displacements bitstream)。对于属性图(Attribute Map)编码，首先根据重构的几何信息与原始几何信息间的差异对特征图进行变换(Texture Transfer)，然后将其进行填补(Padding)、颜色空间转换(Color Space conversion)后用视频编码器(Video Coding)编码形成属性码流(Compressed attribute bitstream)。FIG10A is a schematic diagram of intra-frame coding. As shown in FIG10A , in the intra-frame encoder, a common static mesh encoder (Static Mesh Encoder) can be used to encode the simplified mesh to generate the corresponding bitstream (Compressed base mesh bitstream). Next, the reconstructed simplified mesh is used to update the displacement coefficient (Update Displacements). The updated displacement coefficient is subjected to wavelet transform (Wavelet Transform) and quantization (Quantization) to obtain the displacement coefficient. After image packing (Image Packing), high-efficiency video coding (High Efficiency Video Coding, H.265-HEVC) is used for encoding to generate a bitstream (Compressed displacements bitstream) of the displacement coefficient. For attribute map encoding, the feature map is first transformed (Texture Transfer) according to the difference between the reconstructed geometric information and the original geometric information, and then padded (Padding) and color space converted (Color Space conversion) are performed, and then encoded using a video encoder (Video Coding) to form a compressed attribute bitstream.

图10B为帧间编码的示意图，如图10B所示，帧间编码器与帧内编码器流程大致相同，但帧间编码器并不直接对简化网格进行编码，而是编码当前帧的简化网格与参考帧的简化网格间的运动矢量(Motion Encoder)，并生成相应的运动矢量的码流(Compressed motion bitstream)。FIG10B is a schematic diagram of inter-frame coding. As shown in FIG10B , the inter-frame encoder and the intra-frame encoder have roughly the same process, but the inter-frame encoder The encoder does not encode the simplified grid directly, but encodes the motion vector between the simplified grid of the current frame and the simplified grid of the reference frame (Motion Encoder), and generates a corresponding motion vector bitstream (Compressed motion bitstream).

相应的，在解码过程中，解码器按照作用帧的类型不同，也可以分为帧内解码器和帧间解码器，分别用于执行帧内解码和帧间解码。Correspondingly, during the decoding process, the decoder can also be divided into an intra-frame decoder and an inter-frame decoder according to the type of the frame it acts on, which are used to perform intra-frame decoding and inter-frame decoding respectively.

图11A为帧内解码的示意图，如图11A所示，在帧内解码器中，可以采用静态网格解码器(Static Mesh Decoder)解码出简化网格。采用视频解码器(Video Decoder)解码出移位系数视频，并通过图像解包(Image Unpacking)和小波逆变换(Inverse Wavelet Transform)得到移位系数。通过解码得到的简化网格和移位系数得到解码的网格几何信息。属性图的解码则直接通过视频解码器进行解码。FIG11A is a schematic diagram of intra-frame decoding. As shown in FIG11A , in the intra-frame decoder, a static mesh decoder (Static Mesh Decoder) can be used to decode the simplified mesh. A video decoder (Video Decoder) is used to decode the shift coefficient video, and the shift coefficient is obtained through image unpacking (Image Unpacking) and inverse wavelet transform (Inverse Wavelet Transform). The decoded simplified mesh and shift coefficient are used to obtain the decoded mesh geometry information. The decoding of the attribute graph is directly decoded through the video decoder.

图11B为帧间解码的示意图，如图11B所示，对于帧间解码器，其流程与帧内解码器基本一致。除了不直接解码简化网格，而是解码运动矢量，并通过前一帧(参考帧)的简化网格计算得到当前帧的简化网格。FIG11B is a schematic diagram of inter-frame decoding. As shown in FIG11B , for an inter-frame decoder, the process is basically the same as that of an intra-frame decoder, except that the simplified grid is not directly decoded, but the motion vector is decoded, and the simplified grid of the current frame is calculated by the simplified grid of the previous frame (reference frame).

综上，目前运动图像专家组(Moving Picture Experts Group，MPEG)提供的动态网格编码(Dynamic Mesh Coding)的标准参考软件(下称标准参考软件)中，动态网格编码过程分为如下步骤：In summary, in the standard reference software for dynamic mesh coding (Dynamic Mesh Coding) provided by the Moving Picture Experts Group (MPEG) (hereinafter referred to as the standard reference software), the dynamic mesh coding process is divided into the following steps:

1、对原始网格进行预处理，具体表现为减少网格中的顶点数并简化连接关系。1. Preprocess the original mesh by reducing the number of vertices in the mesh and simplifying the connection relationship.

2、细分步骤1中的简化网格，具体操作为对于步骤1中任意有连接关系的两个顶点，在其连线段的中点加入一个新的点，并重复2次。2. Subdivide the simplified mesh in step 1. For any two connected vertices in step 1, add a new point at the midpoint of the line segment connecting them, and repeat twice.

3、对于步骤2中的每个顶点，寻找原始网格中与其距离最近的点，并计算这两个点的移位系数(Displacement)。3. For each vertex in step 2, find the point in the original mesh that is closest to it and calculate the displacement coefficient of the two points.

4、采用编码器如Draco对步骤1中的简化网格量化后进行编码。4. Use an encoder such as Draco to quantize the simplified grid in step 1 and then encode it.

5、根据步骤4所得到的重构简化网格对步骤3中的移位系数进行调整。5. Adjust the shift coefficients in step 3 according to the reconstructed simplified grid obtained in step 4.

6、对步骤5中的移位系数进行小波变换，并将小波变换后的移位系数进行量化得到量化变换系数。6. Perform wavelet transform on the shift coefficients in step 5, and quantize the shift coefficients after wavelet transform to obtain quantized transform coefficients.

7、将量化变换系数进行三维空间到二维图像的映射(或称“图像打包”)，生成移位系数二维图像。7. Map the quantized transform coefficients from three-dimensional space to a two-dimensional image (or "image packing") to generate a two-dimensional image of shifted coefficients.

8、采用标准视频编码器如H.265对步骤6中的移位系数二维图像进行编码。8. Use a standard video encoder such as H.265 to encode the shift coefficient two-dimensional image in step 6.

动态网格解码过程分为如下步骤：The dynamic grid decoding process is divided into the following steps:

1、基础网格码流经过解码器如draco解码之后生成解码基础网格。1. The basic grid code stream is decoded by a decoder such as draco to generate a decoded basic grid.

2、移位系数码流采用标准视频编码器如H.265解码得到移位系数二维图像。2. The shift coefficient bit stream is decoded using a standard video encoder such as H.265 to obtain a shift coefficient two-dimensional image.

3、将移位系数二维图像进行从二维图像到三维空间的映射(或称“图像解包”)，得到量化变换系数。3. Map the shift coefficient two-dimensional image from the two-dimensional image to the three-dimensional space (or "image unpacking") to obtain the quantized transform coefficients.

4、对量化变换系数进行反量化、小波反变换得到解码移位系数。4. Dequantize and inverse wavelet transform the quantized transform coefficients to obtain the decoded shift coefficients.

5、将解码基础网格和解码移位系数一起生成重建三维网格几何信息。5. The decoded base grid and the decoded shift coefficients are combined to generate the reconstructed 3D grid geometry information.

6、属性码流经过HEVC解码之后生成重建属性图。6. After the attribute code stream is decoded by HEVC, a reconstructed attribute graph is generated.

为了解决上述问题，本申请实施例提供了一种编解码方法，在预测待编解码单元的第一顶点集合的几何位置信息时，自适应的跳过一些顶点移位系数的编解码，而是利用表征初始几何位置信息和原始几何位置信息相关性的映射关系进行拟合，确定初始几何位置信息对应的第一预测几何位置信息，降低移位系数的码率，在保证网格顶点的几何位置信息的重建质量的基础上，提高几何位置信息的编解码效率。In order to solve the above problems, an embodiment of the present application provides a coding and decoding method, which adaptively skips the coding and decoding of some vertex shift coefficients when predicting the geometric position information of the first vertex set of the unit to be coded and decoded, and instead uses a mapping relationship that characterizes the correlation between the initial geometric position information and the original geometric position information for fitting, determines the first predicted geometric position information corresponding to the initial geometric position information, reduces the bit rate of the shift coefficient, and improves the coding and decoding efficiency of the geometric position information on the basis of ensuring the reconstruction quality of the geometric position information of the mesh vertices.

本申请实施例提供了一种包含解码方法和编码方法的编解码系统的网络架构，图12为本申请实施例提供的一种编解码的网络架构示意图。如图12所示，该网络架构包括一个或多个电子设备13至1N和通信网络01，其中，电子设备13至1N可以通过通信网络01进行视频交互。电子设备在实施的过程中可以为各种类型的具有编解码功能的设备，例如，所述电子设备可以包括手机、平板电脑、个人计算机、个人数字助理、导航仪、数字电话、视频电话、电视机、传感设备、服务器等，本申请实施例不作限制。其中，本申请实施例中的解码器或编码器就可以为上述电子设备。The embodiment of the present application provides a network architecture of a codec system including a decoding method and an encoding method, and Figure 12 is a schematic diagram of a network architecture of a codec provided by the embodiment of the present application. As shown in Figure 12, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During the implementation process, the electronic device can be various types of devices with codec functions, for example, the electronic device can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensor device, a server, etc., which is not limited by the embodiment of the present application. Among them, the decoder or encoder in the embodiment of the present application can be the above-mentioned electronic device.

下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

本申请的实施例提出了一种解码方法，图13为本申请实施例提供的一种解码方法的流程示意图，如图13所示，在本申请的实施例中，解码器进行解码处理的方法可以包括以下步骤：The embodiment of the present application proposes a decoding method. FIG. 13 is a flowchart of a decoding method provided by the embodiment of the present application. As shown in FIG. 13 , in the embodiment of the present application, the method for the decoder to perform decoding processing may include the following steps:

步骤101：确定基础网格；Step 101: Determine the basic grid;

在一些实施例中，确定当前图像的基础网格，当前图像可以为待编码网格图像。在一些实施例中，确定当前图像块的基础网格，当前图像块可以为当前待编码网格图像块。具体地，网格图像可以为如图1所示的三维网格图像。In some embodiments, the base grid of the current image is determined, and the current image may be a grid image to be encoded. In some embodiments, the base grid of the current image block is determined, and the current image block may be a current grid image block to be encoded. Specifically, the grid image may be a three-dimensional grid image as shown in FIG1.

在一些实施例中，对于帧内编码，解码基础网格码流，确定基础网格(Base mesh)，也称简化网格。In some embodiments, for intra-frame coding, the base mesh code stream is decoded to determine the base mesh (Base mesh), also known as the simplified mesh.

在一些实施例中，对于帧间编码，根据参考图像块的基础网格，确定当前图像块的基础网格。In some embodiments, for inter-frame coding, the base grid of the current image block is determined according to the base grid of the reference image block.

步骤102：对基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；Step 102: subdividing the basic mesh to determine initial geometric position information of vertices included in the first vertex set in the initial mesh;

在一些实施例中，根据预设的网格细分模式对基础网格进行迭代细分，确定初始网格中顶点的初始几何位置信息；从初始网格中确定第一顶点集合中顶点的初始几何位置信息。In some embodiments, the base mesh is iteratively subdivided according to a preset mesh subdivision mode to determine the initial vertices in the initial mesh. Geometric position information; determining initial geometric position information of vertices in a first vertex set from an initial mesh.

需要说明的是，网格细分模式可以理解为对基础网格每个边界上的顶点进行插值，以在基础网格的结构上得到更多顶点。示例性的，网格细分模式包括细分算法和细分迭代次数。在一些实施例中，细分算法可以为线性插值算法或者非线性插值算法。It should be noted that the mesh subdivision mode can be understood as interpolating the vertices on each boundary of the base mesh to obtain more vertices on the structure of the base mesh. Exemplarily, the mesh subdivision mode includes a subdivision algorithm and a number of subdivision iterations. In some embodiments, the subdivision algorithm can be a linear interpolation algorithm or a nonlinear interpolation algorithm.

示例性的，完成Base mesh的解码重建之后，会利用一定的细分算法，来对Base mesh进行细分得到初始网格。示例性的，基础网格通过线性插值算法，得到初始网格(也可以称为细分网格)。其中，每次迭代过程中新插入的点坐标根据当前边界上的两个顶点的坐标进行线性插值得到：
Exemplarily, after the decoding and reconstruction of the base mesh is completed, a certain subdivision algorithm is used to subdivide the base mesh to obtain an initial mesh. Exemplarily, the base mesh is obtained by a linear interpolation algorithm to obtain an initial mesh (also called a subdivided mesh). In each iteration, the coordinates of the newly inserted points are obtained by linear interpolation based on the coordinates of the two vertices on the current boundary:

其中，pos₁和pos₂为参与本次迭代的当前边界上的两个顶点几何位置坐标，pos_new为本次迭代新增顶点的几何位置坐标。Among them, pos ₁ and pos ₂ are the geometric position coordinates of two vertices on the current boundary participating in this iteration, and pos _new is the geometric position coordinates of the vertex newly added in this iteration.

利用每个边界上的顶点进行线性插值，得到对应的几何位置信息。假设，整个划分迭代了N次，则对基础网格进行细节层次(Level of Detail，LOD)划分。图14A为一种基础网格迭代细分示意图，如图14A所示，对基础网格经过3次迭代细分得到的初始网格的几何位置信息，基础网格视为第0次迭代对应第0层(level0)，第一次迭代新增顶点构成第1层(level1)，第二次迭代新增顶点构成第2层(level2)，第二次迭代新增顶点构成第2层(level2)利用初始得到的几何位置信息与原始网格进行误差计算，得到每个点的移位系数(Displacement)。Linear interpolation is performed using the vertices on each boundary to obtain the corresponding geometric position information. Assuming that the entire division is iterated N times, the basic mesh is divided into Level of Detail (LOD). Figure 14A is a schematic diagram of iterative subdivision of a basic mesh. As shown in Figure 14A, the geometric position information of the initial mesh obtained after three iterative subdivisions of the basic mesh, the basic mesh is regarded as the 0th iteration corresponding to the 0th layer (level0), the first iteration newly added vertices constitute the 1st layer (level1), the second iteration newly added vertices constitute the 2nd layer (level2), and the second iteration newly added vertices constitute the 2nd layer (level2). The initial geometric position information is used to calculate the error with the original mesh to obtain the displacement coefficient (Displacement) of each point.

具体的LOD空间结构如图14B所示，顶层为基础网格，随着迭代的进行，每次迭代过程中新增顶点数依次增加，形成金字塔结构。The specific LOD spatial structure is shown in FIG14B . The top layer is the basic mesh. As the iteration proceeds, the number of newly added vertices increases successively during each iteration, forming a pyramid structure.

第一顶点集合为待解码单元所包含顶点的集合。待解码单元可以为任一种图像划分结构下得到的。示例性的，第一顶点集合可以包括待解码图像块(Block)包含的顶点(如图16所示的一个Block)，第一顶点集合可以包括待解码LOD层包含的顶点(如图14B所示的一个LOD层)。The first vertex set is a set of vertices included in the unit to be decoded. The unit to be decoded can be obtained under any image division structure. Exemplarily, the first vertex set may include vertices included in the image block (Block) to be decoded (such as a Block as shown in FIG16 ), and the first vertex set may include vertices included in the LOD layer to be decoded (such as a LOD layer as shown in FIG14B ).

在一些实施例中，LOD层的解码顺序是从顶层到底层，解码时可以按照从第一次迭代到最后一次迭代的解码顺序，依次解码LOD层。In some embodiments, the decoding order of the LOD layers is from the top layer to the bottom layer, and during decoding, the LOD layers may be decoded in sequence according to the decoding order from the first iteration to the last iteration.

步骤103：解码码流，确定第一顶点集合对应的映射关系；Step 103: Decode the code stream to determine the mapping relationship corresponding to the first vertex set;

其中，映射关系用于指示第一顶点集合中顶点的初始几何位置信息和原始几何位置信息的相关性。具体可以通过一定的参数关系来表征二者的相关性，如图15A和图15B所示。通过编解码映射关系，解码端根据映射关系可以直接确定初始网格中顶点的初始几何位置信息对应的重建网格中顶点的第一预测几何位置信息，无需编解码顶点的移位系数，在保证网格顶点的几何位置信息的重建质量的基础上，提高几何位置信息的编解码效率。Among them, the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set. Specifically, the correlation between the two can be characterized by a certain parameter relationship, as shown in Figures 15A and 15B. Through the encoding and decoding mapping relationship, the decoding end can directly determine the first predicted geometric position information of the vertices in the reconstructed mesh corresponding to the initial geometric position information of the vertices in the initial mesh according to the mapping relationship, without the need to encode and decode the shift coefficients of the vertices, thereby improving the encoding and decoding efficiency of the geometric position information while ensuring the reconstruction quality of the geometric position information of the mesh vertices.

在一些实施例中，解码码流，确定第一顶点集合对应的映射关系，包括：解码码流，确定第一顶点集合的映射关系指示信息；根据映射关系指示信息，确定第一顶点集合对应的映射关系。示例性的，映射关系指示信息可以用于指示映射关系的参数，还可以直接指示某一预设的映射关系。不同顶点集合对应的映射关系可以相同也可以不相同。示例性的，映射关系指示信息可以包括LOD层级或者片级语法元素。映射关系指示信息还可以包括序列级和/或图像级语法元素。In some embodiments, decoding a code stream and determining a mapping relationship corresponding to a first vertex set includes: decoding a code stream and determining mapping relationship indication information of the first vertex set; and determining the mapping relationship corresponding to the first vertex set according to the mapping relationship indication information. Exemplarily, the mapping relationship indication information may be used to indicate parameters of the mapping relationship, and may also directly indicate a preset mapping relationship. The mapping relationships corresponding to different vertex sets may be the same or different. Exemplarily, the mapping relationship indication information may include LOD level or slice level syntax elements. The mapping relationship indication information may also include sequence level and/or image level syntax elements.

在一些实施例中，映射关系为第一顶点集合中顶点的初始几何位置信息和预测几何位置信息的函数关系。示例性的，映射关系包括以下至少之一：恒等映射关系、基于线性函数的映射关系、基于非线性函数的映射关系、基于神经网络的映射关系。需要说明的是，恒等映射关系也可以属于一种特殊的基于线性函数的映射关系。In some embodiments, the mapping relationship is a functional relationship between the initial geometric position information and the predicted geometric position information of the vertices in the first vertex set. Exemplarily, the mapping relationship includes at least one of the following: an identity mapping relationship, a mapping relationship based on a linear function, a mapping relationship based on a nonlinear function, and a mapping relationship based on a neural network. It should be noted that the identity mapping relationship can also belong to a special mapping relationship based on a linear function.

示例性的，映射关系的函数表达式可以为：
predict_mesh＝f(reconMesh,i)Exemplarily, the function expression of the mapping relationship may be:
predict _mesh =f(reconMesh,i)

其中，predict_mesh为通过拟合得到的预测几何位置信息，reconMesh为初始几何位置信息，i为第一顶点集合的标识信息，f(·)为映射关系。Among them, predict _mesh is the predicted geometric position information obtained by fitting, reconMesh is the initial geometric position information, i is the identification information of the first vertex set, and f(·) is the mapping relationship.

在一些实施例中，映射关系指示信息可以包括第一指示信息，其中，第一指示信息用于指示映射关系的常数；根据映射关系指示信息，确定第一顶点集合对应的映射关系，包括：根据第一指示信息，确定映射关系的至少一个常数；根据映射关系的至少一个常数，确定映射关系。In some embodiments, the mapping relationship indication information may include first indication information, wherein the first indication information is used to indicate a constant of the mapping relationship; determining the mapping relationship corresponding to the first vertex set according to the mapping relationship indication information includes: determining at least one constant of the mapping relationship according to the first indication information; determining the mapping relationship according to at least one constant of the mapping relationship.

示例性的，映射关系为基于线性函数的映射关系时，第一指示信息用于指示线性函数的斜率和/或截距。Exemplarily, when the mapping relationship is a mapping relationship based on a linear function, the first indication information is used to indicate the slope and/or intercept of the linear function.

示例性的，映射关系为基于非线性函数的映射关系时，第一指示信息用于指示非线性函数中的至少一个常数。示例性的，非线性函数为指数函数，第一指示信息用于指示非线性函数的常数a。非线性函数为多项式函数，第一指示信息用于指示多项式函数的系数a₀，a₁，a₂…。非线性函数为对数函数，第一指示信息用于指示非线性函数的底数a。 Exemplarily, when the mapping relationship is a mapping relationship based on a nonlinear function, the first indication information is used to indicate at least one constant in the nonlinear function. Exemplarily, the nonlinear function is an exponential function, and the first indication information is used to indicate a constant a of the nonlinear function. The nonlinear function is a polynomial function, and the first indication information is used to indicate coefficients a ₀ , a ₁ , a ₂ … of the polynomial function. The nonlinear function is a logarithmic function, and the first indication information is used to indicate a base a of the nonlinear function.

示例性的，映射关系为基于神经网格的映射关系时，第一指示信息用于指示神经网格的卷积参数，或者，第一指示信息用于指示是否使用神经网络作为映射关系，或者，第一指示信息用于指示使用哪一个神经网络作为映射关系。Exemplarily, when the mapping relationship is a mapping relationship based on a neural grid, the first indication information is used to indicate the convolution parameters of the neural grid, or the first indication information is used to indicate whether a neural network is used as the mapping relationship, or the first indication information is used to indicate which neural network is used as the mapping relationship.

在一些实施例中，映射关系指示信息还可以包括第二指示信息，其中，第二指示信息用于指示映射关系的类型；根据映射关系指示信息，确定第一顶点集合对应的映射关系，还包括：根据第二指示信息，确定映射关系的类型；根据映射关系的类型和至少一个常数，确定映射关系。In some embodiments, the mapping relationship indication information may also include second indication information, wherein the second indication information is used to indicate the type of the mapping relationship; determining the mapping relationship corresponding to the first vertex set according to the mapping relationship indication information also includes: determining the type of the mapping relationship according to the second indication information; determining the mapping relationship according to the type of the mapping relationship and at least one constant.

需要说明的是，当通过预设的一种映射关系表征几何位置信息之间的相关性时，可以通过第一指示信息确定该映射关系的常数，当通过预设的两种及以上映射关系表征相关性时，进一步还可以通过第二指示信息确定映射关系的类型。It should be noted that when the correlation between geometric position information is represented by a preset mapping relationship, the constant of the mapping relationship can be determined by the first indication information; when the correlation is represented by two or more preset mapping relationships, the type of the mapping relationship can be further determined by the second indication information.

在一些实施例中，映射关系指示信息用于指示映射关系的索引信息(Idx)；根据映射关系指示信息，确定第一顶点集合对应的映射关系，包括：根据映射关系指示信息，确定映射关系的索引信息；根据映射关系的索引信息，确定映射关系。In some embodiments, the mapping relationship indication information is used to indicate the index information (Idx) of the mapping relationship; according to the mapping relationship indication information, the mapping relationship corresponding to the first vertex set is determined, including: according to the mapping relationship indication information, determining the index information of the mapping relationship; determining the mapping relationship according to the index information of the mapping relationship.

具体地，考虑编码复杂度与编码效率的平衡，可以利用简单的线性拟合来得到原始网格几何位置信息与初始网格几何位置信息间的关系，具体的如下所示：
predict_mesh＝k*(reconMesh,i)+bSpecifically, considering the balance between coding complexity and coding efficiency, a simple linear fitting can be used to obtain the relationship between the original grid geometric position information and the initial grid geometric position information, as shown below:
predict _mesh =k*(reconMesh,i)+b

其中，predict_mesh为预测几何位置信息，reconMesh为初始几何位置信息，i为第一顶点集合的标识信息，f(·)为映射关系，*和+为向量的数乘和加法。Wherein, predict _mesh is the predicted geometric position information, reconMesh is the initial geometric position information, i is the identification information of the first vertex set, f(·) is the mapping relationship, and * and + are vector multiplication and addition.

在一些实施例中，该方法还包括：解码码流，确定第一顶点集合的第三指示信息；根据第三指示信息确定解码第一顶点集合的映射关系的情况下，解码码流，确定第一顶点集合对应的映射关系。也就是说，第三指示信息用于指示是否解码第一顶点集合的映射关系，在解码映射关系之前，需要先解码第三指示信息，根据第三指示信息确定解码第一顶点集合的映射关系，才进一步解码映射关系指示信息。示例性的，第三指示信息可以为网格编解码中新增语法元素，也可以复用已有语法元素。第三指示信息可以包括以下至少之一：序列级、帧级、LOD层级、片级语法元素。In some embodiments, the method further includes: decoding the code stream to determine the third indication information of the first vertex set; when the mapping relationship for decoding the first vertex set is determined according to the third indication information, decoding the code stream to determine the mapping relationship corresponding to the first vertex set. That is, the third indication information is used to indicate whether to decode the mapping relationship of the first vertex set. Before decoding the mapping relationship, it is necessary to first decode the third indication information, determine the mapping relationship for decoding the first vertex set according to the third indication information, and then further decode the mapping relationship indication information. Exemplarily, the third indication information can be a newly added syntax element in the mesh encoding and decoding, or it can reuse an existing syntax element. The third indication information can include at least one of the following: sequence level, frame level, LOD level, and slice level syntax elements.

步骤104：根据第一顶点集合中顶点的初始几何位置信息和映射关系，确定第一顶点集合中顶点的第一预测几何位置信息。Step 104: Determine first predicted geometric position information of the vertices in the first vertex set according to the initial geometric position information and the mapping relationship of the vertices in the first vertex set.

在一些实施例中，映射关系第一顶点集合中顶点的初始几何位置信息和预测几何位置信息的函数关系。具体地，将初始几何位置信息作为映射关系的输入值，得到映射关系的输出值作为第一预测几何位置信息。In some embodiments, the mapping relationship is a functional relationship between the initial geometric position information and the predicted geometric position information of the vertices in the first vertex set. Specifically, the initial geometric position information is used as an input value of the mapping relationship, and the output value of the mapping relationship is obtained as the first predicted geometric position information.

示例性的，以LOD空间结构为例，第一顶点集合包括当前LOD层的顶点，映射关系的函数表达式还可以为：
predict_mesh＝f(reconMesh,lvl)Exemplarily, taking the LOD space structure as an example, the first vertex set includes the vertices of the current LOD layer, and the function expression of the mapping relationship can also be:
predict _mesh =f(reconMesh,lvl)

其中，predict_mesh为通过拟合得到的预测几何位置信息，reconMesh为初始几何位置信息，lvl为不同的LOD层的标识信息，f(·)为映射关系。Among them, predict _mesh is the predicted geometric position information obtained by fitting, reconMesh is the initial geometric position information, lvl is the identification information of different LOD layers, and f(·) is the mapping relationship.

具体地，若利用简单的线性拟合来得到原始网格几何位置信息与初始网格几何位置信息间的关系，具体的如下所示：
predict_mesh＝k*(reconMesh,lvl)+bSpecifically, if a simple linear fitting is used to obtain the relationship between the original grid geometric position information and the initial grid geometric position information, the specific relationship is as follows:
predict _mesh =k*(reconMesh,lvl)+b

其中，predict_mesh为通过拟合得到的预测几何位置信息，reconMesh为初始几何位置信息，lvl为不同的LOD层的标识信息，f(·)为映射关系，*和+为向量的数乘和加法。Among them, predict _mesh is the predicted geometric position information obtained by fitting, reconMesh is the initial geometric position information, lvl is the identification information of different LOD layers, f(·) is the mapping relationship, and * and + are vector multiplication and addition.

在一些实施例中，在一些实施例中，该方法还可以包括：解码码流，确定第一顶点集合中顶点的第一移位系数；根据第一顶点集合中顶点的初始几何位置信息和第一移位系数，确定第一顶点集合中顶点的第二预测几何位置信息。也就是说，解码端也可以解码移位系数，根据初始几何位置信息和移位系数确定第二预测几何位置信息。In some embodiments, in some embodiments, the method may further include: decoding the code stream to determine a first shift coefficient of the vertices in the first vertex set; and determining second predicted geometric position information of the vertices in the first vertex set based on the initial geometric position information of the vertices in the first vertex set and the first shift coefficient. That is, the decoding end may also decode the shift coefficient and determine the second predicted geometric position information based on the initial geometric position information and the shift coefficient.

在一些实施例中，解码第一顶点集合的第四指示信息；根据第四指示信息确定解码第一顶点集合中顶点的移位系数的情况下，解码码流，确定第一顶点集合中顶点的第一移位系数；根据第一顶点集合中顶点的初始几何位置信息和第一移位系数，确定第一顶点集合中顶点的第二预测几何位置信息。也就是说，第四指示信息用于指示是否解码第一顶点集合的移位系数，在解码移位系数之前，需要先解码第四指示信息，根据第四指示信息确定解码第一顶点集合的移位系数，才进一步解码移位系数。示例性的，第四指示信息可以为网格编解码中新增语法元素，也可以复用已有语法元素。第四指示信息可以包括以下至少之一：序列级、帧级、LOD层级、片级语法元素。In some embodiments, the fourth indication information of the first vertex set is decoded; when the shift coefficients of the vertices in the first vertex set are determined according to the fourth indication information, the code stream is decoded to determine the first shift coefficients of the vertices in the first vertex set; the second predicted geometric position information of the vertices in the first vertex set is determined according to the initial geometric position information of the vertices in the first vertex set and the first shift coefficient. In other words, the fourth indication information is used to indicate whether to decode the shift coefficients of the first vertex set. Before decoding the shift coefficients, the fourth indication information needs to be decoded first, and the shift coefficients of the first vertex set need to be determined according to the fourth indication information before further decoding the shift coefficients. Exemplarily, the fourth indication information can be a new syntax element added to the mesh encoding and decoding, or an existing syntax element can be reused. The fourth indication information can include at least one of the following: sequence level, frame level, LOD level, and slice level syntax elements.

在一些实施例中，该方法还可以包括：解码第五指示信息；根据第五指示信息确定第一顶点集合中顶点的移位系数为预设系数。也就是说，第一顶点集合中所有顶点的移位系数还可以为相同系数，此时无需解码每个顶点的移位系数也无需解码映射关系。示例性的，预设系数可以为0。 In some embodiments, the method may further include: decoding fifth indication information; determining that the shift coefficients of the vertices in the first vertex set are preset coefficients according to the fifth indication information. That is, the shift coefficients of all vertices in the first vertex set may also be the same coefficients, and in this case, there is no need to decode the shift coefficient of each vertex or the mapping relationship. Exemplarily, the preset coefficient may be 0.

需要说明的是，上述指示信息可以通过设置一个语法元素的取值来指示，也可以通过一个语法元素的不同取值来作为不同指示信息，也可以通过二进制字符串来指示。It should be noted that the above indication information can be indicated by setting the value of a syntax element, or by using different values of a syntax element as different indication information, or by a binary character string.

在一些实施例中，解码码流，确定第一顶点集合中顶点的第一移位系数，包括：采用视频解码器解码码流，确定第一顶点集合中顶点的第一移位系数；利用熵解码器解码码流，确定第一顶点集合中顶点的第一移位系数。In some embodiments, decoding a bitstream to determine first shift coefficients of vertices in a first vertex set includes: decoding the bitstream using a video decoder to determine first shift coefficients of vertices in the first vertex set; and decoding the bitstream using an entropy decoder to determine first shift coefficients of vertices in the first vertex set.

示例性的，利用视频解码器解码移位系数码流，得到移位系数的二维图像，再对二维图像进行解压缩得到变换量化后的移位系数。如图16所示，利用Video-Codec来对二维图像进行解码重建恢复得到对应的二维图像。其次，按照系数重组的方式，恢复得到每个点对应的提升变换系数。最终利用提升小波变换的逆变换恢复得到每个点的Displacement值，在得到每个点的Displacement之后，利用根据Base mesh细分得到的初始几何位置信息与Displacement重建恢复得到当前mesh对应的几何位置信息，如图17所示。Exemplarily, a video decoder is used to decode the shift coefficient bitstream to obtain a two-dimensional image of the shift coefficient, and then the two-dimensional image is decompressed to obtain the transformed and quantized shift coefficient. As shown in FIG16 , the Video-Codec is used to decode and reconstruct the two-dimensional image to obtain the corresponding two-dimensional image. Secondly, the lifting transform coefficient corresponding to each point is recovered in accordance with the coefficient reorganization method. Finally, the inverse transform of the lifting wavelet transform is used to recover the Displacement value of each point. After obtaining the Displacement of each point, the initial geometric position information obtained according to the subdivision of the Base mesh and the Displacement are used to reconstruct and recover the geometric position information corresponding to the current mesh, as shown in FIG17 .

示例性的，利用熵解码器解码移位系数码流，得到变换量化后的移位系数，再进行反量化和提升小波变换的逆变换恢复得到每个点的Displacement值。Exemplarily, an entropy decoder is used to decode the displacement coefficient code stream to obtain the transformed and quantized displacement coefficients, which are then dequantized and restored by inverse transformation of the lifting wavelet transform to obtain the Displacement value of each point.

在一些实施例中，根据第一顶点集合中顶点的初始几何位置信息和第一移位系数，确定第一顶点集合中顶点的第二预测几何位置信息，包括：对第一移位系数进行反量化和提升小波变换的逆变换，得到第二移位系数；根据根据第一顶点集合中顶点的初始几何位置信息和第二移位系数，确定第二预测几何位置信息。也就是说，第一移位系数可以为量化后的移位系数，第二移位系数可以为反量化和反变换后的移位系数。In some embodiments, determining the second predicted geometric position information of the vertices in the first vertex set according to the initial geometric position information of the vertices in the first vertex set and the first shift coefficient includes: performing inverse quantization and inverse transformation of the lifting wavelet transform on the first shift coefficient to obtain the second shift coefficient; determining the second predicted geometric position information according to the initial geometric position information of the vertices in the first vertex set and the second shift coefficient. That is, the first shift coefficient may be a quantized shift coefficient, and the second shift coefficient may be a dequantized and inversely transformed shift coefficient.

在上述实施例的基础上，对本申请实施例提供的解码方法进行进一步地举例说明，具体实现步骤可以包括：Based on the above embodiment, the decoding method provided in the embodiment of the present application is further illustrated by example, and the specific implementation steps may include:

首先，利用Video-Codec解析得到移位系数的二维图像，在利用二维图像恢复得到点的移位系数(Displacement)之前，需要判断当前LOD层中顶点的Displacement是否需要进行解码。First, a two-dimensional image of the displacement coefficient is obtained by parsing the Video-Codec. Before the displacement coefficient (Displacement) of the point is restored by using the two-dimensional image, it is necessary to determine whether the displacement of the vertex in the current LOD layer needs to be decoded.

如果当前层的Displacement需要解码，则利用当前层顶点的Base mesh的细分得到的初始点位置信息与Displacement进行重建恢复得到当前层顶点的重建点位置信息；If the displacement of the current layer needs to be decoded, the initial point position information obtained by subdividing the base mesh of the current layer vertex is reconstructed and restored with the displacement to obtain the reconstructed point position information of the current layer vertex;

如果当前层的Displacement跳过编码，即不需要解码，则直接利用当前层Base mesh的细分得到的初始点位置信息与一定的参数关系进行拟合恢复得到当前层点的重建位置信息。If the Displacement of the current layer skips encoding, that is, decoding is not required, the initial point position information obtained by subdividing the Base mesh of the current layer is directly used to fit and restore a certain parameter relationship to obtain the reconstructed position information of the current layer point.

本申请实施例还提供了一种编码方法，图18为本申请实施例提供的一种编码方法的流程示意图，如图18所示，在本申请的实施例中，编码器进行编码处理的方法可以包括以下步骤：The embodiment of the present application further provides an encoding method. FIG. 18 is a flow chart of an encoding method provided by the embodiment of the present application. As shown in FIG. 18 , in the embodiment of the present application, the method for encoding by the encoder may include the following steps:

步骤201：根据原始网格，确定基础网格；Step 201: Determine a basic grid according to the original grid;

示例性的，对原始网格进行下采样确定基础网格。Exemplarily, the original grid is downsampled to determine the base grid.

在一些实施例中，根据当前图像的原始网格，确定当前图像的基础网格，当前图像可以为当前待编码网格图像。在一些实施例中，根据当前图像块的原始网格，确定当前图像块的基础网格，当前图像块可以为当前待编码网格图像块。具体地，网格图像可以为如图1所示的三维网格图像。In some embodiments, the base grid of the current image is determined according to the original grid of the current image, and the current image may be the current grid image to be encoded. In some embodiments, the base grid of the current image block is determined according to the original grid of the current image block, and the current image block may be the current grid image block to be encoded. Specifically, the grid image may be a three-dimensional grid image as shown in FIG1.

步骤202：对基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；Step 202: subdividing the basic mesh to determine initial geometric position information of vertices included in the first vertex set in the initial mesh;

在一些实施例中，根据预设的网格细分模式对基础网格进行迭代细分，确定初始网格中顶点的初始几何位置信息；从初始网格中确定第一顶点集合中顶点的初始几何位置信息。In some embodiments, the base mesh is iteratively subdivided according to a preset mesh subdivision mode to determine initial geometric position information of vertices in the initial mesh; and initial geometric position information of vertices in the first vertex set is determined from the initial mesh.

需要说明的是，网格细分模式可以理解为对基础网格每个边界上的顶点进行插值，以在基础网格的结构上得到更多顶点。示例性的，网格细分模式包括细分算法和细分迭代次数。在一些实施例中，细分算法可以为线性插值算法，非线性插值算法。It should be noted that the mesh subdivision mode can be understood as interpolating the vertices on each boundary of the base mesh to obtain more vertices on the structure of the base mesh. Exemplarily, the mesh subdivision mode includes a subdivision algorithm and a number of subdivision iterations. In some embodiments, the subdivision algorithm can be a linear interpolation algorithm or a nonlinear interpolation algorithm.

示例性的，完成Base mesh的编码重建之后，会利用一定的细分算法，来对Base mesh进行细分得到初始网格。示例性的，基础网格通过线性插值算法，得到初始网格(也可以称为细分网格)。其中，每次迭代过程中新插入的点坐标根据当前边界上的两个顶点的坐标进行线性插值得到：
Exemplarily, after the encoding and reconstruction of the base mesh is completed, a certain subdivision algorithm is used to subdivide the base mesh to obtain an initial mesh. Exemplarily, the base mesh is obtained by a linear interpolation algorithm to obtain an initial mesh (also called a subdivided mesh). In each iteration, the coordinates of the newly inserted points are obtained by linear interpolation based on the coordinates of the two vertices on the current boundary:

利用每个边界上的顶点进行线性插值，得到对应的几何位置信息。假设，整个划分迭代了N次，则对基础网格进行细节层次(Level of Detail，LOD)划分。图14A为一种基础网格迭代细分示意图，如图14A所示，对基础网格经过3次迭代细分得到的初始网格的几何位置信息，基础网格视为第0次迭代对应第0层(level0)，第一次迭代新增顶点构成第1层(level1)，第二次迭代新增顶点构成第2层(level2)，第二次迭代新增顶点构成第2层(level2)利用初始得到的几何位置信息与原始的mesh进行误差计算，得到每个点的移位系数(Displacement)。 Linear interpolation is performed using the vertices on each boundary to obtain the corresponding geometric position information. Assuming that the entire division is iterated N times, the basic mesh is divided into levels of detail (LOD). Figure 14A is a schematic diagram of iterative subdivision of a basic mesh. As shown in Figure 14A, the geometric position information of the initial mesh obtained after three iterative subdivisions of the basic mesh, the basic mesh is regarded as the 0th iteration corresponding to the 0th layer (level0), the first iteration newly added vertices constitute the 1st layer (level1), the second iteration newly added vertices constitute the 2nd layer (level2), and the second iteration newly added vertices constitute the 2nd layer (level2). The initial geometric position information is used to calculate the error with the original mesh to obtain the displacement coefficient (Displacement) of each point.

第一顶点集合为待编码单元所包含顶点的集合。待编码单元可以为任一种图像划分结构下得到的。示例性的，第一顶点集合可以包括待编码图像块(Block)包含的顶点(如图16所示的一个Block)，第一顶点集合可以包括待编码LOD层包含的顶点(如图14B所示的一个LOD层)。The first vertex set is a set of vertices included in the unit to be encoded. The unit to be encoded can be obtained under any image division structure. Exemplarily, the first vertex set may include vertices included in the image block (Block) to be encoded (a Block as shown in FIG. 16 ), and the first vertex set may include vertices included in the LOD layer to be encoded (a LOD layer as shown in FIG. 14B ).

在一些实施例中，LOD层的编码顺序是从底层到顶层，编码时可以按照从最后一次迭代到第一次迭代的编码顺序，依次编码LOD层。In some embodiments, the encoding order of the LOD layers is from the bottom layer to the top layer. During encoding, the LOD layers may be encoded in sequence according to the encoding order from the last iteration to the first iteration.

步骤203：根据原始网格中第一顶点集合所包含顶点的原始几何位置信息和初始几何位置信息，确定第一顶点集合对应的映射关系；Step 203: determining a mapping relationship corresponding to the first vertex set according to the original geometric position information and the initial geometric position information of the vertices included in the first vertex set in the original mesh;

在一些实施例中，根据原始网格中第一顶点集合所包含顶点的原始几何位置信息和初始几何位置信息，确定第一顶点集合对应的映射关系，包括：根据第一顶点集合的原始几何位置信息和初始几何位置信息，以及预设的函数关系进行函数拟合；确定最小误差参考值对应的最佳拟合函数为第一顶点集合对应的映射关系。In some embodiments, a mapping relationship corresponding to the first vertex set is determined based on the original geometric position information and initial geometric position information of the vertices included in the first vertex set in the original mesh, including: performing function fitting based on the original geometric position information and initial geometric position information of the first vertex set and a preset functional relationship; and determining that the best fitting function corresponding to the minimum error reference value is the mapping relationship corresponding to the first vertex set.

如图15A和图15B所示，在编码端，利用一定的算法通过拟合初始网格的初始几何位置信息与原始网格的原始几何位置信息之间的关系，将初始几何位置信息作为拟合函数的输入值，将原始几何位置信息作为目标输出值，假设两者之间的映射关系如下所示：
predict_mesh＝f(reconMesh,i)As shown in FIG. 15A and FIG. 15B , at the encoding end, a certain algorithm is used to fit the relationship between the initial geometric position information of the initial grid and the original geometric position information of the original grid, and the initial geometric position information is used as the input value of the fitting function, and the original geometric position information is used as the target output value. It is assumed that the mapping relationship between the two is as follows:
predict _mesh =f(reconMesh,i)

其中，predict_mesh为通过拟合得到的预测几何位置信息，reconMesh为初始几何位置信息，i为第一顶点集合的标识信息，f(·)为映射关系，*和+为向量的数乘和加法。Wherein, predict _mesh is the predicted geometric position information obtained by fitting, reconMesh is the initial geometric position information, i is the identification information of the first vertex set, f(·) is the mapping relationship, and * and + are vector multiplication and addition.

根据一定误差参考值确定二者之间的关系。示例性的，误差参考值可以包括以下至少之一：均方误差(Mean Square Error，MSE)、误差平方和(Sum of Squared Difference，SSD)、绝对误差和(Sum of Absolute Difference，SAD)、绝对误差和(Sum of Absolute Transformed Difference，SATD)、峰值信噪比(Peak Signal to Noise Ratio，PSNR)等参数来度量编码失真度。The relationship between the two is determined according to a certain error reference value. Exemplarily, the error reference value may include at least one of the following parameters: mean square error (MSE), sum of squared difference (SSD), sum of absolute difference (SAD), sum of absolute transformed difference (SATD), peak signal to noise ratio (PSNR), etc. to measure the coding distortion.

例如，利用MSE算法得到第一顶点集合中几何位置信息的误差参考值，具体如下所示：
For example, the error reference value of the geometric position information in the first vertex set is obtained by using the MSE algorithm, as shown below:

其中，predict_mesh为通过拟合得到的预测几何位置信息，orgMesh为原始几何位置信息，N指示顶点数量。Among them, predict _mesh is the predicted geometric position information obtained by fitting, orgMesh is the original geometric position information, and N indicates the number of vertices.

步骤204：对映射关系进行代价计算，确定第一代价参考值；Step 204: Calculate the cost of the mapping relationship to determine a first cost reference value;

在一些实施例中，对映射关系进行代价计算，确定第一代价参考值，包括：根据第一顶点集合中顶点的初始几何位置信息和映射关系，确定第一顶点集合中顶点的第一预测几何位置信息；根据第一预测几何位置信息和原始几何位置信息，确定映射关系对应的第一代价参考值。In some embodiments, a cost calculation is performed on the mapping relationship to determine a first cost reference value, including: determining first predicted geometric position information of the vertices in the first vertex set based on the initial geometric position information of the vertices in the first vertex set and the mapping relationship; determining a first cost reference value corresponding to the mapping relationship based on the first predicted geometric position information and the original geometric position information.

示例性的，映射关系第一顶点集合中顶点的初始几何位置信息和预测几何位置信息的函数关系，将初始几何位置信息作为映射关系的输入值，得到映射关系的输出值作为第一预测几何位置信息，映射关系的函数表达式可以为：predict_mesh＝f(reconMesh,i)。Exemplarily, the functional relationship between the initial geometric position information and the predicted geometric position information of the vertices in the first vertex set of the mapping relationship is used, the initial geometric position information is used as the input value of the mapping relationship, and the output value of the mapping relationship is obtained as the first predicted geometric position information. The functional expression of the mapping relationship can be: predict _mesh = f(reconMesh, i).

在一些实施例中，该方法还包括：第一代价参考值大于第一代价阈值的情况下，确定编码映射关系。示例性的，第一代价阈值可以是预先设定的为保证网格重建质量的代价阈值，第一代价阈值还可以是根据移位系数确定的第二代价参考值而设定的代价阈值。当第一代价参考值大于第一代价阈值的情况下，不编码第一顶点集合的移位系数，编码映射关系即可保证网格顶点的几何位置信息的重建质量，降低码率。In some embodiments, the method further includes: determining the encoding mapping relationship when the first cost reference value is greater than a first cost threshold. Exemplarily, the first cost threshold may be a pre-set cost threshold to ensure the quality of mesh reconstruction, or the first cost threshold may be a cost threshold set according to a second cost reference value determined by a shift coefficient. When the first cost reference value is greater than the first cost threshold, The shift coefficients of the first vertex set are not encoded, and the encoding mapping relationship can ensure the reconstruction quality of the geometric position information of the mesh vertices and reduce the bit rate.

在一些实施例中，第一代价参考值可以为误差参考值，示例性的，误差参考值包括以下至少之一：MSE、SSD、SAD、SATD、PSNR等参数来度量编码失真度。In some embodiments, the first cost reference value may be an error reference value. Exemplarily, the error reference value includes at least one of the following parameters: MSE, SSD, SAD, SATD, PSNR, etc. to measure coding distortion.

在一些实施例中，第一代价阈值可以还可以为率失真代价值。综合考虑失真D和码率大小，确定是否编码映射关系，解码端利用映射关系进行几何信息重建。In some embodiments, the first cost threshold may also be a rate-distortion cost value. Taking the distortion D and the code rate into consideration, it is determined whether to encode the mapping relationship, and the decoding end uses the mapping relationship to reconstruct the geometric information.

利用MSE算法得到第一顶点集合中几何位置信息的误差参考值，具体如下所示：
The error reference value of the geometric position information in the first vertex set is obtained using the MSE algorithm, as shown below:

其中，predict_mesh为通过拟合得到的预测几何位置信息，orgMesh为原始几何位置信息，lvl为不同的LOD层的标识信息，N指示LOD层中顶点数量。在编码端比较根据拟合函数确定的MSE误差值与Th(第一误差阈值)，来决定Displacement的编码方式。在本申请实施例中，引入一种跳过编码单元的编码方式：如果当前编码单元(例如当前LOD层或者当前编码块)与原始网格顶点的几何位置信息满足一定映射关系时，并且利用一定映射关系拟合得到的MSE误差值小于一定Th时，直接编码映射关系，利用Base Mesh划分之后的几何位置信息与映射关系进行拟合重建得到当前编码单元中顶点的第一预测几何位置信息。Among them, predict _mesh is the predicted geometric position information obtained by fitting, orgMesh is the original geometric position information, lvl is the identification information of different LOD layers, and N indicates the number of vertices in the LOD layer. The MSE error value determined according to the fitting function is compared with Th (the first error threshold) at the encoding end to determine the encoding method of Displacement. In an embodiment of the present application, a coding method that skips the coding unit is introduced: if the geometric position information of the current coding unit (such as the current LOD layer or the current coding block) and the original mesh vertex satisfies a certain mapping relationship, and the MSE error value obtained by fitting using a certain mapping relationship is less than a certain Th, the mapping relationship is directly encoded, and the geometric position information after the Base Mesh division is used to fit and reconstruct the first predicted geometric position information of the vertices in the current coding unit.

步骤205：在根据第一代价参考值确定编码映射关系的情况下，对第一顶点集合对应的映射关系进行编码，将所得到的编码比特写入码流。Step 205: When the coding mapping relationship is determined according to the first cost reference value, the mapping relationship corresponding to the first vertex set is encoded, and the obtained coding bits are written into the bitstream.

在一些实施例中，根据第一顶点集合的映射关系，确定映射关系指示信息；对映射关系指示信息进行编码。示例性的，映射关系指示信息可以用于指示映射关系的参数，还可以直接指示某一预设的映射关系。不同顶点集合对应的映射关系可以相同也可以不相同。示例性的，映射关系指示信息可以包括LOD层级或者片级语法元素。映射关系指示信息还可以包括序列级和/或图像级语法元素。In some embodiments, mapping relationship indication information is determined based on the mapping relationship of the first vertex set; and the mapping relationship indication information is encoded. Exemplarily, the mapping relationship indication information can be used to indicate the parameters of the mapping relationship, and can also directly indicate a preset mapping relationship. The mapping relationships corresponding to different vertex sets can be the same or different. Exemplarily, the mapping relationship indication information can include LOD level or slice level syntax elements. The mapping relationship indication information can also include sequence level and/or image level syntax elements.

在一些实施例中，映射关系指示信息包括第一指示信息，其中，第一指示信息用于指示映射关系的常数；根据第一顶点集合的映射关系，确定映射关系指示信息，包括：根据映射关系的至少一个常数，确定第一指示信息。In some embodiments, the mapping relationship indication information includes first indication information, wherein the first indication information is used to indicate a constant of the mapping relationship; determining the mapping relationship indication information according to the mapping relationship of the first vertex set includes: determining the first indication information according to at least one constant of the mapping relationship.

示例性的，映射关系为基于非线性函数的映射关系时，第一指示信息用于指示非线性函数中的至少一个常数。示例性的，非线性函数为指数函数，第一指示信息用于指示非线性函数的常数a。非线性函数为多项式函数，第一指示信息用于指示多项式函数的系数a₀，a₁，a₂…。非线性函数为对数函数，第一指示信息用于指示非线性函数的底数a。Exemplarily, when the mapping relationship is a mapping relationship based on a nonlinear function, the first indication information is used to indicate at least one constant in the nonlinear function. Exemplarily, the nonlinear function is an exponential function, and the first indication information is used to indicate a constant a of the nonlinear function. The nonlinear function is a polynomial function, and the first indication information is used to indicate coefficients a ₀ , a ₁ , a ₂ … of the polynomial function. The nonlinear function is a logarithmic function, and the first indication information is used to indicate a base a of the nonlinear function.

在一些实施例中，映射关系指示信息还包括第二指示信息，其中，第二指示信息用于指示映射关系的函数类型；该方法还包括：根据映射关系的函数类型，确定第二指示信息。In some embodiments, the mapping relationship indication information further includes second indication information, wherein the second indication information is used to indicate a function type of the mapping relationship; the method further includes: determining the second indication information according to the function type of the mapping relationship.

在一些实施例中，映射关系指示信息用于指示映射关系的索引信息。In some embodiments, the mapping relationship indication information is used to indicate index information of the mapping relationship.

在一些实施例中，该方法还可以包括：在根据第一代价参考值确定编码映射关系的情况下，编码第一顶点集合的第三指示信息；其中，第三指示信息用于指示解码端解码第一顶点集合的映射关系。也就是说，第三指示信息用于指示是否解码第一顶点集合的映射关系，在解码映射关系之前，需要先解码第三指示信息，根据第三指示信息确定解码第一顶点集合的映射关系，才进一步解码映射关系指示信息。示例性的，第三指示信息可以为网格编解码中新增语法元素，也可以复用已有语法元素。第三指示信息可以包括以下至少之一：序列级、帧级、LOD层级、片级语法元素。In some embodiments, the method may further include: in the case of determining the encoding mapping relationship according to the first cost reference value, encoding the first The third indication information of a vertex set; wherein the third indication information is used to instruct the decoding end to decode the mapping relationship of the first vertex set. That is to say, the third indication information is used to indicate whether to decode the mapping relationship of the first vertex set. Before decoding the mapping relationship, the third indication information needs to be decoded first, and the mapping relationship of the first vertex set needs to be determined according to the third indication information before further decoding the mapping relationship indication information. Exemplarily, the third indication information can be a new syntax element in the mesh encoding and decoding, or it can reuse the existing syntax element. The third indication information can include at least one of the following: sequence level, frame level, LOD level, and slice level syntax elements.

在一些实施例中，该方法还包括：根据原始网格中第一顶点集合所包含顶点的原始几何位置信息和初始几何位置信息，确定第一顶点集合中顶点的第一移位系数；对第一移位系数进行代价计算，确定第二代价参考值；根据第一代价参考值和第二代价参考值确定编码映射关系的情况下，对第一顶点集合对应的映射关系进行编码，将所得到的编码比特写入码流；根据第一代价参考值和第二代价参考值确定编码第一移位系数的情况下，对第一顶点集合对应的第一移位系数进行编码，将所得到的编码比特写入码流。也就是说，编码端通过比较这两种编码方案的代价参考值进行编码决策，判断编码第一顶点集合的映射关系还是编码移位系数。示例性的，代价参考值可以为简单的误差参考值，也可以为率失真代价值。In some embodiments, the method further includes: determining the first shift coefficient of the vertices in the first vertex set according to the original geometric position information and the initial geometric position information of the vertices included in the first vertex set in the original mesh; performing cost calculation on the first shift coefficient to determine the second cost reference value; when the encoding mapping relationship is determined according to the first cost reference value and the second cost reference value, encoding the mapping relationship corresponding to the first vertex set, and writing the obtained encoding bits into the bitstream; when the encoding first shift coefficient is determined according to the first cost reference value and the second cost reference value, encoding the first shift coefficient corresponding to the first vertex set, and writing the obtained encoding bits into the bitstream. In other words, the encoding end makes an encoding decision by comparing the cost reference values of the two encoding schemes to determine whether to encode the mapping relationship of the first vertex set or the encoding shift coefficient. Exemplarily, the cost reference value can be a simple error reference value or a rate-distortion cost value.

在一些实施例中，对第一移位系数进行代价计算，确定第二代价参考值，包括：根据第一顶点集合中顶点的初始几何位置信息和第一移位系数，确定第一顶点集合中顶点的第二预测几何位置信息；根据第二预测几何位置信息和原始几何位置信息，确定第二代价参考值。In some embodiments, a cost calculation is performed on the first shift coefficient to determine a second cost reference value, including: determining second predicted geometric position information of the vertices in the first vertex set based on initial geometric position information of the vertices in the first vertex set and the first shift coefficient; determining a second cost reference value based on the second predicted geometric position information and the original geometric position information.

在一些实施例中，该方法还包括：第一代价参考值小于第二代价参考值，确定编码映射关系；第一代价参考值大于或者等于第二代价参考值，确定编码第一移位系数。In some embodiments, the method further includes: if the first cost reference value is less than the second cost reference value, determining a coding mapping relationship; if the first cost reference value is greater than or equal to the second cost reference value, determining a first shift coefficient for coding.

在一些实施例中，该方法还包括：计算第一代价参考值和第二代价参考值的比值；比值小于第二代价阈值时，确定编码映射关系；比值大于或者等于第二代价阈值时，确定编码第一移位系数。In some embodiments, the method further includes: calculating the ratio of the first cost reference value to the second cost reference value; when the ratio is less than the second cost threshold, determining the encoding mapping relationship; when the ratio is greater than or equal to the second cost threshold, determining the encoding first shift coefficient.

在一些实施例中，根据原始网格中第一顶点集合所包含顶点的原始几何位置信息和初始几何位置信息，确定第一顶点集合中顶点的第一移位系数，包括：根据原始网格中第一顶点集合所包含顶点的原始几何位置信息和初始几何位置信息，确定第一顶点集合中顶点的第二移位系数；对第一顶点集合中顶点的移位系数进行提升小波变换，得到变换系数；对变换系数进行量化，得到量化后的第一移位系数。也就是说，第二移位系数可以为变换前的移位系数，第一移位系数可以为量化后的移位系数。In some embodiments, determining first shift coefficients of vertices in the first vertex set according to original geometric position information and initial geometric position information of vertices included in the first vertex set in the original mesh includes: determining second shift coefficients of vertices in the first vertex set according to original geometric position information and initial geometric position information of vertices included in the first vertex set in the original mesh; performing lifting wavelet transform on the shift coefficients of vertices in the first vertex set to obtain transform coefficients; and quantizing the transform coefficients to obtain quantized first shift coefficients. That is, the second shift coefficient may be a shift coefficient before the transform, and the first shift coefficient may be a quantized shift coefficient.

示例性的，本申请实施例提供的移位系数编码方法的实现步骤可以包括：Exemplarily, the implementation steps of the shift coefficient encoding method provided in the embodiment of the present application may include:

首先，会根据Base mesh来按照一定的算法进行迭代划分来得到对应的mesh位置信息，具体的划分算法和背景介绍一致，利用每个边界上的顶点进行线性插值，得到对应的几何位置信息。假设，整个划分迭代了N次，则根据不同迭代划分得到的Displacement进行LOD划分，得到对应的mesh几何位置信息，利用初始得到的几何位置信息与原始网格进行误差计算，得到每个点的Displacement。First, the Base mesh will be iteratively divided according to a certain algorithm to obtain the corresponding mesh position information. The specific division algorithm is consistent with the background introduction. The vertices on each boundary are used for linear interpolation to obtain the corresponding geometric position information. Assuming that the entire division is iterated N times, the Displacement obtained by different iterative divisions is divided into LODs to obtain the corresponding mesh geometric position information. The initial geometric position information is used to calculate the error with the original mesh to obtain the Displacement of each point.

其次，基于LOD空间结构对移位系数进行提升小波变换(Lifting Transform)，其中包括预测和更新两个步骤，其中预测步骤如下：
Secondly, based on the LOD spatial structure, the shift coefficients are subjected to a lifting wavelet transform, which includes two steps: prediction and update. The prediction step is as follows:

更新步骤如下：
The update steps are as follows:

最终对变换之后的系数进行量化，并且对量化之后的系数进行系数重组，目前的V-DMC基于Block进行系数重组，每个Block的大小为16×16，对每个Block内部的系数按照莫顿码进行排列得到对应的二维图像。在完成一系列操作之后，最后利用Video-Codec来对二维图像进行编码。Finally, the transformed coefficients are quantized and reorganized. The current V-DMC reorganizes coefficients based on blocks. The size of each block is 16×16. The coefficients in each block are arranged according to the Morton code to obtain the corresponding two-dimensional image. After completing a series of operations, the two-dimensional image is finally encoded using Video-Codec.

示例性的，还可以利用熵编码器直接编码移位系数，得到移位系数码流。Exemplarily, the shift coefficients may be directly encoded using an entropy encoder to obtain a shift coefficient code stream.

在上述实施例的基础上，对本申请实施例提供的编码方法进行进一步地举例说明，具体实现步骤可以包括：Based on the above embodiment, the encoding method provided in the embodiment of the present application is further illustrated by example, and the specific implementation steps may include:

在编码端，利用一定的算法通过拟合初始mesh重建点几何位置信息与原始mesh几何位置信息之间的关系，假设两者之间的映射关系如下所示：
predict_mesh＝f(reconMesh,lvl)At the encoding end, a certain algorithm is used to fit the relationship between the geometric position information of the initial mesh reconstruction point and the geometric position information of the original mesh. It is assumed that the mapping relationship between the two is as follows:
predict _mesh =f(reconMesh,lvl)

其中，lvl代表不同的lod层，reconMesh表示初始mesh信息，predict_mesh代表利用一定的函数关系得到对应的第一预测几何位置信息。Among them, lvl represents different lod layers, reconMesh represents the initial mesh information, and predict _mesh represents the first predicted geometric position information obtained by using a certain functional relationship.

在本方案中，考虑带编码复杂度与编码效率的平衡，通过利用简单的线性拟合来得到原始mesh几何位置信息与初始mesh位置信息之间的关系，具体的如下所示：
predict_mesh＝k*(reconMesh,lvl)+bIn this scheme, the balance between coding complexity and coding efficiency is considered, and the relationship between the original mesh geometric position information and the initial mesh position information is obtained by using simple linear fitting, as shown below:
predict _mesh =k*(reconMesh,lvl)+b

其中，*和+为向量的数乘和加法。Among them, * and + are vector multiplication and addition.

最终通过一定的失真，利用MSE算法得到不同LOD层mesh几何位置信息的误差，具体如下所示：
Finally, through a certain distortion, the MSE algorithm is used to obtain the error of the geometric position information of the mesh at different LOD layers, as shown below:

在编码端不同比较不同LOD层MSE误差值与Th(误差阈值)，来决定不同LOD层Displacement的编码方式，在本方案中，引入一种跳过编码层的编码方式：如果当前LOD层与与原始网格顶点的几何位置信息满足一定关系时，并且利用一定关系拟合得到的误差小于一定Th时，则认为当前LOD层的Displacement可以跳过编码，即：对当前层的Displacement不进行编码，直接利用Base Mesh划分之后的几何位置信息与参数进行拟合重建得到当前LOD层顶点的几何位置信息。At the encoding end, the MSE error values of different LOD layers are compared with Th (error threshold) to determine the encoding method of different LOD layer Displacement. In this scheme, a coding method of skipping the coding layer is introduced: if the geometric position information of the current LOD layer and the original mesh vertices satisfy a certain relationship, and the error obtained by fitting a certain relationship is less than a certain Th, then it is considered that the Displacement of the current LOD layer can skip encoding, that is: the Displacement of the current layer is not encoded, and the geometric position information and parameters after the Base Mesh division are directly used to fit and reconstruct the geometric position information of the vertices of the current LOD layer.

本申请实施例在重建当前网格的几何位置信息时，考虑到原始网格几何位置信息与基础网格划分之后几何位置信息之间的相关性，为了提高V-DMC对mesh几何位置信息的编码效率，可以通过利用一些算法，来确定原始网格的几何位置信息与基础网格划分之后几何位置信息之间的相关性，利用两者之间的相关性，来自适应的决定Displacement的编码方式，例如，可以对一些LOD层的Displacement进行跳过编码或者利用不同点之间Displacement之间的相关性，来减少一部分点Displacement分量的编码，这样可以降低Displacement编码的码率，同时可以保证重建网格的质量，进一步提升网格的几何编码效率。When reconstructing the geometric position information of the current grid, the embodiment of the present application takes into account the correlation between the geometric position information of the original grid and the geometric position information after the basic grid is divided. In order to improve the V-DMC encoding efficiency of the mesh geometric position information, some algorithms can be used to determine the correlation between the geometric position information of the original grid and the geometric position information after the basic grid is divided. The correlation between the two is used to adaptively determine the encoding method of the Displacement. For example, the Displacement encoding of some LOD layers can be skipped or the correlation between the Displacements of different points can be used to reduce the encoding of the Displacement components of some points. This can reduce the bit rate of the Displacement encoding, while ensuring the quality of the reconstructed grid, further improving the geometric encoding efficiency of the grid.

进一步地，对本申请实施例提供的移位系数编码方法的编码性能进行测试。Furthermore, the coding performance of the shift coefficient coding method provided in the embodiment of the present application is tested.

1)MPEG DMC的通用测试条件测试条件共2种：1) Common test conditions for MPEG DMC There are 2 types of test conditions:

条件1：全帧内(all intra)几何有损、属性有损；Condition 1: All intra geometry lossy and attribute lossy;

条件2：随机介入(random access)几何有损、属性有损；Condition 2: Random access is lossy in geometry and lossy in attributes;

2)通用测试序列包括Cat1-A，Cat1-B和Cat1-C共三类，均包含几何和颜色属性信息。2) The general test sequences include three categories: Cat1-A, Cat1-B and Cat1-C, all of which contain geometric and color attribute information.

以帧内和帧间测试环境为例，其中BD-Rate为衡量有损压缩效率的性能指标，当BD-Rate小于0时，代表编码效率相对于现有的编码方案提升。Taking the intra-frame and inter-frame test environment as an example, BD-Rate is a performance indicator for measuring lossy compression efficiency. When BD-Rate is less than 0, it means that the coding efficiency is improved compared to the existing coding scheme.

表1为测试条件1下的实验结果
Table 1 shows the experimental results under test condition 1

表2为测试条件2下的实验结果
Table 2 shows the experimental results under test condition 2

本方案通过利用Base Mesh划分之后不同网格编码单元顶点的几何位置信息与原始网格的原始几何位置信息之间的关系，来自适应决定不同网格编码单元的Displacement的编码方式。具体的，在编码端首先利用一定的方式得到LOD空间结构，其次基于LOD空间结构，对初始几何位置信息与原始几何位置信息之间的关系进行拟合，并且对拟合之后的位置信息与原始位置信息之间的误差进行计算，当误差小于一定阈值时，可以决定当前层的Displacement采用跳过编码。这里需要注意的是，本方案不限制拟合的方式或者采用线性拟合、曲线拟合或者卷积参数拟合，本方案更多的是建立Base Mesh点的初始几何位置信息与原始几何位置信息之间的关系，从而自适应地跳过编码一些LOD层Displacement的编码，从而可以降低Displacement的码率大小，并且可以保证网格顶点几何位置信息的重建质量，从而进一步提升网格的几何信息编码效率。This solution adaptively determines the encoding method of the Displacement of different mesh coding units by utilizing the relationship between the geometric position information of the vertices of different mesh coding units after the Base Mesh is divided and the original geometric position information of the original mesh. Specifically, at the encoding end, a certain method is first used to obtain the LOD spatial structure. Secondly, based on the LOD spatial structure, the relationship between the initial geometric position information and the original geometric position information is fitted, and the error between the fitted position information and the original position information is calculated. When the error is less than a certain threshold, it can be decided to skip encoding the Displacement of the current layer. It should be noted here that this solution does not limit the fitting method or use linear fitting, curve fitting or convolution parameter fitting. This solution is more about establishing the relationship between the initial geometric position information of the Base Mesh point and the original geometric position information, so as to adaptively skip encoding some LOD layer Displacement encoding, thereby reducing the bit rate of the Displacement and ensuring the reconstruction quality of the mesh vertex geometric position information, thereby further improving the mesh's geometric information encoding efficiency.

基于上述实施例，在本申请的再一实施例中，基于前述实施例相同的发明构思，图19为编码器的组成结构示意图一，如图19所示，编码器110可以包括：第一确定单元111、拟合单元112、第一预测单元113和编码单元114，其中，Based on the above embodiment, in another embodiment of the present application, based on the same inventive concept as the above embodiment, FIG. 19 is a diagram of an encoder. Schematic diagram of composition structure 1, as shown in FIG19, the encoder 110 may include: a first determination unit 111, a fitting unit 112, a first prediction unit 113 and an encoding unit 114, wherein:

第一确定单元111，配置为根据原始网格，确定基础网格；对所述基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；The first determining unit 111 is configured to determine a basic mesh according to the original mesh; subdivide the basic mesh to determine initial geometric position information of vertices included in the first vertex set in the initial mesh;

拟合单元112，配置为根据所述原始网格中所述第一顶点集合所包含顶点的原始几何位置信息和所述初始几何位置信息，确定所述第一顶点集合对应的映射关系；A fitting unit 112, configured to determine a mapping relationship corresponding to the first vertex set according to original geometric position information of vertices included in the first vertex set in the original mesh and the initial geometric position information;

第一预测单元113，配置对所述映射关系进行代价计算，确定第一代价参考值；The first prediction unit 113 is configured to perform cost calculation on the mapping relationship to determine a first cost reference value;

编码单元114，配置为在根据所述第一代价参考值确定编码所述映射关系的情况下，对第一顶点集合对应的所述映射关系进行编码，将所得到的编码比特写入码流。The encoding unit 114 is configured to, when determining to encode the mapping relationship according to the first cost reference value, encode the mapping relationship corresponding to the first vertex set, and write the obtained encoding bits into a bitstream.

可以理解地，在本实施例中，“单元”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是模块，还可以是非模块化的。而且在本实施例中的各组成部分可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。It can be understood that in this embodiment, a "unit" can be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module.

所述集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

因此，本申请实施例提供了一种计算机可读存储介质，应用于编码器110，该计算机可读存储介质存储有计算机程序，所述计算机程序被第一处理器执行时实现前述实施例中任一项所述的方法。Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 110. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, the method described in any one of the aforementioned embodiments is implemented.

基于上述编码器110的组成以及计算机可读存储介质，图20为编码器的组成结构示意图二，如图20所示，编码器110可以包括：第一存储器115和第一处理器116，第一通信接口117和第一总线系统118。第一存储器115、第一处理器116、第一通信接口117通过第一总线系统118耦合在一起。可理解，第一总线系统118用于实现这些组件之间的连接通信。第一总线系统118除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图20中将各种总线都标为第一总线系统118。其中，Based on the composition of the above-mentioned encoder 110 and the computer-readable storage medium, Figure 20 is a second schematic diagram of the composition structure of the encoder. As shown in Figure 20, the encoder 110 may include: a first memory 115 and a first processor 116, a first communication interface 117 and a first bus system 118. The first memory 115, the first processor 116, and the first communication interface 117 are coupled together through the first bus system 118. It can be understood that the first bus system 118 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 118 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 118 in Figure 20. Among them,

第一通信接口117，用于在与其他外部网元之间进行收发信息过程中，信号的接收和发送；The first communication interface 117 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

所述第一存储器115，用于存储能够在所述第一处理器上运行的计算机程序；The first memory 115 is used to store a computer program that can be run on the first processor;

所述第一处理器116，用于在运行所述计算机程序时，根据原始网格，确定基础网格；对所述基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；根据所述原始网格中所述第一顶点集合所包含顶点的原始几何位置信息和所述初始几何位置信息，确定所述第一顶点集合对应的映射关系；对所述映射关系进行代价计算，确定第一代价参考值；在根据所述第一代价参考值确定编码所述映射关系的情况下，对第一顶点集合对应的所述映射关系进行编码，将所得到的编码比特写入码流。The first processor 116 is used to determine a basic mesh based on the original mesh when running the computer program; subdivide the basic mesh to determine initial geometric position information of vertices included in a first vertex set in the initial mesh; determine a mapping relationship corresponding to the first vertex set based on the original geometric position information and the initial geometric position information of the vertices included in the first vertex set in the original mesh; perform cost calculation on the mapping relationship to determine a first cost reference value; and when determining to encode the mapping relationship based on the first cost reference value, encode the mapping relationship corresponding to the first vertex set and write the obtained encoding bits into a bitstream.

可以理解，本申请实施例中的第一存储器115可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请描述的系统和方法的第一存储器115旨在包括但不限于这些和任意其它适合类型的存储器。It can be understood that the first memory 115 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). The first memory 115 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

而第一处理器116可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过第一处理器116中的硬件的集成逻辑电路或者软件形式的指令完成。上述的第一处理器114可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于第一存储器115，第一处理器116读取第一存储器115中的信息，结合其硬件完成上述方法的步骤。The first processor 116 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by a hardware integrated logic circuit or software instructions in the first processor 116. The above-mentioned first processor 114 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. It can implement or execute the embodiments of the present application. The disclosed methods, steps and logic block diagrams. The general processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 115, and the first processor 116 reads the information in the first memory 115, and completes the steps of the above method in conjunction with its hardware.

可以理解的是，本申请描述的这些实施例可以用硬件、软件、固件、中间件、微码或其组合来实现。对于硬件实现，处理单元可以实现在一个或多个专用集成电路(Application Specific Integrated Circuits，ASIC)、数字信号处理器(Digital Signal Processing，DSP)、数字信号处理设备(DSP Device，DSPD)、可编程逻辑设备(Programmable Logic Device，PLD)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)、通用处理器、控制器、微控制器、微处理器、用于执行本申请所述功能的其它电子单元或其组合中。对于软件实现，可通过执行本申请所述功能的模块(例如过程、函数等)来实现本申请所述的技术。软件代码可存储在存储器中并通过处理器执行。存储器可以在处理器中或在处理器外部实现。It is understood that the embodiments described in this application can be implemented in hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing devices (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application or a combination thereof. For software implementation, the technology described in this application can be implemented by a module (such as a process, function, etc.) that performs the functions described in this application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

可选地，作为另一个实施例，第一处理器116还配置为在运行所述计算机程序时，执行前述实施例中任一项所述的方法。Optionally, as another embodiment, the first processor 116 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

图21为解码器的组成结构示意图一，如图21所示，解码器120可以包括：第二确定单元121、解码单元122和第二预测单元123；其中，FIG. 21 is a schematic diagram of a structure of a decoder. As shown in FIG. 21 , the decoder 120 may include: a second determination unit 121, a decoding unit 122, and a second prediction unit 123; wherein,

第二确定单元121，配置为确定基础网格；对所述基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；The second determining unit 121 is configured to determine a basic mesh; subdivide the basic mesh to determine initial geometric position information of vertices included in the first vertex set in the initial mesh;

解码单元122，配置为解码码流，确定所述第一顶点集合对应的映射关系；其中，所述映射关系用于指示所述第一顶点集合中顶点的初始几何位置信息和原始几何位置信息的相关性；The decoding unit 122 is configured to decode the code stream to determine a mapping relationship corresponding to the first vertex set; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

第二预测单元123，配置为根据所述第一顶点集合中顶点的所述初始几何位置信息和所述映射关系，确定所述第一顶点集合中所述顶点的第一预测几何位置信息。The second prediction unit 123 is configured to determine first predicted geometric position information of the vertices in the first vertex set according to the initial geometric position information of the vertices in the first vertex set and the mapping relationship.

因此，本申请实施例提供了一种计算机可读存储介质，应用于解码器120，该计算机可读存储介质存储有计算机程序，所述计算机程序被第一处理器执行时实现前述实施例中任一项所述的方法。Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the decoder 120. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, the method described in any one of the above embodiments is implemented.

基于上述解码器120的组成以及计算机可读存储介质，图22为解码器的组成结构示意图二，如图22所示，解码器120可以包括：第二存储器127和第二处理器124，第二通信接口125和第二总线系统126。第二存储器127和第二处理器124，第二通信接口125通过第二总线系统126耦合在一起。可理解，第二总线系统126用于实现这些组件之间的连接通信。第二总线系统126除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图22中将各种总线都标为第二总线系统126。其中，Based on the composition of the above-mentioned decoder 120 and the computer-readable storage medium, Figure 22 is a second schematic diagram of the composition structure of the decoder. As shown in Figure 22, the decoder 120 may include: a second memory 127 and a second processor 124, a second communication interface 125 and a second bus system 126. The second memory 127 and the second processor 124, and the second communication interface 125 are coupled together through the second bus system 126. It can be understood that the second bus system 126 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 126 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 126 in Figure 22. Among them,

第二通信接口125，用于在与其他外部网元之间进行收发信息过程中，信号的接收和发送；The second communication interface 125 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

所述第二存储器127，用于存储能够在所述第二处理器上运行的计算机程序；The second memory 127 is used to store a computer program that can be run on the second processor;

所述第二处理器124，用于在运行所述计算机程序时，确定基础网格；对所述基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；解码码流，确定所述第一顶点集合对应的映射关系；其中，所述映射关系用于指示所述第一顶点集合中顶点的初始几何位置信息和原始几何位置信息的相关性；根据所述第一顶点集合中顶点的所述初始几何位置信息和所述映射关系，确定所述第一顶点集合中所述顶点的第一预测几何位置信息。The second processor 124 is used to determine the basic mesh when running the computer program; subdivide the basic mesh to determine the initial geometric position information of the vertices included in the first vertex set in the initial mesh; decode the code stream to determine the mapping relationship corresponding to the first vertex set; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set; determine the first predicted geometric position information of the vertices in the first vertex set based on the initial geometric position information of the vertices in the first vertex set and the mapping relationship.

可以理解，本申请实施例中的第二存储器127可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请描述的系统和方法的第二存储器127旨在包括但不限于这些和任意其它适合类型的存储器。It can be understood that the second memory 127 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory. Random Access Memory (RAM), which is used as an external high-speed cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM). The second memory 127 of the system and method described in the present application is intended to include but is not limited to these and any other suitable types of memory.

而第二处理器124可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过第二处理器124中的硬件的集成逻辑电路或者软件形式的指令完成。上述的第二处理器124可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于第二存储器127，第二处理器124读取第二存储器127中的信息，结合其硬件完成上述方法的步骤。The second processor 124 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the second processor 124. The above-mentioned second processor 124 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the second memory 127, and the second processor 124 reads the information in the second memory 127 and completes the steps of the above method in combination with its hardware.

需要说明的是，在本申请的实施例中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。It should be noted that, in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

本申请所提供的几个方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

本申请所提供的几个产品实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的产品实施例。The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

本申请所提供的几个方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Industrial Applicability

本申请实施例提供了一种编解码方法、编码器、解码器以及存储介质，在编解码端，对基础网格进行细分，确定初始网格中第一顶点集合所包含顶点的初始几何位置信息；根据所述第一顶点集合中顶点的所述初始几何位置信息和第一顶点集合对应的映射关系，确定所述第一顶点集合中所述顶点的第一预测几何位置信息。这样，在预测待编解码单元的第一顶点集合的几何位置信息时，自适应的跳过一些顶点移位系数的编解码，而是利用表征初始几何位置信息和原始几何位置信息相关性的映射关系进行拟合，确定初始几何位置信息对应的第一预测几何位置信息，降低移位系数的码率，在保证网格顶点的几何位置信息的重建质量的基础上，提高几何位置信息的编解码效率。 The embodiment of the present application provides a coding and decoding method, an encoder, a decoder and a storage medium. At the coding and decoding end, the basic mesh is subdivided to determine the initial geometric position information of the vertices contained in the first vertex set in the initial mesh; according to the mapping relationship between the initial geometric position information of the vertices in the first vertex set and the first vertex set, the first predicted geometric position information of the vertices in the first vertex set is determined. In this way, when predicting the geometric position information of the first vertex set of the unit to be coded and decoded, the coding and decoding of some vertex shift coefficients are adaptively skipped, and the mapping relationship that characterizes the correlation between the initial geometric position information and the original geometric position information is used for fitting to determine the first predicted geometric position information corresponding to the initial geometric position information, reduce the code rate of the shift coefficient, and improve the coding and decoding efficiency of the geometric position information on the basis of ensuring the reconstruction quality of the geometric position information of the mesh vertices.

Claims

A decoding method, applied to a decoder, wherein the method comprises:

Determine the base grid;

Subdividing the basic mesh to determine initial geometric position information of vertices included in a first vertex set in the initial mesh;

Decoding the code stream to determine a mapping relationship corresponding to the first vertex set; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

First predicted geometric position information of the vertices in the first vertex set is determined according to the initial geometric position information of the vertices in the first vertex set and the mapping relationship.

The method according to claim 1, wherein the mapping relationship is a functional relationship between initial geometric position information and predicted geometric position information of vertices in the first vertex set;

The determining, according to the initial geometric position information of the vertices in the first vertex set and the mapping relationship, first predicted geometric position information of the vertices in the first vertex set includes:

The initial geometric position information is used as an input value of the mapping relationship, and an output value of the mapping relationship is obtained as the first predicted geometric position information.

The method according to claim 2, wherein the mapping relationship includes at least one of the following: an identity mapping relationship, a mapping relationship based on a linear function, a mapping relationship based on a nonlinear function, and a mapping relationship based on a neural network.

The method according to any one of claims 1 to 3, wherein the decoding of the code stream to determine the mapping relationship corresponding to the first vertex set comprises:

Decoding the code stream to determine mapping relationship indication information of the first vertex set;

The mapping relationship corresponding to the first vertex set is determined according to the mapping relationship indication information.

The method according to claim 4, wherein the mapping relationship indication information comprises first indication information, wherein the first indication information is used to indicate a constant of the mapping relationship;

The determining, according to the mapping relationship indication information, the mapping relationship corresponding to the first vertex set includes:

Determine at least one constant of the mapping relationship according to the first indication information;

The mapping relationship is determined according to at least one constant of the mapping relationship.

The method according to claim 5, wherein the mapping relationship indication information further comprises second indication information, wherein the second indication information is used to indicate the type of the mapping relationship;

The determining, according to the mapping relationship indication information, the mapping relationship corresponding to the first vertex set further includes:

Determine the type of the mapping relationship according to the second indication information;

The mapping relationship is determined according to the type of the mapping relationship and the at least one constant.

The method according to claim 4, wherein the mapping relationship indication information is used to indicate index information of the mapping relationship;

Determining index information of the mapping relationship according to the mapping relationship indication information;

The mapping relationship is determined according to the index information of the mapping relationship.

The method according to any one of claims 1 to 7, wherein the decoding of the code stream to determine the mapping relationship corresponding to the first vertex set comprises:

Decoding the code stream to determine third indication information of the first vertex set;

When the mapping relationship for decoding the first vertex set is determined according to the third indication information, the code stream is decoded to determine the mapping relationship corresponding to the first vertex set.

The method according to any one of claims 1 to 8, wherein the method further comprises:

Decoding fourth indication information of the first vertex set;

In a case where the shift coefficients of the vertices in the first vertex set are determined according to the fourth indication information, decoding the bitstream to determine the first shift coefficients of the vertices in the first vertex set;

Second predicted geometric position information of the vertices in the first vertex set is determined according to the initial geometric position information of the vertices in the first vertex set and the first shift coefficient.

The method according to claim 9, wherein determining the second predicted geometric position information of the vertices in the first vertex set according to the initial geometric position information of the vertices in the first vertex set and the first shift coefficient comprises:

Performing inverse quantization and lifting wavelet transform on the first shift coefficient to obtain a second shift coefficient;

The second predicted geometric position information is determined according to the initial geometric position information of the vertices in the first vertex set and the second shift coefficient.

The method according to claim 1, wherein subdividing the base grid comprises:

The basic mesh is iteratively subdivided according to a preset mesh subdivision algorithm to determine initial geometric position information of vertices included in different detail levels LOD layers in the initial mesh.

The method according to claim 11, wherein the grid subdivision algorithm is a linear interpolation algorithm.

The method according to claim 11, wherein the first vertex set includes vertices included in the current LOD layer to be decoded.

The method according to claim 1, wherein the first vertex set includes vertices included in the current image block to be decoded.

A coding method, applied to an encoder, wherein the method comprises:

According to the original grid, determine the basic grid;

Determine a mapping relationship corresponding to the first vertex set according to the original geometric position information of the vertices included in the first vertex set in the original mesh and the initial geometric position information; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

Performing cost calculation on the mapping relationship to determine a first cost reference value;

In case of determining to encode the mapping relationship according to the first cost reference value, the mapping relationship corresponding to the first vertex set is encoded, and the obtained encoding bits are written into a bitstream.

The method according to claim 15, wherein the mapping relationship is a functional relationship between initial geometric position information and predicted geometric position information of vertices in the first vertex set.

The method according to claim 16, wherein the mapping relationship includes at least one of the following: an identity mapping relationship, a mapping relationship based on a linear function, a mapping relationship based on a nonlinear function, and a mapping relationship based on a neural network.

The method according to claim 16 or 17, wherein the determining the mapping relationship corresponding to the first vertex set according to the original geometric position information of the vertices included in the first vertex set in the original mesh and the initial geometric position information comprises:

Performing function fitting according to the original geometric position information and the initial geometric position information of the first vertex set and a preset functional relationship;

Determine that the best fitting function corresponding to the minimum error reference value is a mapping relationship corresponding to the first vertex set.

The method according to any one of claims 15 to 17, wherein encoding the mapping relationship corresponding to the first vertex set comprises:

Determine mapping relationship indication information according to the mapping relationship of the first vertex set;

The mapping relationship indication information is encoded.

The method according to claim 19, wherein the mapping relationship indication information comprises first indication information, wherein the first indication information is used to indicate a constant of the mapping relationship;

The determining, according to the mapping relationship of the first vertex set, mapping relationship indication information includes:

The first indication information is determined according to at least one constant of the mapping relationship.

The method according to claim 20, wherein the mapping relationship indication information further comprises second indication information, wherein the second indication information is used to indicate a function type of the mapping relationship;

The determining, according to the mapping relationship of the first vertex set, mapping relationship indication information further includes:

The second indication information is determined according to the function type of the mapping relationship.

The method according to claim 18, wherein the mapping relationship indication information is used to indicate index information of the mapping relationship.

The method according to any one of claims 15 to 22, wherein the method further comprises:

In case of determining to encode the mapping relationship according to the first cost reference value, third indication information of the first vertex set is encoded; wherein the third indication information is used to instruct a decoding end to decode the mapping relationship of the first vertex set.

According to the method of claim 15, the step of performing cost calculation on the mapping relationship to determine the first cost reference value comprises:

Determine first predicted geometric position information of the vertices in the first vertex set according to the initial geometric position information of the vertices in the first vertex set and the mapping relationship;

Determining the first cost reference value according to the first predicted geometric position information and the original geometric position information;

In a case where the first cost reference value is greater than a first cost threshold, it is determined to encode the mapping relationship.

The method according to any one of claims 15 to 23, wherein the method further comprises:

Determine first shift coefficients of vertices in the first vertex set according to original geometric position information of vertices included in the first vertex set in the original mesh and the initial geometric position information;

Performing cost calculation on the first shift coefficient to determine a second cost reference value;

In the case where the mapping relationship is determined according to the first cost reference value and the second cost reference value, the first vertex set The corresponding mapping relationship is encoded, and the obtained encoded bits are written into the bit stream;

When the first shift coefficient to be encoded is determined according to the first cost reference value and the second cost reference value, the first shift coefficient corresponding to the first vertex set is encoded, and the obtained encoding bits are written into a bitstream.

The method according to claim 25, wherein the method further comprises:

The first cost reference value is less than the second cost reference value, determining to encode the mapping relationship;

The first cost reference value is greater than or equal to the second cost reference value, and it is determined to encode the first shift coefficient.

The method according to claim 25, wherein the method further comprises:

calculating a ratio of the first cost reference value to the second cost reference value;

When the ratio is less than a second cost threshold, determining to encode the mapping relationship;

When the ratio is greater than or equal to a second cost threshold, it is determined to encode the first shift coefficient.

The method according to any one of claims 25 to 27, wherein the method further comprises:

When determining the encoding of the first shift coefficient according to the first cost reference value and the second cost reference value, encode fourth indication information of the first vertex set; wherein the fourth indication information is used to instruct a decoding end to decode the shift coefficients of the vertices in the first vertex set.

The method according to claim 25, wherein the determining the first shift coefficients of the vertices in the first vertex set according to the original geometric position information of the vertices included in the first vertex set in the original mesh and the initial geometric position information comprises:

Determine second shift coefficients of vertices in the first vertex set according to original geometric position information of vertices included in the first vertex set in the original mesh and the initial geometric position information;

Performing a lifting wavelet transform on the second shift coefficient to obtain a transform coefficient;

The transform coefficient is quantized to obtain the quantized first shift coefficient.

The method according to claim 15, wherein subdividing the base grid comprises:

The method according to claim 30, wherein the grid subdivision algorithm is a linear interpolation algorithm.

The method according to claim 30, wherein the first vertex set includes vertices included in the current LOD layer to be encoded.

The method according to claim 15, wherein the first vertex set includes vertices contained in the current image block to be encoded.

The method according to any one of claims 15 to 33, wherein the cost reference value is an error reference value;

The error reference value includes at least one of the following: mean square error, sum of squared errors, sum of absolute errors, sum of absolute errors, and peak signal-to-noise ratio.

The method according to any one of claims 15 to 33, wherein the cost reference value is a rate-distortion cost value.

A code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: mapping relationship indication information for indicating a mapping relationship corresponding to a first vertex set, third indication information for indicating decoding of the mapping relationship, and fourth indication information for indicating decoding shift coefficients of vertices in the first vertex set.

An encoder, comprising:

A first determining unit is configured to determine a basic mesh according to the original mesh; subdivide the basic mesh to determine initial geometric position information of vertices included in a first vertex set in the initial mesh;

A fitting unit, configured to determine a mapping relationship corresponding to the first vertex set according to the original geometric position information of the vertices included in the first vertex set in the original mesh and the initial geometric position information; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

A first prediction unit is configured to perform cost calculation on the mapping relationship to determine a first cost reference value;

The encoding unit is configured to, when determining to encode the mapping relationship according to the first cost reference value, encode the mapping relationship corresponding to the first vertex set, and write the obtained encoding bits into a bitstream.

An encoder comprises: a first memory and a first processor; wherein:

The first memory is used to store a computer program that can be run on the first processor;

The first processor is configured to execute the method according to any one of claims 15 to 35 when running the computer program.

A decoder, comprising:

A second determining unit is configured to determine a basic mesh; subdivide the basic mesh to determine initial geometric position information of vertices included in a first vertex set in an initial mesh;

A decoding unit configured to decode the code stream to determine a mapping relationship corresponding to the first vertex set; wherein the mapping relationship is used to indicate the correlation between the initial geometric position information and the original geometric position information of the vertices in the first vertex set;

The second prediction unit is configured to determine first predicted geometric position information of the vertices in the first vertex set according to the initial geometric position information of the vertices in the first vertex set and the mapping relationship.

A decoder, comprising: a second memory and a second processor; wherein:

The second memory is used to store a computer program that can be run on the second processor;

The second processor is configured to execute the method according to any one of claims 1 to 14 when running the computer program.

A computer-readable storage medium storing a computer program, wherein the computer program, when executed, implements the decoding method according to any one of claims 1 to 14, or implements the encoding method according to any one of claims 15 to 35.