WO2025220707A1 - Encoding method, decoding method, encoding device, and decoding device - Google Patents

Abstract

This encoding method is executed by an encoding device. The encoding method includes acquiring a plurality of vertices included in a three-dimensional mesh (S1561), and prediction-encoding one vertex of the plurality of vertices (S1562). In the prediction-encoding, a first prediction value of one vertex is calculated with decimal accuracy, a second prediction value is calculated by reducing the number of bits of the decimal part of the first prediction value, and one vertex is prediction-encoded using the second prediction value.

Description

Encoding method, decoding method, encoding device, and decoding device

This disclosure relates to encoding methods, etc.

Patent document 1 proposes a method and apparatus for encoding and decoding three-dimensional mesh data.

Japanese Patent Application Laid-Open No. 2006-187015

Further improvements are desired in the encoding or decoding process for three-dimensional meshes. The present disclosure aims to improve the encoding or decoding process for three-dimensional meshes.

An encoding method according to one aspect of the present disclosure is an encoding method executed by an encoding device, which obtains multiple vertices included in a three-dimensional mesh and predictively encodes one of the multiple vertices. In the predictive encoding, a first predicted value for the one vertex is calculated with decimal precision, a second predicted value is calculated by reducing the number of bits in the decimal part of the first predicted value, and the one vertex is predictively encoded using the second predicted value.

A decoding method according to one aspect of the present disclosure is a decoding method executed by a decoding device, which obtains multiple vertices included in a three-dimensional mesh and predictively decodes one of the multiple vertices. In the predictive decoding, a first predicted value for the one vertex is calculated with decimal precision, a second predicted value is calculated by reducing the number of bits in the decimal part of the first predicted value, and the one vertex is predictively decoded using the second predicted value.

Note that these comprehensive or specific aspects may be realized as a system, device, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or as any combination of a system, device, integrated circuit, computer program, and recording medium.

This disclosure may contribute to improving coding processes related to three-dimensional meshes.

FIG. 1 is a conceptual diagram illustrating a three-dimensional mesh according to an embodiment. FIG. 2 is a conceptual diagram showing basic elements of a three-dimensional mesh according to an embodiment. FIG. 1 is a conceptual diagram illustrating mapping according to an embodiment. 1 is a block diagram illustrating an example of the configuration of an encoding/decoding system according to an embodiment. 1 is a block diagram illustrating an example of the configuration of an encoding device according to an embodiment. FIG. 10 is a block diagram showing another example configuration of an encoding device according to an embodiment. FIG. 2 is a block diagram illustrating an example of a configuration of a decoding device according to an embodiment. FIG. 10 is a block diagram illustrating another example configuration of a decoding device according to an embodiment. FIG. 2 is a conceptual diagram illustrating an example of the configuration of a bitstream according to an embodiment. FIG. 10 is a conceptual diagram showing another example of the configuration of a bitstream according to an embodiment. FIG. 10 is a conceptual diagram showing yet another example configuration of a bitstream according to an embodiment. 1 is a block diagram showing a specific example of an encoding/decoding system according to an embodiment; FIG. 2 is a conceptual diagram illustrating an example of the configuration of point cloud data according to the embodiment. FIG. 2 is a conceptual diagram illustrating an example of a data file of point cloud data according to the embodiment. FIG. 2 is a conceptual diagram showing an example of the configuration of mesh data according to the embodiment. FIG. 2 is a conceptual diagram illustrating an example of a data file of mesh data according to the embodiment. FIG. 2 is a conceptual diagram showing types of three-dimensional data according to the embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of a three-dimensional data encoder according to an embodiment. FIG. 2 is a block diagram showing an example of the configuration of a three-dimensional data decoder according to an embodiment. FIG. 10 is a block diagram showing another example configuration of a three-dimensional data encoder according to an embodiment. FIG. 10 is a block diagram showing another example configuration of the three-dimensional data decoder according to the embodiment. FIG. 10 is a conceptual diagram showing a specific example of encoding processing according to an embodiment. FIG. 10 is a conceptual diagram showing a specific example of a decoding process according to an embodiment. FIG. 1 is a block diagram illustrating an example implementation of an encoding device according to an embodiment. FIG. 2 is a block diagram illustrating an example implementation of a decoding device according to an embodiment. FIG. 10 is a block diagram showing another example configuration of the encoding/decoding system according to the embodiment. FIG. 10 is a block diagram showing another example configuration of an encoding device according to an embodiment. FIG. 10 is a block diagram illustrating another example configuration of a decoding device according to an embodiment. FIG. 10 is a block diagram showing another example configuration of an encoding device according to an embodiment. FIG. 10 is a block diagram illustrating another example configuration of a decoding device according to an embodiment. FIG. 10 is a flowchart showing the processing of the encoding device according to the embodiment. 1 is an explanatory diagram conceptually illustrating encoding of a mesh frame according to an embodiment; FIG. 10 is a flowchart showing the process of the decoding device according to the embodiment. FIG. 10 is an explanatory diagram conceptually illustrating decoding of a mesh frame according to an embodiment. FIG. 2 is a block diagram illustrating an example of a configuration of a decoding device according to an embodiment. FIG. 2 is a block diagram illustrating an example of a configuration of a decoding device according to an embodiment. FIG. 10 is an explanatory diagram illustrating an example of subdivision according to an embodiment. 10A and 10B are explanatory diagrams showing examples of displacement of vertices after subdivision according to an embodiment; 10A and 10B are explanatory diagrams showing examples of vertices of an original mesh according to an embodiment; FIG. 10 is an explanatory diagram showing an example of a mesh according to an embodiment. 10A and 10B are explanatory diagrams illustrating an example of dividing a mesh into sub-meshes according to an embodiment. FIG. 10 is a first explanatory diagram showing an example of packing displacement information into an image frame according to the embodiment; FIG. 10 is a second explanatory diagram showing an example of packing displacement information into an image frame according to the embodiment. FIG. 10 is a third explanatory diagram showing an example of packing displacement information into an image frame according to the embodiment. 10A and 10B are diagrams for explaining examples of attribute information in mesh data according to an embodiment; 10A and 10B are diagrams for explaining examples of attribute information in mesh data according to an embodiment; 10A and 10B are diagrams for explaining examples of attribute information in mesh data according to an embodiment; FIG. 10 is a diagram illustrating a specific example of a method for encoding attribute information. FIG. 10 is a diagram illustrating a specific example of a method for encoding attribute information. FIG. 10 is a diagram illustrating a specific example of a method for encoding attribute information. FIG. 10 is a diagram illustrating a specific example of a method for encoding attribute information. FIG. 10 is a diagram illustrating a specific example of a method for encoding attribute information. FIG. 10 is a diagram illustrating a specific example of a method for encoding attribute information. FIG. 10 is a diagram illustrating an example of a flowchart for encoding attribute information of a three-dimensional point according to an embodiment. 10 is a flowchart illustrating an example of a calculation process of a predicted value in a third prediction mode according to an embodiment. FIG. 10 is a diagram illustrating an example of a flowchart for decoding attribute information of a three-dimensional point according to an embodiment. FIG. 10 is a diagram illustrating an example of the syntax of mesh_attribute_coding according to the embodiment. FIG. 10 is a diagram illustrating an example of the syntax of mesh_attribute_header according to the embodiment. FIG. 10 is a diagram illustrating an example of the syntax of mesh_attribute_data according to the embodiment. FIG. 10 is a diagram showing an example of attribute information according to a modified example of the embodiment. FIG. 10 is a diagram illustrating an example of the syntax of mesh_attribute_header according to a modified example of the embodiment. FIG. 10 is a diagram illustrating an example of a geometry map according to the embodiment. FIG. 2 is a diagram illustrating an example of a texture map according to an embodiment; FIG. 10 is a diagram illustrating an example of the syntax of mesh_position_coding according to the embodiment. FIG. 10 is a diagram illustrating an example of the syntax of mesh_position_header according to the embodiment. FIG. 10 is a diagram illustrating an example of the syntax of mesh_position_data according to the embodiment. FIG. 1 illustrates an example of a configuration of an encoding device according to an embodiment. 1 is a flowchart illustrating an example of an encoding method performed by an encoding device according to an embodiment. FIG. 2 is a diagram illustrating an example of a configuration of a decoding device according to an embodiment. 10 is a flowchart illustrating an example of a decoding method performed by a decoding device according to an embodiment. 10 is a flowchart illustrating another example of the process of calculating a predicted value in the third prediction mode according to the embodiment. FIG. 10 is a diagram illustrating an example of vectors in a three-dimensional space when fine prediction is performed according to the embodiment. FIG. 10 is a diagram illustrating an example of vectors on a two-dimensional plane when fine prediction is performed according to the embodiment. A figure showing an example of calculating a predicted value when the absolute value of dot(gNgP, gNgC) in the embodiment exceeds a threshold value TH. A figure showing an example of calculating a predicted value when the absolute value of dot(gNgP, gNgC) in the embodiment is less than or equal to the threshold value TH. FIG. 10 is a diagram illustrating another example of calculation of a predicted value according to the embodiment. 10 is a flowchart illustrating an example of a calculation process of a predicted value according to an embodiment. FIG. 10 is a diagram illustrating an example of vectors in a three-dimensional space when fine prediction is performed according to a modification of the embodiment. FIG. 10 is a diagram illustrating an example of vectors on a two-dimensional plane when fine prediction is performed according to a modification of the embodiment. FIG. 10 is a diagram illustrating another example of calculation of a predicted value according to the embodiment. 10 is a flowchart illustrating another example of the process of calculating a predicted value according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. 10A to 10C are diagrams for explaining another specific example of the square root calculation process (sqrt) according to the embodiment. FIG. 10 is a diagram illustrating another example of the configuration of an encoding device according to an embodiment. 10 is a flowchart illustrating another example of an encoding method performed by an encoding device according to an embodiment. FIG. 10 is a diagram illustrating another example of the configuration of a decoding device according to an embodiment. 10 is a flowchart illustrating another example of a decoding method performed by a decoding device according to an embodiment.

<Summary of the Disclosure>
For example, three-dimensional (3D) meshes are used in computer graphics images, which may be composed of multiple temporally distinct frames, each of which may be represented by a 3D mesh.

A three-dimensional mesh is composed of vertex information indicating the position of each of multiple vertices in three-dimensional space, connection information indicating the connections between the multiple vertices, and attribute information indicating the attributes of each vertex or each face. Each face is constructed according to the connection relationships between the multiple vertices. A variety of computer graphics images can be expressed using such three-dimensional meshes.

Furthermore, for transmission and storage of three-dimensional meshes, efficient encoding and decoding of three-dimensional meshes is expected. Arithmetic coding and arithmetic decoding may be used for efficient encoding and decoding of three-dimensional meshes.

Further improvements are desired in the encoding or decoding process for three-dimensional data. The present disclosure aims to improve the encoding or decoding process for three-dimensional data.

Below, we will present examples of inventions that can be obtained from the disclosures of this specification and explain the effects and other benefits that can be obtained from these inventions.

The encoding method according to a first aspect of the present disclosure is an encoding method executed by an encoding device, which obtains multiple vertices included in a three-dimensional mesh and predictively encodes one of the multiple vertices. In the predictive encoding, a first predicted value for the one vertex is calculated with decimal precision, a second predicted value is calculated by reducing the number of bits in the decimal part of the first predicted value, and the one vertex is predictively encoded using the second predicted value.

By reducing the number of bits in the decimal part of the predicted value calculated with decimal precision, it is possible to reduce the amount of data while controlling the accuracy during calculation. Reducing the number of bits reduces the amount of encoding processing, thereby improving processing efficiency.

The encoding method according to the second aspect of the present disclosure is the encoding method according to the first aspect, wherein in calculating the first predicted value, two or more third predicted values are calculated with decimal precision based on each of the two or more processed vertices, and the first predicted value is calculated by integrating the two or more third predicted values.

By combining multiple predicted values, it is possible to obtain a highly accurate predicted value. This reduces prediction errors and improves coding efficiency. Furthermore, combining multiple predicted values allows for greater flexibility in the method for selecting an appropriate predicted value.

The encoding method according to the third aspect of the present disclosure is the encoding method according to the second aspect, wherein the first predicted value is calculated by integrating the two or more third predicted values with decimal precision.

By performing the integration process with decimal precision and then converting to integers later, it is possible to perform the encoding process while maintaining the precision of the integration process. By performing high-precision integration and then converting to integers, it is possible to suppress an increase in prediction error.

The encoding method according to the fourth aspect of the present disclosure is the encoding method according to the second or third aspect, wherein the two or more third predicted values and the first predicted value have decimal parts with a predetermined number of bits, and the second predicted value is calculated by reducing the predetermined number of bits of the decimal part of the first predicted value.

This allows for easy integer conversion by reducing the decimal portion by a predetermined number of bits. Reducing the number of bits reduces the amount of code required, improving the efficiency of calculation processing.

The encoding method according to the fifth aspect of the present disclosure is the encoding method according to any one of the second to fourth aspects, wherein the first predicted value is the average of the two or more third predicted values.

This allows the overall prediction error to be reduced by averaging multiple predicted values. Averaging can potentially cancel out noise components and prediction errors, improving coding accuracy.

The encoding method according to the sixth aspect of the present disclosure is the encoding method according to the fourth aspect, wherein the predictive encoding does not reduce the predetermined number of bits before calculating the second predicted value.

This allows the accuracy of the predicted value to be maintained by not reducing the number of bits before converting to integers. Since there is no loss of accuracy during calculation, high-precision encoding can be achieved.

The encoding method according to a seventh aspect of the present disclosure is an encoding method according to any one of the first to sixth aspects, wherein in the predictive encoding of the one vertex using the second predicted value, a fourth predicted value is calculated by rounding the second predicted value, and the one vertex is predictively encoded using the fourth predicted value.

This allows rounding to be performed to reduce errors after the decimal point. This improves the accuracy of the encoded data and minimizes errors when reducing the number of bits.

The encoding method according to an eighth aspect of the present disclosure is the encoding method according to the seventh aspect, wherein the rounding process rounds the second predicted value in a different direction depending on whether the second predicted value is a positive number or a negative number.

This allows for improved encoding accuracy by switching the rounding direction depending on whether the predicted value is positive or negative. In particular, it allows for optimal rounding processing based on the magnitude and sign of the predicted value.

A decoding method according to a ninth aspect of the present disclosure is a decoding method executed by a decoding device, which obtains multiple vertices included in a three-dimensional mesh and predictively decodes one of the multiple vertices. In the predictive decoding, a first predicted value for the one vertex is calculated with decimal precision, a second predicted value is calculated by reducing the number of bits in the decimal part of the first predicted value, and the one vertex is predictively decoded using the second predicted value.

By reducing the number of bits in the decimal part of the predicted value calculated with decimal precision, it is possible to reduce the amount of data while controlling the accuracy during calculation. Reducing the number of bits reduces the amount of decoding processing, thereby improving processing efficiency.

A decoding method according to a tenth aspect of the present disclosure is the decoding method according to the ninth aspect, wherein in calculating the first predicted value, two or more third predicted values are calculated with decimal precision based on each of the two or more processed vertices, and the first predicted value is calculated by integrating the two or more third predicted values.

By combining multiple predicted values, a highly accurate predicted value can be obtained. This reduces decoding errors and improves decoding accuracy.

A decoding method according to an eleventh aspect of the present disclosure is the decoding method according to the tenth aspect, wherein the first predicted value is calculated by integrating the two or more third predicted values with decimal precision.

This allows for more accurate predictions to be calculated by integrating with decimal precision. By converting the values to integers later, the decoding process can be performed efficiently.

A decoding method according to a twelfth aspect of the present disclosure is a decoding method according to the tenth or eleventh aspect, wherein the two or more third predicted values and the first predicted value have decimal parts with a predetermined number of bits, and the second predicted value is calculated by reducing the predetermined number of bits of the decimal part of the first predicted value.

This reduces the number of bits in the decimal part and converts it to an integer, thereby reducing the burden of the decoding process and enabling the use of highly accurate prediction values.

A decoding method according to a thirteenth aspect of the present disclosure is a decoding method according to any one of the tenth to twelfth aspects, in which the first predicted value is an average of the two or more third predicted values.

This allows prediction errors to be reduced by averaging multiple predicted values. The averaging process reduces the variance in predicted values, enabling highly accurate decoding.

A decoding method according to a fourteenth aspect of the present disclosure is the decoding method according to the twelfth aspect, wherein in the predictive decoding, the predetermined number of bits is not reduced before the second predicted value is calculated.

This means that there is no loss of accuracy due to bit reduction until integer conversion occurs during the decoding process, allowing for high-precision decoding.

A decoding method according to a fifteenth aspect of the present disclosure is a decoding method according to any one of the ninth to fourteenth aspects, wherein in the predictive decoding of the one vertex using the second predicted value, a fourth predicted value is calculated by rounding the second predicted value, and the one vertex is predictively decoded using the fourth predicted value.

By applying rounding, errors can be minimized when reducing the number of bits, allowing highly accurate prediction values to be obtained during the decoding process.

A decoding method according to a sixteenth aspect of the present disclosure is the decoding method according to the fifteenth aspect, wherein the rounding process rounds the second predicted value in a different direction depending on whether the second predicted value is a positive number or a negative number.

This allows the accuracy of the predicted value to be improved by adjusting the direction of the rounding process depending on whether the predicted value is positive or negative. It also reduces the occurrence of rounding errors.

A coding device according to a seventeenth aspect of the present disclosure includes a circuit and a memory connected to the circuit, and in operation, the circuit acquires multiple vertices included in a three-dimensional mesh and predictively encodes one of the multiple vertices, calculating a first predicted value for the one vertex with decimal precision, reducing the number of bits in the decimal part of the first predicted value to calculate a second predicted value, and predictively encoding the one vertex using the second predicted value.

By reducing the number of bits in the decimal part of the predicted value calculated with decimal precision, it is possible to reduce the amount of data while controlling the accuracy during calculation. Reducing the number of bits reduces the amount of encoding processing, thereby improving processing efficiency.

A decoding device according to an eighteenth aspect of the present disclosure includes a circuit and a memory connected to the circuit, and in operation, the circuit acquires multiple vertices included in a three-dimensional mesh and predictively decodes one of the multiple vertices. In the predictive decoding, the circuit calculates a first predicted value for the one vertex with decimal precision, calculates a second predicted value by reducing the number of bits in the decimal part of the first predicted value, and predictively decodes the one vertex using the second predicted value.

By reducing the number of bits in the decimal part of the predicted value calculated with decimal precision, it is possible to reduce the amount of data while controlling the accuracy during calculation. Reducing the number of bits reduces the amount of decoding processing, thereby improving processing efficiency.

Note that these comprehensive or specific aspects may be realized as a system, device, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or as any combination of a system, device, integrated circuit, computer program, or recording medium.

The following describes the embodiments in detail with reference to the drawings.

The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components related to the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

(Embodiment)
In this embodiment, an encoding method, a decoding method, etc. will be described.

<Expressions and terms>
The following expressions and terms are used herein:

(1) Three-dimensional Mesh A three-dimensional mesh is a collection of multiple faces, and represents, for example, a three-dimensional object. A three-dimensional mesh is mainly composed of vertex information, connectivity information, and attribute information. A three-dimensional mesh may be expressed as a polygon mesh or a mesh. A three-dimensional mesh may also vary over time. A three-dimensional mesh may include metadata related to the vertex information, connectivity information, and attribute information, and may also include other additional information.

(2) Vertex Information Vertex information is information indicating a vertex. For example, the vertex information indicates the position of a vertex in a three-dimensional space. Furthermore, a vertex corresponds to a vertex of a face that constitutes a three-dimensional mesh. Vertex information may be expressed as "geometry." Furthermore, vertex information may be expressed as position information.

(3) Connection Information Connection information is information that indicates connections between vertices. For example, connection information indicates connections for forming faces or edges of a three-dimensional mesh. Connection information may be expressed as "Connectivity." Connection information may also be expressed as face information.

(4) Attribute Information Attribute information is information that indicates attributes of a vertex or a face. For example, attribute information indicates attributes such as a color, an image, and a normal vector associated with a vertex or a face. Attribute information may be expressed as "texture."

(5) Faces A face is an element that makes up a three-dimensional mesh. Specifically, a face is a polygon on a plane in three-dimensional space. For example, a face can be defined as a triangle in three-dimensional space.

(6) Plane A plane is a two-dimensional plane in a three-dimensional space. For example, a polygon is formed on a plane, and multiple polygons are formed on multiple planes.

(7) Bitstream: A bitstream corresponds to coded information. A bitstream may also be referred to as a stream, a coded bitstream, a compressed bitstream, or a coded signal.

(8) Encoding and Decoding The term encoding may be substituted with terms such as storing, including, writing, describing, signaling, sending, notifying, saving, or compressing, and these terms may be interchangeable. For example, encoding information may mean including the information in a bitstream. Also, encoding information into a bitstream may mean encoding the information to generate a bitstream that includes the encoded information.

Furthermore, the expression "decode" may be replaced with expressions such as "read out," "interpret," "read," "load," "derive," "obtain," "receive," "extract," "reconstruct," "reconstruct," "decompress," or "decompress," and these expressions may be interchangeable. For example, decoding information may mean obtaining information from a bitstream. Furthermore, decoding information from a bitstream may mean decoding the bitstream to obtain the information contained in the bitstream.

(9) Ordinal Numbers In the description, ordinal numbers such as first and second may be assigned to components, etc. These ordinal numbers may be changed as appropriate. Furthermore, new ordinal numbers may be assigned to components, etc., or removed. Furthermore, these ordinal numbers may be assigned to elements in order to identify them, and may not correspond to a meaningful order.

<3D mesh>
FIG. 1 is a conceptual diagram showing a three-dimensional mesh according to this embodiment. The three-dimensional mesh is composed of a plurality of faces. For example, each face is a triangle. The vertices of these triangles are defined in three-dimensional space. The three-dimensional mesh then represents a three-dimensional object. Each face may have a color or an image.

Figure 2 is a conceptual diagram showing the basic elements of a three-dimensional mesh according to this embodiment. A three-dimensional mesh is composed of vertex information, connection information, and attribute information. Vertex information indicates the positions of the vertices of a face in three-dimensional space. Connection information indicates the connections between vertices. A face can be identified by the vertex information and connection information. In other words, a colorless three-dimensional object is formed in three-dimensional space by the vertex information and connection information.

Attribute information may be associated with a vertex or a face. Attribute information associated with a vertex may be expressed as "Attribute Per Point." Attribute information associated with a vertex may indicate the attributes of the vertex itself, or may indicate the attributes of the face connected to the vertex.

For example, a color may be associated with a vertex as attribute information. The color associated with a vertex may be the color of the vertex itself, or the color of the face connected to the vertex. The color of a face may be the average of multiple colors associated with multiple vertices of the face. Furthermore, a normal vector may be associated with a vertex or face as attribute information. Such a normal vector can represent the front and back of a face.

Furthermore, a two-dimensional image may be associated with a surface as attribute information. The two-dimensional image associated with a surface is also referred to as a texture image or an "Attribute Map." Furthermore, information indicating the mapping between the surface and the two-dimensional image may be associated with the surface as attribute information. Information indicating such mapping may be referred to as mapping information, vertex information of the texture image, texture coordinates, or "Attribute UV Coordinate."

Furthermore, information such as colors, images, and moving images used as attribute information may be expressed as "Parametric Space."

This attribute information allows texture to be reflected on the three-dimensional object. In other words, a three-dimensional object with color is formed in three-dimensional space using vertex information, connection information, and attribute information.

Note that in the above, attribute information is associated with vertices or faces, but it may also be associated with edges.

FIG. 3 is a conceptual diagram showing mapping according to this embodiment. For example, a region of a two-dimensional image on a two-dimensional plane can be mapped onto the surface of a three-dimensional mesh in three-dimensional space. Specifically, coordinate information of the region in the two-dimensional image is associated with the surface of the three-dimensional mesh. As a result, the image of the mapped region in the two-dimensional image is reflected on the surface of the three-dimensional mesh.

By using mapping, the two-dimensional image used as attribute information can be separated from the three-dimensional mesh. For example, when encoding the three-dimensional mesh, the two-dimensional image may be encoded using an image encoding method or a video encoding method.

<System Configuration>
4 is a block diagram showing an example of the configuration of a coding/decoding system according to this embodiment. In FIG. 4, the coding/decoding system includes a coding device 100 and a decoding device 200.

For example, the encoding device 100 acquires a three-dimensional mesh and encodes the three-dimensional mesh into a bitstream. The encoding device 100 then outputs the bitstream to the network 300. For example, the bitstream includes the encoded three-dimensional mesh and control information for decoding the encoded three-dimensional mesh. By encoding the three-dimensional mesh, the information about the three-dimensional mesh is compressed.

Network 300 transmits the bitstream from encoding device 100 to decoding device 200. Network 300 may be the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. Network 300 is not necessarily limited to two-way communication, and may also be a one-way communication network for terrestrial digital broadcasting, satellite broadcasting, etc.

Furthermore, network 300 can be replaced by a recording medium such as a DVD (Digital Versatile Disc) or a BD (Blu-Ray Disc (registered trademark)).

The decoding device 200 obtains a bitstream and decodes a three-dimensional mesh from the bitstream. By decoding the three-dimensional mesh, the information of the three-dimensional mesh is expanded. For example, the decoding device 200 decodes the three-dimensional mesh according to a decoding method that corresponds to the encoding method used by the encoding device 100 to encode the three-dimensional mesh. In other words, the encoding device 100 and the decoding device 200 perform encoding and decoding according to encoding methods and decoding methods that correspond to each other.

Note that the 3D mesh before encoding can also be referred to as the original 3D mesh. The 3D mesh after decoding can also be referred to as the reconstructed 3D mesh.

<Encoding device>
5 is a block diagram showing an example of the configuration of an encoding device 100 according to this embodiment. For example, the encoding device 100 includes a vertex information encoder 101, a connection information encoder 102, and an attribute information encoder 103.

The vertex information encoder 101 is an electrical circuit that encodes vertex information. For example, the vertex information encoder 101 encodes the vertex information into a bitstream according to a format specified for the vertex information.

The connection information encoder 102 is an electrical circuit that encodes the connection information. For example, the connection information encoder 102 encodes the connection information into a bitstream according to a format specified for the connection information.

The attribute information encoder 103 is an electrical circuit that encodes attribute information. For example, the attribute information encoder 103 encodes the attribute information into a bitstream according to a format specified for the attribute information.

Vertex information, connectivity information, and attribute information may be encoded using variable-length coding or fixed-length coding. Variable-length coding may correspond to Huffman coding, context-adaptive binary arithmetic coding (CABAC), or the like.

The vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be integrated. Alternatively, each of the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be further subdivided into multiple component parts.

FIG. 6 is a block diagram showing another example configuration of the encoding device 100 according to this embodiment. For example, the encoding device 100 includes a pre-processor 104 and a post-processor 105 in addition to the configuration shown in FIG. 5.

The preprocessor 104 is an electrical circuit that performs processing on the vertex information, connectivity information, and attribute information before encoding. For example, the preprocessor 104 may perform conversion processing, separation processing, multiplexing processing, etc. on the three-dimensional mesh before encoding. More specifically, for example, the preprocessor 104 may separate the vertex information, connectivity information, and attribute information from the three-dimensional mesh before encoding.

The post-processor 105 is an electrical circuit that performs processing after the vertex information, connection information, and attribute information are encoded. For example, the post-processor 105 may perform conversion processing, separation processing, multiplexing processing, etc. on the encoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 105 may multiplex the encoded vertex information, connection information, and attribute information into a bitstream. Furthermore, for example, the post-processor 105 may further perform variable-length coding on the encoded vertex information, connection information, and attribute information.

<Decryption device>
7 is a block diagram showing an example of the configuration of a decoding device 200 according to this embodiment. For example, the decoding device 200 includes a vertex information decoder 201, a connection information decoder 202, and an attribute information decoder 203.

The vertex information decoder 201 is an electrical circuit that decodes vertex information. For example, the vertex information decoder 201 decodes vertex information from the bitstream according to a format specified for the vertex information.

The connection information decoder 202 is an electrical circuit that decodes the connection information. For example, the connection information decoder 202 decodes the connection information from the bitstream according to a format specified for the connection information.

The attribute information decoder 203 is an electrical circuit that decodes attribute information. For example, the attribute information decoder 203 decodes attribute information from the bitstream according to a format specified for the attribute information.

Vertex information, connectivity information, and attribute information may be decoded using variable-length decoding or fixed-length decoding. Variable-length decoding may be compatible with Huffman coding or context-adaptive binary arithmetic coding (CABAC), etc.

The vertex information decoder 201, connection information decoder 202, and attribute information decoder 203 may be integrated. Alternatively, each of the vertex information decoder 201, connection information decoder 202, and attribute information decoder 203 may be further subdivided into multiple component parts.

FIG. 8 is a block diagram showing another example configuration of the decoding device 200 according to this embodiment. For example, the decoding device 200 includes a pre-processor 204 and a post-processor 205 in addition to the configuration shown in FIG. 7.

The preprocessor 204 is an electrical circuit that performs processing before decoding the vertex information, connection information, and attribute information. For example, the preprocessor 204 may perform conversion processing, separation processing, multiplexing processing, etc. on the bitstream before decoding the vertex information, connection information, and attribute information.

More specifically, for example, the preprocessor 204 may separate from the bitstream a sub-bitstream corresponding to vertex information, a sub-bitstream corresponding to connectivity information, and a sub-bitstream corresponding to attribute information. Also, for example, the preprocessor 204 may perform variable-length decoding on the bitstream in advance before decoding the vertex information, connectivity information, and attribute information.

The post-processor 205 is an electrical circuit that performs processing after the vertex information, connection information, and attribute information are decoded. For example, the post-processor 205 may perform conversion processing, separation processing, multiplexing processing, etc. on the decoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 205 may multiplex the decoded vertex information, connection information, and attribute information onto a three-dimensional mesh.

<Bitstream>
The vertex information, connection information, and attribute information are coded and stored in a bitstream. The relationship between these pieces of information and the bitstream is shown below.

FIG. 9 is a conceptual diagram showing an example of the configuration of a bitstream according to this embodiment. In this example, connection information, vertex information, and attribute information are integrated in the bitstream. For example, connection information, vertex information, and attribute information may be included in a single file.

Furthermore, multiple parts of this information may be stored sequentially, such as a first part of connection information, a first part of vertex information, a first part of attribute information, a second part of connection information, a second part of vertex information, a second part of attribute information, etc. These multiple parts may correspond to multiple parts that are different in time, multiple parts that are different in space, or multiple different faces.

Furthermore, the storage order of connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

FIG. 10 is a conceptual diagram showing another example of the configuration of a bitstream according to this embodiment. In this example, multiple files are included in the bitstream, with connection information, vertex information, and attribute information stored in different files. Here, a file containing connection information, a file containing vertex information, and a file containing attribute information are shown, but the storage format is not limited to this example. For example, two types of information out of connection information, vertex information, and attribute information may be included in one file, and the remaining type of information may be included in another file.

Alternatively, this information may be split and stored in more files. For example, multiple parts of the connection information may be stored in multiple files, multiple parts of the vertex information may be stored in multiple files, or multiple parts of the attribute information may be stored in multiple files. These multiple parts may correspond to multiple parts that are different in time, multiple parts that are different in space, or multiple different faces.

Furthermore, the storage order of connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

FIG. 11 is a conceptual diagram showing another example of the configuration of a bitstream according to this embodiment. In this example, the bitstream is composed of multiple separable sub-bitstreams, and connection information, vertex information, and attribute information are each stored in different sub-bitstreams.

Here, a sub-bitstream containing connection information, a sub-bitstream containing vertex information, and a sub-bitstream containing attribute information are shown, but the storage format is not limited to this example.

For example, two types of information out of connection information, vertex information, and attribute information may be included in one sub-bitstream, and the remaining type of information may be included in another sub-bitstream. Specifically, attribute information for a two-dimensional image, etc., may be stored in a sub-bitstream that complies with an image coding method, separate from the sub-bitstreams for connection information and vertex information.

Furthermore, each sub-bitstream may contain multiple files. Multiple pieces of connection information may be stored in multiple files, multiple pieces of vertex information may be stored in multiple files, or multiple pieces of attribute information may be stored in multiple files.

Furthermore, the storage order of the connection information, vertex information, and attribute information is not limited to the examples in Figures 9, 10, and 11, and a storage order different from the above examples may be used. For example, the vertex information, connection information, and attribute information may be stored in the bitstream in this order. Alternatively, the bitstream may be stored in any of the following orders: connection information, attribute information, and vertex information; vertex information, attribute information, and connection information; attribute information, connection information, and vertex information; or attribute information, vertex information, and connection information.

Furthermore, each of the connection information, vertex information, and attribute information may be divided into multiple pieces of data, and the multiple pieces of data may be stored in a cyclical or random order within the bitstream.

<Specific examples>
12 is a block diagram showing a specific example of an encoding/decoding system according to this embodiment. In FIG. 12, the encoding/decoding system includes a three-dimensional data encoding system 110, a three-dimensional data decoding system 210, and an external connector 310.

The three-dimensional data encoding system 110 comprises a controller 111, an input/output processor 112, a three-dimensional data encoder 113, a three-dimensional data generator 115, and a system multiplexer 114. The three-dimensional data decoding system 210 comprises a controller 211, an input/output processor 212, a three-dimensional data decoder 213, a system demultiplexer 214, a presenter 215, and a user interface 216.

In the three-dimensional data encoding system 110, sensor data is input from the sensor terminal to the three-dimensional data generator 115. The three-dimensional data generator 115 generates three-dimensional data, such as point cloud data or mesh data, from the sensor data and inputs it to the three-dimensional data encoder 113.

For example, the three-dimensional data generator 115 generates vertex information, and generates connection information and attribute information corresponding to the vertex information. The three-dimensional data generator 115 may process the vertex information when generating the connection information and attribute information. For example, the three-dimensional data generator 115 may reduce the amount of data by deleting duplicate vertices, or may transform the vertex information (such as by shifting its position, rotating it, or normalizing it). The three-dimensional data generator 115 may also render the attribute information.

Furthermore, although the three-dimensional data generator 115 is a component of the three-dimensional data encoding system 110 in FIG. 12, it may also be located externally and independently of the three-dimensional data encoding system 110.

The sensor terminal that provides the sensor data for generating the three-dimensional data may be, for example, a moving object such as an automobile, a flying object such as an airplane, a mobile terminal, or a camera. Furthermore, distance sensors such as LIDAR, millimeter-wave radar, infrared sensors, or range finders, stereo cameras, or combinations of multiple monocular cameras may also be used as sensor terminals.

Sensor data may include the distance (position) of the object, monocular camera images, stereo camera images, color, reflectance, sensor attitude, orientation, gyro, sensing position (GPS information or altitude), speed, acceleration, sensing time, temperature, air pressure, humidity, or magnetism.

The three-dimensional data encoder 113 corresponds to the encoding device 100 shown in FIG. 5, etc. For example, the three-dimensional data encoder 113 encodes three-dimensional data to generate encoded data. The three-dimensional data encoder 113 also generates control information when encoding the three-dimensional data. The three-dimensional data encoder 113 then inputs the encoded data together with the control information to the system multiplexer 114.

The encoding method for three-dimensional data may be an encoding method that uses geometry, or an encoding method that uses a video codec. Here, an encoding method that uses geometry may also be referred to as a geometry-based encoding method. An encoding method that uses a video codec may also be referred to as a video-based encoding method.

The system multiplexer 114 multiplexes the coded data and control information input from the three-dimensional data encoder 113, generating multiplexed data using a specified multiplexing method. Along with the coded data and control information for the three-dimensional data, the system multiplexer 114 may also multiplex other media such as video, audio, subtitles, application data, or document files, or reference time information. Furthermore, the system multiplexer 114 may also multiplex attribute information related to the sensor data or three-dimensional data.

For example, the multiplexed data may have a file format for storage or a packet format for transmission. These formats may include ISOBMFF or an ISOBMFF-based format. Also, MPEG-DASH, MMT, MPEG-2 TS Systems, RTP, or the like may be used.

The multiplexed data is then output as a transmission signal to the external connector 310 by the input/output processor 112. The multiplexed data may be transmitted as a transmission signal via a wired connection or wirelessly. Alternatively, the multiplexed data may be stored in internal memory or a storage device. The multiplexed data may be transmitted to a cloud server via the Internet, or may be stored in an external storage device.

For example, the transmission or storage of multiplexed data is performed using a method appropriate to the medium for transmission or storage, such as broadcasting or communication. The communication protocol may be http, ftp, TCP, UDP, IP, or a combination of these. Furthermore, a pull-type communication method or a push-type communication method may be used.

For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), or coaxial cable may be used. For wireless transmission, 3GPP (registered trademark), 3G/4G/5G as defined by IEEE, wireless LAN, Wi-Fi, Bluetooth, or millimeter waves may be used. For broadcasting, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 may be used, for example.

In addition, the sensor data may be input to the three-dimensional data generator 115 or the system multiplexer 114. Also, the three-dimensional data or encoded data may be output directly as a transmission signal to the external connector 310 via the input/output processor 112. The transmission signal output from the three-dimensional data encoding system 110 is input to the three-dimensional data decoding system 210 via the external connector 310.

Furthermore, each operation of the three-dimensional data encoding system 110 may be controlled by a controller 111 that executes an application program.

In the three-dimensional data decoding system 210, a transmission signal is input to the input/output processor 212. The input/output processor 212 decodes multiplexed data in file format or packet format from the transmission signal and inputs the multiplexed data to the system demultiplexer 214. The system demultiplexer 214 obtains coded data and control information from the multiplexed data and inputs them to the three-dimensional data decoder 213. The system demultiplexer 214 may also extract other media or reference time information from the multiplexed data.

The three-dimensional data decoder 213 corresponds to the decoding device 200 shown in FIG. 7, etc. For example, the three-dimensional data decoder 213 decodes three-dimensional data from the encoded data based on a predefined encoding method. The three-dimensional data is then presented to the user by the presenter 215.

Furthermore, additional information such as sensor data may be input to the presenter 215. The presenter 215 may present three-dimensional data based on the additional information. Furthermore, user instructions may be input from a user terminal to the user interface 216. The presenter 215 may then present three-dimensional data based on the input instructions.

In addition, the input/output processor 212 may obtain three-dimensional data and encoded data from the external connector 310.

Furthermore, each operation of the three-dimensional data decoding system 210 may be controlled by a controller 211 that executes an application program.

FIG. 13 is a conceptual diagram showing an example of the configuration of point cloud data according to this embodiment. The point cloud data is data of a group of points representing a three-dimensional object.

Specifically, a point cloud is made up of multiple points, and has position information indicating the three-dimensional coordinate position of each point, and attribute information indicating the attributes of each point. Position information is also expressed as geometry.

The type of attribute information may be, for example, color or reflectance. A single point may be associated with attribute information of a single type, or a single point may be associated with attribute information of multiple different types, or a single point may be associated with attribute information having multiple values for the same type.

FIG. 14 is a conceptual diagram showing an example data file of point cloud data according to this embodiment. In this example, there is a one-to-one correspondence between location information items and attribute information items, and the location information and attribute information of N points that make up the point cloud data are shown. In this example, the location information is information that indicates a three-dimensional coordinate position using the three axes x, y, and z, and the attribute information is information that indicates a color using RGB. A PLY file or the like can be used as a typical data file for point cloud data.

FIG. 15 is a conceptual diagram showing an example of the configuration of mesh data according to this embodiment. Mesh data is data used in CG (Computer Graphics) and the like, and is three-dimensional mesh data that shows the three-dimensional shape of an object using multiple surfaces. Each surface is also expressed as a polygon, and has a polygonal shape such as a triangle or a rectangle.

Specifically, a 3D mesh is made up of multiple points that make up a point cloud, as well as multiple edges and faces. Each point is also expressed as a vertex or position. Each edge corresponds to a line segment connected by two vertices. Each face corresponds to an area surrounded by three or more edges.

A three-dimensional mesh also has position information that indicates the three-dimensional coordinate positions of its vertices. This position information is also expressed as vertex information or geometry. A three-dimensional mesh also has connection information that indicates the relationship between the multiple vertices that make up an edge or face. This connection information is also expressed as connectivity. A three-dimensional mesh also has attribute information that indicates the attributes of the vertices, edges, or faces. The attribute information in a three-dimensional mesh is also expressed as texture.

For example, attribute information may indicate the color, reflectance, or normal vector for a vertex, edge, or face. The direction of the normal vector may represent the front and back of the face.

An object file or similar format may be used as the data file format for mesh data.

FIG. 16 is a conceptual diagram showing an example data file of mesh data according to this embodiment. In this example, the data file contains position information G(1) to G(N) for the N vertices that make up the three-dimensional mesh, and attribute information A1(1) to A1(N) for the N vertices. In this example, M pieces of attribute information A2(1) to A2(M) are also included. The attribute information items do not have to correspond one-to-one to vertices or faces. In addition, attribute information does not have to exist.

Connectivity information is indicated by a combination of vertex indices. n[1, 3, 4] indicates a triangular face made up of three vertices, n=1, n=3, and n=4. Also, m[2, 4, 6] indicates that the attribute information for m=2, m=4, and m=6 corresponds to the three vertices, respectively.

Furthermore, the actual contents of the attribute information may be recorded in a separate file. A pointer to that content may then be associated with a vertex, face, or the like. For example, attribute information indicating an image for a face may be stored in a two-dimensional attribute map file. The file name of the attribute map and the two-dimensional coordinate values in the attribute map may then be recorded in attribute information A2(1) to A2(M). The method of specifying attribute information for a face is not limited to these methods, and any method may be used.

FIG. 17 is a conceptual diagram showing the types of three-dimensional data according to this embodiment. Point cloud data and mesh data may represent static objects or dynamic objects. A static object is an object that does not change over time, and a dynamic object is an object that changes over time. A static object may correspond to three-dimensional data for any point in time.

For example, point cloud data for a given point in time may be expressed as a PCC frame. Mesh data for a given point in time may be expressed as a mesh frame. Furthermore, PCC frames and mesh frames may be simply expressed as frames.

Furthermore, the area of the object may be limited to a certain range, as in normal video data, or it may not be limited, as in map data. The density of points or surfaces may be defined in various ways. Sparse point cloud data or sparse mesh data may be used, or dense point cloud data or dense mesh data may be used.

Next, the encoding and decoding of point clouds or three-dimensional meshes will be described. The device, process, or syntax for encoding and decoding vertex information of three-dimensional meshes in this disclosure may be applied to the encoding and decoding of point clouds. The device, process, or syntax for encoding and decoding vertex information of three-dimensional meshes in this disclosure may be applied to the encoding and decoding of point clouds.

Furthermore, the device, process, or syntax for encoding and decoding attribute information of point clouds in the present disclosure may be applied to encoding and decoding connectivity information or attribute information of three-dimensional meshes.Furthermore, the device, process, or syntax for encoding and decoding connectivity information or attribute information of three-dimensional meshes in the present disclosure may be applied to encoding and decoding attribute information of point clouds.

Furthermore, at least some of the processing may be shared between the encoding and decoding of point cloud data and the encoding and decoding of mesh data. This can reduce the scale of the circuit and software program.

FIG. 18 is a block diagram showing an example configuration of a three-dimensional data encoder 113 according to this embodiment. In this example, the three-dimensional data encoder 113 includes a vertex information encoder 121, an attribute information encoder 122, a metadata encoder 123, and a multiplexer 124. The vertex information encoder 121, the attribute information encoder 122, and the multiplexer 124 may correspond to the vertex information encoder 101, the attribute information encoder 103, and the post-processor 105 in FIG. 6, etc.

Also, in this example, the three-dimensional data encoder 113 encodes the three-dimensional data according to a geometry-based encoding method. Encoding according to the geometry-based encoding method takes into account the three-dimensional structure. Also, encoding according to the geometry-based encoding method encodes attribute information using configuration information obtained in encoding the vertex information.

Specifically, first, the vertex information, attribute information, and metadata contained in the three-dimensional data generated from sensor data are input to a vertex information encoder 121, an attribute information encoder 122, and a metadata encoder 123, respectively. Here, the connectivity information contained in the three-dimensional data may be treated in the same way as attribute information. Furthermore, in the case of point cloud data, position information may be treated as vertex information.

The vertex information encoder 121 encodes the vertex information into compressed vertex information and outputs the compressed vertex information to the multiplexer 124 as encoded data. The vertex information encoder 121 also generates metadata for the compressed vertex information and outputs it to the multiplexer 124. The vertex information encoder 121 also generates configuration information and outputs it to the attribute information encoder 122.

The attribute information encoder 122 uses the configuration information generated by the vertex information encoder 121 to encode the attribute information into compressed attribute information and outputs the compressed attribute information as encoded data to the multiplexer 124. The attribute information encoder 122 also generates metadata for the compressed attribute information and outputs it to the multiplexer 124.

The metadata encoder 123 encodes compressible metadata into compressed metadata and outputs the compressed metadata to the multiplexer 124 as encoded data. The metadata encoded by the metadata encoder 123 may be used to encode vertex information and attribute information.

The multiplexer 124 multiplexes the compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata into a bitstream. The multiplexer 124 then inputs the bitstream to the system layer.

FIG. 19 is a block diagram showing an example configuration of a three-dimensional data decoder 213 according to this embodiment. In this example, the three-dimensional data decoder 213 includes a vertex information decoder 221, an attribute information decoder 222, a metadata decoder 223, and a demultiplexer 224. The vertex information decoder 221, the attribute information decoder 222, and the demultiplexer 224 may correspond to the vertex information decoder 201, the attribute information decoder 203, and the preprocessor 204 in FIG. 8, etc.

Also, in this example, the three-dimensional data decoder 213 decodes three-dimensional data according to a geometry-based encoding method. Decoding according to a geometry-based encoding method takes into account three-dimensional structures. Also, decoding according to a geometry-based encoding method decodes attribute information using configuration information obtained in decoding vertex information.

Specifically, first, a bitstream is input from the system layer to the demultiplexer 224. The demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream. The compressed vertex information and compressed vertex information metadata are input to the vertex information decoder 221. The compressed attribute information and compressed attribute information metadata are input to the attribute information decoder 222. The metadata is input to the metadata decoder 223.

The vertex information decoder 221 decodes vertex information from the compressed vertex information using the metadata of the compressed vertex information. The vertex information decoder 221 also generates configuration information and outputs it to the attribute information decoder 222. The attribute information decoder 222 decodes attribute information from the compressed attribute information using the configuration information generated by the vertex information decoder 221 and the metadata of the compressed attribute information. The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 may be used to decode the vertex information and the attribute information.

Then, the vertex information, attribute information, and metadata are output from the 3D data decoder 213 as 3D data. Note that this metadata is, for example, metadata about the vertex information and attribute information, and can be used in an application program.

FIG. 20 is a block diagram showing another example configuration of the three-dimensional data encoder 113 according to this embodiment. In this example, the three-dimensional data encoder 113 includes a vertex image generator 131, an attribute image generator 132, a metadata generator 133, a video encoder 134, a metadata encoder 123, and a multiplexer 124. The vertex image generator 131, the attribute image generator 132, and the video encoder 134 may correspond to the vertex information encoder 101 and the attribute information encoder 103 in FIG. 6, etc.

Also, in this example, the three-dimensional data encoder 113 encodes the three-dimensional data according to a video-based encoding method. In encoding according to the video-based encoding method, multiple two-dimensional images are generated from the three-dimensional data, and the multiple two-dimensional images are encoded according to a video encoding method. Here, the video encoding method may be HEVC (High Efficiency Video Coding) or VVC (Versatile Video Coding), etc.

Specifically, first, the vertex information and attribute information contained in the three-dimensional data generated from the sensor data is input to the metadata generator 133. The vertex information and attribute information are then input to the vertex image generator 131 and the attribute image generator 132, respectively. The metadata contained in the three-dimensional data is then input to the metadata encoder 123. Here, the connectivity information contained in the three-dimensional data may be treated in the same way as the attribute information. In the case of point cloud data, the position information may be treated as vertex information.

The metadata generator 133 generates map information for multiple two-dimensional images from the vertex information and attribute information. The metadata generator 133 then inputs the map information to the vertex image generator 131, the attribute image generator 132, and the metadata encoder 123.

The vertex image generator 131 generates a vertex image based on the vertex information and map information and inputs it to the video encoder 134. The attribute image generator 132 generates an attribute image based on the attribute information and map information and inputs it to the video encoder 134.

The video encoder 134 encodes the vertex images and attribute images into compressed vertex information and compressed attribute information, respectively, according to a video encoding method, and outputs the compressed vertex information and compressed attribute information to the multiplexer 124 as encoded data. The video encoder 134 also generates metadata for the compressed vertex information and metadata for the compressed attribute information, and outputs these to the multiplexer 124.

The metadata encoder 123 encodes compressible metadata into compressed metadata and outputs the compressed metadata to the multiplexer 124 as encoded data. The compressible metadata includes map information. The metadata encoded by the metadata encoder 123 may also be used to encode vertex information and attribute information.

The multiplexer 124 multiplexes the compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata into a bitstream. The multiplexer 124 then inputs the bitstream to the system layer.

FIG. 21 is a block diagram showing another example configuration of the 3D data decoder 213 according to this embodiment. In this example, the 3D data decoder 213 includes a vertex information generator 231, an attribute information generator 232, a video decoder 234, a metadata decoder 223, and a demultiplexer 224. The vertex information generator 231, the attribute information generator 232, and the video decoder 234 may correspond to the vertex information decoder 201 and the attribute information decoder 203 in FIG. 8, etc.

Also, in this example, the 3D data decoder 213 decodes the 3D data according to a video-based coding method. In decoding according to a video-based coding method, multiple 2D images are decoded according to a video coding method, and 3D data is generated from the multiple 2D images. Here, the video coding method may be HEVC (High Efficiency Video Coding) or VVC (Versatile Video Coding), etc.

Specifically, first, the bitstream is input from the system layer to the demultiplexer 224. The demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream. The compressed vertex information, compressed vertex information metadata, compressed attribute information, and compressed attribute information metadata are input to the video decoder 234. The compressed metadata is input to the metadata decoder 223.

The video decoder 234 decodes the vertex images in accordance with the video encoding method. In doing so, the video decoder 234 decodes the vertex images from the compressed vertex information using the metadata of the compressed vertex information. The video decoder 234 then inputs the vertex images to the vertex information generator 231. The video decoder 234 also decodes the attribute images in accordance with the video encoding method. In doing so, the video decoder 234 decodes the attribute images from the compressed attribute information using the metadata of the compressed attribute information. The video decoder 234 then inputs the attribute images to the attribute information generator 232.

The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 includes map information used to generate vertex information and attribute information. The metadata decoded by the metadata decoder 223 may also be used to decode vertex images and attribute images.

The vertex information generator 231 reproduces vertex information from the vertex image according to the map information contained in the metadata decoded by the metadata decoder 223. The attribute information generator 232 reproduces attribute information from the attribute image according to the map information contained in the metadata decoded by the metadata decoder 223.

Then, the vertex information, attribute information, and metadata are output from the 3D data decoder 213 as 3D data. Note that this metadata is, for example, metadata about the vertex information and attribute information, and can be used in an application program.

FIG. 22 is a conceptual diagram showing a specific example of the encoding process according to this embodiment. FIG. 22 shows a three-dimensional data encoder 113 and a description encoder 148. In this example, the three-dimensional data encoder 113 includes a two-dimensional data encoder 141 and a mesh data encoder 142. The two-dimensional data encoder 141 includes a texture encoder 143. The mesh data encoder 142 includes a vertex information encoder 144 and a connectivity information encoder 145.

The vertex information encoder 144, the connection information encoder 145, and the texture encoder 143 may correspond to the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 in Figure 6, etc.

For example, the two-dimensional data encoder 141 operates as a texture encoder 143, and generates a texture file by encoding the texture corresponding to the attribute information as two-dimensional data according to an image encoding method or a video encoding method.

The mesh data encoder 142 also operates as a vertex information encoder 144 and a connectivity information encoder 145, and generates a mesh file by encoding the vertex information and connectivity information. The mesh data encoder 142 may also encode mapping information for textures. The encoded mapping information may then be included in the mesh file.

Furthermore, the description encoder 148 generates a description file by encoding a description corresponding to metadata such as text data. The description encoder 148 may encode the description at the system layer. For example, the description encoder 148 may be included in the system multiplexer 114 of FIG. 12.

The above operations generate a bitstream containing texture files, mesh files, and description files. These files may be multiplexed into the bitstream in file formats such as glTF (Graphics Language Transmission Format) or USD (Universal Scene Description).

The three-dimensional data encoder 113 may include two mesh data encoders as the mesh data encoder 142. For example, one mesh data encoder encodes vertex information and connectivity information of a static three-dimensional mesh, and the other mesh data encoder encodes vertex information and connectivity information of a dynamic three-dimensional mesh.

In response to this, two mesh files may be included in the bitstream. For example, one mesh file corresponds to a static 3D mesh, and the other mesh file corresponds to a dynamic 3D mesh.

Furthermore, the static three-dimensional mesh may be a three-dimensional mesh of an intraframe that is coded using intraprediction, and the dynamic three-dimensional mesh may be a three-dimensional mesh of an interframe that is coded using interprediction. Furthermore, differential information between the vertex information or connectivity information of the three-dimensional mesh of an intraframe and the vertex information or connectivity information of the three-dimensional mesh of an interframe may be used as information on the dynamic three-dimensional mesh.

FIG. 23 is a conceptual diagram showing a specific example of the decoding process according to this embodiment. FIG. 23 shows a three-dimensional data decoder 213, a description decoder 248, and a presenter 247. In this example, the three-dimensional data decoder 213 includes a two-dimensional data decoder 241, a mesh data decoder 242, and a mesh reconstructor 246. The two-dimensional data decoder 241 includes a texture decoder 243. The mesh data decoder 242 includes a vertex information decoder 244 and a connectivity information decoder 245.

The vertex information decoder 244, the connection information decoder 245, the texture decoder 243, and the mesh reconstructor 246 may correspond to the vertex information decoder 201, the connection information decoder 202, the attribute information decoder 203, and the post-processor 205 in FIG. 8. The presenter 247 may correspond to the presenter 215 in FIG. 12.

For example, the two-dimensional data decoder 241 operates as a texture decoder 243, and decodes the texture corresponding to the attribute information from the texture file as two-dimensional data in accordance with an image encoding method or a video encoding method.

The mesh data decoder 242 also operates as a vertex information decoder 244 and a connectivity information decoder 245, decoding vertex information and connectivity information from the mesh file. The mesh data decoder 242 may also decode mapping information for textures from the mesh file.

Furthermore, the description decoder 248 decodes descriptions corresponding to metadata such as text data from the description file. The description decoder 248 may decode the descriptions at the system layer. For example, the description decoder 248 may be included in the system demultiplexer 214 of FIG. 12.

The mesh reconstructor 246 reconstructs a three-dimensional mesh from vertex information, connectivity information, and texture according to the description. The renderer 247 renders and outputs the three-dimensional mesh according to the description.

The above operation reconstructs and outputs a 3D mesh from a bitstream containing a texture file, mesh file, and description file.

The three-dimensional data decoder 213 may be equipped with two mesh data decoders as the mesh data decoder 242. For example, one mesh data decoder decodes vertex information and connectivity information of a static three-dimensional mesh, and the other mesh data decoder decodes vertex information and connectivity information of a dynamic three-dimensional mesh.

In response to this, two mesh files may be included in the bitstream. For example, one mesh file corresponds to a static 3D mesh, and the other mesh file corresponds to a dynamic 3D mesh.

Furthermore, the static three-dimensional mesh may be a three-dimensional mesh of an intraframe that is coded using intraprediction, and the dynamic three-dimensional mesh may be a three-dimensional mesh of an interframe that is coded using interprediction. Furthermore, differential information between the vertex information or connectivity information of the three-dimensional mesh of an intraframe and the vertex information or connectivity information of the three-dimensional mesh of an interframe may be used as information on the dynamic three-dimensional mesh.

The dynamic 3D mesh coding method is sometimes called DMC (Dynamic Mesh Coding). The video-based dynamic 3D mesh coding method is sometimes called V-DMC (Video-based Dynamic Mesh Coding).

The point cloud encoding method is sometimes called PCC (Point Cloud Compression). The video-based point cloud encoding method is sometimes called V-PCC (Video-based Point Cloud Compression). The geometry-based point cloud encoding method is sometimes called G-PCC (Geometry-based Point Cloud Compression).

<Implementation example>
Fig. 24 is a block diagram showing an example implementation of the encoding device 100 according to this embodiment. The encoding device 100 includes a circuit 151 and a memory 152. For example, multiple components of the encoding device 100 shown in Fig. 5 and the like are implemented by the circuit 151 and memory 152 shown in Fig. 24.

Circuit 151 is a circuit that performs information processing and is capable of accessing memory 152. For example, circuit 151 is a dedicated or general-purpose electrical circuit that encodes a three-dimensional mesh. Circuit 151 may be a processor such as a CPU. Circuit 151 may also be a collection of multiple electrical circuits.

Memory 152 is a dedicated or general-purpose memory that stores information used by circuit 151 to encode the three-dimensional mesh. Memory 152 may be an electrical circuit and may be connected to circuit 151. Memory 152 may also be included in circuit 151. Memory 152 may also be a collection of multiple electrical circuits. Memory 152 may also be a magnetic disk, an optical disk, etc., and may also be expressed as storage or recording medium, etc. Memory 152 may also be non-volatile memory or volatile memory.

For example, memory 152 may store a three-dimensional mesh or a bitstream. Memory 152 may also store a program that circuit 151 uses to encode the three-dimensional mesh.

Note that the encoding device 100 does not need to implement all of the multiple components shown in FIG. 5 and the like, and does not need to perform all of the multiple processes shown here. Some of the multiple components shown in FIG. 5 and the like may be included in another device, and some of the multiple processes shown here may be performed by another device. Furthermore, the encoding device 100 may implement any combination of the multiple components of the present disclosure, and may perform any combination of the multiple processes of the present disclosure.

FIG. 25 is a block diagram showing an example implementation of a decoding device 200 according to this embodiment. The decoding device 200 includes a circuit 251 and a memory 252. For example, multiple components of the decoding device 200 shown in FIG. 7 and other figures are implemented using the circuit 251 and memory 252 shown in FIG. 25.

Circuit 251 is a circuit that performs information processing and is capable of accessing memory 252. For example, circuit 251 is a dedicated or general-purpose electrical circuit that decodes three-dimensional meshes. Circuit 251 may be a processor such as a CPU. Circuit 251 may also be a collection of multiple electrical circuits.

Memory 252 is a dedicated or general-purpose memory that stores information used by circuit 251 to decode the three-dimensional mesh. Memory 252 may be an electrical circuit and may be connected to circuit 251. Memory 252 may also be included in circuit 251. Memory 252 may also be a collection of multiple electrical circuits. Memory 252 may also be a magnetic disk, an optical disk, etc., and may also be expressed as storage or recording medium, etc. Memory 252 may also be non-volatile memory or volatile memory.

For example, the memory 252 may store a three-dimensional mesh or a bitstream. The memory 252 may also store a program that allows the circuit 251 to decode the three-dimensional mesh.

Note that the decoding device 200 does not necessarily have to implement all of the multiple components shown in FIG. 7 and the like, and does not necessarily have to perform all of the multiple processes shown here. Some of the multiple components shown in FIG. 7 and the like may be included in another device, and some of the multiple processes shown here may be executed by another device. Furthermore, the decoding device 200 may implement any combination of the multiple components of the present disclosure, and may perform any combination of the multiple processes of the present disclosure.

The encoding method and decoding method, including the steps performed by each component of the encoding device 100 and decoding device 200 of the present disclosure, may be executed by any device or system. For example, part or all of the encoding method and decoding method may be executed by a computer equipped with a processor, memory, input/output circuitry, etc. In this case, the encoding method and decoding method may be executed by the computer executing a program for causing the computer to execute the encoding method and decoding method.

Furthermore, the program or bitstream may be recorded on a non-transitory computer-readable recording medium such as a CD-ROM.

An example of a program may be a bitstream. For example, a bitstream containing an encoded three-dimensional mesh includes syntax elements for causing the decoding device 200 to decode the three-dimensional mesh. The bitstream then causes the decoding device 200 to decode the three-dimensional mesh in accordance with the syntax elements included in the bitstream. Therefore, the bitstream may play a role similar to that of a program.

The bitstream may be an encoded bitstream containing an encoded 3D mesh, or a multiplexed bitstream containing an encoded 3D mesh and other information.

Furthermore, each component of the encoding device 100 and the decoding device 200 may be configured with dedicated hardware, general-purpose hardware that executes the above-mentioned programs, or a combination of these. Furthermore, the general-purpose hardware may be configured with a memory in which the program is recorded, and a general-purpose processor that reads and executes the program from the memory. Here, the memory may be a semiconductor memory or a hard disk, and the general-purpose processor may be a CPU, etc.

Furthermore, the dedicated hardware may be composed of a memory and a dedicated processor. For example, a dedicated processor may execute the encoding method and decoding method by referencing a memory for recording data.

Furthermore, as described above, each component of the encoding device 100 and the decoding device 200 may be an electrical circuit. These electrical circuits may form a single electrical circuit as a whole, or each may be a separate electrical circuit. Furthermore, these electrical circuits may correspond to dedicated hardware, or may correspond to general-purpose hardware that executes the above-mentioned programs, etc. Furthermore, the encoding device 100 and the decoding device 200 may be implemented as an integrated circuit.

Furthermore, the encoding device 100 may be a transmitting device that transmits a three-dimensional mesh. The decoding device 200 may be a receiving device that receives a three-dimensional mesh.

Displacement Encoding and Decoding
The following terms are used herein by way of example:

(1) Image An image is a data unit made up of a set of pixels, and includes a picture or a block smaller than a picture. Images include both moving images and still images.

(2) Picture A picture is a unit of image processing that is made up of a set of pixels, and is also called a frame or field.

(3) Block A block is a processing unit consisting of a specific number of pixels. The term shown in the following example is also used for a block. The shape of a block is not particularly limited. A block may be, for example, a rectangular shape of M×N pixels or a square shape of M×M pixels. A block may also be a triangular shape, a circular shape, or another shape. Examples of blocks are as follows:

Slice, tile, or brick CTU, superblock, or basic division unit VPDU, processing division unit for hardware CU, processing block unit, prediction block unit (PU), or orthogonal transform block unit (TU)
・Sub-block

(4) Pixel or Sample A pixel or sample is the smallest point of an image, in other words, the smallest unit. Pixels or samples include not only pixels at integer positions, but also pixels at sub-pixel positions generated based on pixels at integer positions.

(5) Pixel Value or Sample Value: A pixel value or sample value is a unique value of a pixel. The pixel value or sample value may include a luma value, a chroma value, or an RGB gradation level, and may also include a depth value or a binary value of 0 or 1.

(6) Flags A flag indicates one or more bits. A flag is, for example, a parameter or index represented by two or more bits. A flag may indicate not only a value represented by a binary number, but also a value represented by a number other than a binary number.

(7) Signal: A signal is something that is symbolized or coded to transmit information. A signal includes a discrete digital signal or a continuous analog signal.

(8) Stream or Bit Stream A stream or bit stream is a digital data sequence that indicates the flow of digital data. A stream or bit stream may be a single stream, or may be configured to include multiple streams with multiple layers. A stream or bit stream may be transmitted by serial communication using a single transmission path, or may be transmitted by packet communication using multiple transmission paths.

(9) Difference: For scalar quantities, difference can include simple difference (x - y) and difference calculations, such as absolute difference (|x - y|), squared difference (x^2 - y^2), square root difference (√(x - y)), weighted difference (ax - b, where a and b are constants), or offset difference (x - y + a, where a is an offset).

(10) Sum. For scalar quantities, sums can include simple sum (x + y) and addition operations. Sum can also include absolute sum (|x + y|), sum of squares (x^2 + y^2), square root of sum (√(x + y)), weighted sum (ax + by, where a and b are constants), or offset sum (x + y + a, where a is an offset).

（１１）「基于（based on）」
The phrase "based on something" means that something other than that "something" may be taken into consideration. Also, "based on" can be used both when a direct result is obtained and when a result is obtained through an intermediate result.

(12) "Used" or "Use"
The phrases "something was used" or "used something" mean that something other than the "something" may be taken into consideration. Also, the phrases "used" or "used" may be used in cases where a direct result is obtained, or in cases where a result is obtained via an intermediate result.

(13) Prohibition "Prohibit" can be rephrased as "not permitted." Also, "not prohibited/prohibited" or "permitted/permitted" does not necessarily mean "obligation."

(14) "Restriction" or "Limitation"
"Restriction" or "limitation" can be rephrased as "not permitted/not allowed" or "not permitted/allowed." Furthermore, "prohibited/not prohibited" or "not permitted/allowed" does not necessarily mean "obligation." Furthermore, what is quantitatively or qualitatively prohibited may be partial or total.

(15) Chroma
The term chroma is an adjective, represented by the symbols Cb or Cr, that indicates that a sample array or a single sample represents one of the two color-difference signals associated with a primary color. The term chroma is sometimes used instead of the term chrominance.

(16) Luma
The term luma is an adjective, denoted by the symbols or subscripts Y or L, that indicates that a sample array or a single sample represents a monochrome signal for a primary color. The term luma is sometimes used instead of the term luminance.

The encoding/decoding system of this embodiment will be described below.

A typical three-dimensional model (also known as a 3D model) digitally represents an object in a way that allows a user to explore the model using zoom, pan, and rotation in all three dimensions while it is rendered over time. One way to build such a representation is to construct a 3D mesh using triangles. The model stores the positions of the triangle vertices, their connectivity to each other, and their associated attributes (such as normals or UV patches).

Storing all this information in uncompressed form would require a very large amount of storage space and therefore a very large bandwidth for transmission. The triangles that form the mesh often have repeating patterns and similar properties, especially in temporal and spatial neighborhoods. These repetitions can be exploited to develop efficient encoding and decoding methods for storage and transmission. One such encoding and decoding method is Video-based Dynamic Mesh Coding (V-DMC).

FIG. 26 is a block diagram showing another example configuration of an encoding/decoding system according to this embodiment. As shown in FIG. 26, the encoding/decoding system includes an encoding device 100 and a decoding device 200.

The encoding/decoding system accepts a three-dimensional mesh (also called a 3D mesh) input in the form of three-dimensional coordinates of vertices (vertex information), connectivity (connectivity information), and associated attributes (attribute information). Note that the 3D mesh can also include texture maps as well as geometry.

The encoding device 100 takes in an input 3D mesh (also referred to as an input 3D mesh or input mesh) in the form of three-dimensional coordinates of vertices, connectivity, and associated attributes. The encoding device 100 encodes all associated information into a stream. The stream may consist of a single bitstream or multiple bitstreams.

Network 300 transmits the stream generated by the encoding device to decoding device 200. Network 300 may be the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), or any combination of these. Furthermore, network 300 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting or satellite broadcasting. Furthermore, instead of network 300, a recording medium such as a DVD (Digital Versatile Disc) or BD (Blu-Ray Disc) on which a stream is recorded may be used.

The stream is transmitted to the decoding device 200 via the network 300. The decoding device 200 decodes the bitstream and generates a 3D mesh using the 3D coordinates, connectivity, and associated attributes of the decoded vertices. The decoding device 200 outputs the generated 3D mesh (also referred to as an output 3D mesh or output mesh).

FIG. 27 shows another example configuration of the encoding device 100.

As shown in FIG. 27, the encoding device 100 includes a preprocessor 1103 and a compressor 1106.

The encoding device 100 reads the input mesh 1101 and attribute map 1102 and passes them to the preprocessor 1103. The preprocessor 1103 processes the input mesh to extract a base mesh 1104 and displacement data 1105. The attribute map 1102, along with the extracted base mesh 1104 and displacement data 1105, is passed to the compressor 1106.

The compressor 1106 also compresses the base mesh 1104, displacement data 1105, and attribute map 1102 to generate a bitstream 1107. The compressor 1106 can transmit additional information to the decoding device 200 by further including metadata 1108 in the bitstream 1107.

Figure 28 shows another example configuration of the decoding device 200.

As shown in FIG. 28, the decoding device 200 includes a decompressor 2102 and a post-processor 2106.

The decoding device 200 reads the bitstream 2101 and passes it to the decompressor 2102. The decompressor 2102 decompresses the base mesh 2103, displacement data 2104, and attribute map 2108 from the bitstream 2101 and passes them to the post-processor 2106. An example of the displacement data 2104 is a displacement vector.

The post-processor 2106 also processes the base mesh 2103 according to the displacement data 2104 and the attribute map 2108 to generate the output mesh 2107. The post-processor 2106 may further use information from the metadata 2105 to generate the output mesh 2107.

FIG. 29 is a block diagram showing yet another example configuration of the encoding device 100 according to this embodiment.

In this example, the encoding device 100 includes a volumetric capturer 511, a projector 512, a base mesh encoder 513, a displacement encoder 514, an attribute encoder 515, and optionally one or more other type encoders 516.

The volumetric capturer 511 captures content and outputs the captured content to the projector 512.

The projector 512 projects the content onto a 3D mesh frame, which includes vertex geometry coordinates (vertex coordinates indicating the positions of the vertices), texture coordinates, and connectivity data (connection information). The data is output to a base mesh encoder 513, a displacement encoder 514, an attribute encoder 515, and optionally one or more other type encoders 516. Each encoder compresses the data into a bitstream.

Figure 30 is a block diagram showing yet another example configuration of a decoding device 200 according to this embodiment.

In this example, the decoding device 200 includes a base mesh decoder 613, a displacement decoder 614, an attribute decoder 615, one or more other type decoders 616, and a 3D reconstructor 617.

The bitstream is sent to a base mesh decoder 613, a displacement decoder 614, an attribute decoder 615, and optionally one or more other type decoders 616. These decoders decode the bitstream to generate decoded data including vertex geometry coordinates, texture coordinates, and connectivity data. The decoded data is then sent to a 3D reconstructor 617, which reconstructs a 3D mesh frame.

The encoding process performed by the encoding device 100 will be explained in detail below.

Figure 31 is a flow diagram showing the processing of the encoding device 100. Figure 32 is an explanatory diagram conceptually showing the encoding of mesh frames. The processing of the encoding device 100 will be explained with reference to Figures 31 and 32.

In step S101, the encoding device 100 reads a 3D mesh frame, which is an input mesh frame, and its attributes. The input mesh frame is a mesh frame input to the encoding device 100. An example of a 3D mesh frame that is an input mesh frame is shown as mesh frame 1301 (see Figure 32).

In step S102, the encoding device 100 performs a decimation process on the input mesh frame read in step S101 to generate a base mesh frame with fewer vertices than the input mesh frame. The base mesh frame generated by decimating mesh frame 1301 is shown as base mesh frame 1302 (see Figure 32).

In step S103, the encoding device 100 calculates displacement information that the decoding device 200 uses to reconstruct a mesh frame. The displacement information corresponds to a displacement vector pointing from a vertex of the base mesh frame generated in step S102 to a vertex of the input mesh frame. One method for calculating the displacement information is to subtract the coordinates of the vertex of the base mesh frame from the coordinates of the vertex of the input mesh frame. The displacement information calculated from the mesh frame 1301 and the base mesh frame 1302 is shown as displacement information 1303 (see Figure 32). The displacement information 1303 is in vector format, in other words, it is expressed as a displacement vector.

In step S104, the encoding device 100 encodes the base mesh frame generated in step S102, the displacement information generated in step S103, and the attributes of the input mesh frame into a bitstream (equivalent to a compressed bitstream). An example of the bitstream is shown as bitstream 1304 (see Figure 32).

Specifically, bitstream 1304 includes vertex coordinates and connectivity information for vertices A, C, E, and F, displacement information, a video bitstream including texture data, and a compressed attribute map (see FIG. 32). The displacement information includes displacement information for displacing vertices based on vertex coordinates obtained from the subdivided base mesh frame. The compressed attribute map is texture coordinates for applying texture data to a mesh frame reconstructed using the base mesh frame and displacement information.

The decoding process performed by the decoding device 200 will be explained in detail below.

Figure 33 is a flow diagram showing the processing of the decoding device 200. Figure 34 is an explanatory diagram conceptually showing the decoding of a 3D mesh. The processing of the decoding device 200 will be explained with reference to Figures 33 and 34.

In step S201, the decoding device 200 decodes the base mesh frame and attributes from the bitstream (equivalent to the compressed bitstream). An example of the decoded base mesh frame (equivalent to the decoded base mesh frame) is shown as decoded base mesh frame 2301 (see Figure 34).

In step S202, the decoding device 200 generates subdivided vertices by performing a subdivision process on the base mesh frame decoded in step S201. An example of a base mesh frame including subdivided vertices is shown as base mesh frame 2302 (see Figure 34).

In step S203, the decoding device 200 decodes the displacement information from the bitstream (equivalent to the compressed bitstream). An example of the decoded displacement information is shown as displacement information 2303 (see Figure 34). The displacement information 2303 is in vector format, in other words, expressed as a displacement vector.

In step S204, the decoding device 200 reconstructs the shape of the mesh frame by moving the vertices of the base mesh frame, including the subdivided vertices, to new positions using the displacement information, and then restores the mesh frame by applying attribute information. An example of an attribute is texture. An example of a reconstructed mesh frame is shown as mesh frame 2304 (see Figure 34).

FIG. 35 is a block diagram showing an example configuration of a decoding device according to this embodiment.

Figure 35 shows an example of a block diagram for general intra-decoding.

The decoding device shown in FIG. 35 includes a demultiplexer 1231, a switch 1232, a static mesh decoder 1233, a mesh buffer 1234, a motion decoder 1235, a base mesh reconstructor 1236, an inverse quantizer 1237, a video decoder 1238, an image unpacker 1239, an inverse quantizer 1240, an inverse wavelet transformer 1241, a reconstructor 1242, a video decoder 1243, and a color transformer 1244.

The demultiplexer 1231 takes the compressed bitstream and separates it into compressed data for the base mesh, video containing displacement data (also called a displacement bitstream), and video containing attribute data (also called an attribute bitstream). The compressed data for the base mesh is passed to the switch 1232, which determines whether to perform intra-decoding or inter-decoding based on parameters in the bitstream.

If intra-decoding is selected, the bitstream is passed to a static mesh decoder 1233, which generates a quantized base mesh. The static mesh decoder 1233 is, for example, a decoder that uses an edge breaker algorithm to decode 3D mesh data. The static mesh decoder 1233 generates a quantized base mesh from the bitstream. The quantized base mesh generated by the static mesh decoder 1233 is stored in a mesh buffer 1234 for reference when an inter-decoding process is selected.

If inter-decoding is selected, switch 1232 passes compressed data for the base mesh to motion decoder 1235. Motion decoder 1235 receives a previously decoded quantized base mesh and decodes motion data representing the difference in vertex coordinates between the quantized base mesh stored in mesh buffer 1234 and the current quantized base mesh. The motion data and the quantized base mesh stored in mesh buffer 1234 are used by base mesh reconstructor 1236 to reconstruct the current quantized base mesh. The quantized base mesh resulting from either inter-decoding or intra-decoding is passed to inverse quantizer 1237 to obtain a decoded base mesh.

The video containing the displacement data is passed to the video decoder 1238, as the bitstream contains the displacement data in an image format with two chroma pieces of information and one luma piece of information. The video decoder 1238 decodes the data using a video frame decompression method. Alternatively, the displacement data can be decoded using an arithmetic decoder. This decompressed data is passed to the image unpacker 1239, which extracts wavelet coefficients associated with each vertex from the image-format decompressed data. The inverse quantizer 1240 dequantizes the quantized wavelet coefficients into the three components associated with each vertex. The inverse wavelet transformer 1241 inversely transforms the result to finally obtain the decoded displacement data. The decoded displacement data and the decoded base mesh are passed to the reconstructor 1242, which performs edge refinement on the decoded base mesh and displaces the vertices using the decoded displacement data to obtain the decoded mesh.

The video containing the attribute data is passed to another video decoder 1243 to obtain a decoded attribute bitstream. The decoded attribute bitstream is further processed by a color converter 1244 for color space and color format conversion to obtain a decoded attribute map.

FIG. 36 is a block diagram showing an example configuration of a decoding device according to this embodiment.

Figure 36 shows an example of a reconstructor that obtains a decoded 3D mesh 1256 from a decoded base mesh 1251 and decoded displacement data 1254.

The decoded base mesh 1251 is passed to the subdivision unit 1252.

The subdivision unit 1252 subdivides any two connected vertices across the 3D mesh by adding a new vertex between them. This process can be repeated several times, including vertices created in previous subdivision steps, to generate a predefined number of vertices. Each subdivision iteration across the 3D mesh generates a new level of detail (LoD). The subdivided mesh 1253 and the decoded displacement data 1254 are passed to the displacer 1255, which generates the decoded 3D mesh 1256 by moving each vertex to a new position according to the corresponding displacement data.

Subdivision will be described below. Subdivision is performed by a subdivider (specifically, subdivider 1206 or subdivider 2204).

Figure 37 is an explanatory diagram showing an example of subdivision.

The base mesh shown in Figure 37(a) includes vertices A, B, and C, along with connectivity information indicating their connectivity.

(b) in Figure 37 shows the mesh generated by the first subdivision, in other words, the mesh after the first subdivision. In the first subdivision, the subdivider generates vertices D, E, and F and connectivity information indicating their connectivity. The mesh generated by the subdivider is also called LoD1 or first LoD.

Vertex D of the mesh after the first subdivision is a vertex generated by subdivision based on vertices A and B. Similarly, vertex E is a vertex generated by subdivision based on vertices B and C. Vertex F is a vertex generated by subdivision based on vertices A and C.

As an example, vertex D may be the midpoint of the line segment AB (in other words, side AB) connecting the vertices A and B that were the basis for its creation. Similarly, vertex E may be the midpoint of the line segment AC. Vertex F may be the midpoint of the line segment BC.

(c) of Figure 37 shows the mesh generated by the second subdivision, in other words, the mesh after the second subdivision. In the second subdivision, the subdivider generates vertices G, H, I, J, K, L, M, N, and O, and connectivity information indicating their connectivity. The mesh generated by the subdivider is also called LoD2 or second LoD.

Vertex G of the mesh after the second subdivision is a vertex generated by subdivision based on vertices A and D. Similarly, vertex H is a vertex generated by subdivision based on vertices A and E. Vertex I is a vertex generated by subdivision based on vertices B and D. Vertex J is a vertex generated by subdivision based on vertices D and F. Vertex K is a vertex generated by subdivision based on vertices E and F. Vertex L is a vertex generated by subdivision based on vertices C and E. Vertex M is a vertex generated by subdivision based on vertices B and F. Vertex N is a vertex generated by subdivision based on vertices C and F. Vertex O is a vertex generated by subdivision based on vertices D and E.

As an example, vertex G may be the midpoint of line segment AD (in other words, side AD) connecting vertices A and D, which were the basis for its creation. Similarly, vertex H may be the midpoint of line segment AE. vertex I may be the midpoint of line segment BD. vertex J may be the midpoint of line segment DF. vertex K may be the midpoint of line segment EF. vertex L may be the midpoint of line segment CE. vertex M may be the midpoint of line segment BF. vertex N may be the midpoint of line segment CF. vertex O may be the midpoint of line segment DE.

The following describes vertex displacement with reference to Figures 38 and 39. Vertex displacement is performed by the reconstructor 2209.

Figure 38 is an explanatory diagram showing an example of vertex displacement after subdivision. Figure 39 is an explanatory diagram showing an example of vertices of the original mesh.

The base mesh shown in Figure 38(a) includes vertices A, B, C, and Z, along with connectivity information indicating their connectivity.

(b) in Figure 38 shows the mesh generated by the first subdivision, in other words, the mesh after the first subdivision (i.e., the first LoD). In the first subdivision, the subdivider generates vertices S, T, U, X, or Y and connectivity information indicating their connectivity. Vertices S, T, U, X, or Y are similar to vertices D, E, and F shown in (b) in Figure 37.

(c) of Figure 38 shows the mesh generated by the second subdivision, in other words, the mesh after the second subdivision (i.e., the second LoD). In the second subdivision, the subdivider generates vertices D, E, F, G, and H and connectivity information indicating their connectivity. Vertices D, E, F, G, and H are the same as vertices G, H, I, J, K, L, M, N, or O shown in (c) of Figure 37.

(d) in Figure 38 shows a mesh including the vertices after they have been displaced following subdivision. Vertices A, B, C, D, E, F, G, H, S, T, U, X, Y, and Z shown in (d) in Figure 38 are each located at positions displaced using displacement information from the positions of the vertices shown in (c) in Figure 38.

The original mesh shown in Figure 39 is an example of the mesh input to the encoding device 100, i.e., the mesh before encoding.

The mesh shown in Figure 38 has a shape similar to the original mesh shown in Figure 39. The displacement information is generated by the displacement vector calculator 1207 of the encoding device 100 as information indicating the displacement from the vertices of the base mesh to the vertices of the original mesh, so by reconstructing the mesh using the displacement information generated in this way, a mesh having a shape similar to the original mesh is generated.

The decoding device 200 can output the mesh shown in Figure 38 (d).

Next, we will explain how to divide a mesh into sub-meshes with reference to Figures 40 and 41.

A mesh can be divided into multiple smaller parts and each part can be coded. When dividing a mesh, the vertices of the mesh are divided in such a way that the coordinates and connectivity of the vertices in each part can be coded independently.

Figure 40 is an explanatory diagram showing an example of a mesh. Figure 41 is an explanatory diagram showing an example of dividing a mesh into sub-meshes.

The mesh shown in Figure 40 is the original mesh, and is sometimes called a full mesh in contrast to a submesh.

Figure 41 shows how the full mesh shown in Figure 40 is divided into two sub-meshes. For vertices A, B, and C of the full mesh (see Figure 40), vertex A is duplicated to vertex A1 and vertex A2, vertex B is duplicated to vertex B1 and vertex B2, and vertex C is duplicated to vertex C1 and vertex C2, thereby creating two sub-meshes (i.e., the first sub-mesh and the second sub-mesh). The first sub-mesh and the second sub-mesh are meshes that can be decoded independently.

The following describes packing of displacement information into image frames with reference to Figures 42, 43, and 44.

Figures 42, 43, and 44 are explanatory diagrams showing examples of packing displacement information into image frames. Note that image frames can also be referred to as video frames.

Vertex displacement data is encoded as image frame data by, for example, mapping it to each component of a YUV format image frame (i.e., each of the Y component (Y Plane), U component (U Plane), and V component (V Plane)). This case is explained below as an example. As another example, vertex displacement data may be encoded as image frame data by mapping it to each component of an RGB format image frame (each of the R component, G component, and B component).

The decoding device 200 can use an image encoding module to extract displacement data. The displacement data can be in the form of an X, Y, or Z component in a global coordinate system (e.g., a Cartesian coordinate system), or a normal, tangential, or both tangential components in a local coordinate system. Methods for mapping displacement data to an image frame include the following:

For example, in the first method, the displacement data is arranged in the image frame in scan order. An example of packing the displacement data in this case is shown in Figure 42. The displacement data is directly mapped to the image frame according to a predefined scan order.

Note that because image frames have fixed height and width, the displacement data may not fit perfectly within the frame. In such cases, the remaining part of the image frame is padded with padding data (see Figure 42).

For example, in the second method, displacement data is separated into multiple LoDs and mapped to the Y, U, and V components of the image frame. An example of packing of displacement data in this case is shown in Figure 43. Here, the displacement data for the image frame of the next LoD starts immediately after the displacement data for the previous LoD ends. As with the first method, if the displacement data does not fit exactly into the image frame, padding is added to the end of the image frame (see Figure 43).

For example, in the third method, displacement data corresponding to the LoD is mapped to the Y, U, and V components of the image frame in a manner different from that of the second method. An example of packing of displacement data in this case is shown in Figure 44. In this way, each LoD can be decoded independently. In the third method, middle padding is performed on the displacement data of each LoD, and CTU alignment is performed along with padding at the end of the video frame (see Figure 44).

<Other examples>
Although the aspects of the encoding device and the decoding device have been described above according to the embodiments, the aspects of the encoding device and the decoding device are not limited to the embodiments. Modifications that a person skilled in the art can conceive of may be applied to the embodiments, and multiple components according to the embodiments may be combined in any manner.

For example, in an embodiment, a process performed by a specific component may be performed by another component instead of the specific component. Furthermore, the order of multiple processes may be changed, or multiple processes may be performed in parallel.

Furthermore, as described above, at least some of the configurations of the present disclosure may be implemented as an integrated circuit. At least some of the processes of the present disclosure may be used as an encoding method or a decoding method. A program for causing a computer to execute the encoding method or the decoding method may be used. A non-transitory computer-readable recording medium on which the program is recorded may also be used. A bitstream for causing a decoding device to perform the decoding process may also be used.

Furthermore, at least some of the configurations and processes of the present disclosure may be used as a transmitting device, a receiving device, a transmitting method, and a receiving method. A program for causing a computer to execute the transmitting method or the receiving method may also be used. Furthermore, a non-transitory computer-readable recording medium on which the program is recorded may also be used.

In the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program appropriate for that component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that realizes the encoding device, etc., of the above embodiments is a program such as the following:

In other words, the software is a program that causes a computer to execute an encoding method that acquires a first frame to be encoded, encodes first data having one or more layers contained in the first frame by referencing second data having one or more layers contained in a second frame, and, when encoding the first data, determines one of a plurality of processes for each of the one or more layers contained in the first data using a value indicating that layer and a value related to the one or more layers contained in the second data, and executes the determined one process on the first data of that layer.

Furthermore, the software that realizes the decoding device and the like in the above embodiment is the following program.

In other words, the software is a program that causes a computer to execute a decoding method that acquires a first frame to be decoded, decodes first data having one or more layers contained in the first frame by referencing second data having one or more layers contained in a second frame, and, when decoding the first data, determines one of a plurality of processes for each of the one or more layers contained in the first data using a value indicating that layer and a value related to the one or more layers contained in the second data, and executes the determined one process on the first data of that layer.

Next, examples of attribute information in mesh data will be described. Here, "device" refers to an encoding device or a decoding device. Figures 45 to 47 are diagrams for explaining examples of attribute information in mesh data according to an embodiment.

As shown in Figure 45, multiple three-dimensional points G (vertices) in mesh data are projected onto a certain plane X, and the two-dimensional coordinates on plane X after the projection are sometimes set as attribute information for these three-dimensional points. For example, if the two-dimensional coordinates after projection of a three-dimensional point (x, y, z) onto plane X are (u, v), the two-dimensional coordinates (u, v) are set as attribute information for the three-dimensional point (x, y, z). Figure 45 shows an example in which a set 1501 of vertex coordinates of a three-dimensional mesh represented by mesh data is projected onto plane X and converted into a set 1502 of two-dimensional coordinates on plane X.

Also, as shown in Figure 46, multiple three-dimensional points within mesh data are projected onto plane X, and the plane information to which the two-dimensional coordinates on plane X after the projection belong is sometimes set as the attribute information of the three-dimensional point. For example, if the two-dimensional coordinates (u, v) after the projection of a three-dimensional point (x, y, z) onto plane X belong to plane information face0, then plane information face0 is set as the attribute information of the three-dimensional point (x, y, z). In this case, by setting (u, v) = (face0, 0), the three-dimensional point (x, y, z) and plane information face0 are associated as UV coordinates. This method allows the relationship between the three-dimensional point (x, y, z) and plane information face0 to be defined using the same mechanism as UV coordinates.

Furthermore, when projecting connected 3D points onto a 2D image, attribute information can be added to the 3D points by setting the same plane information. For example, the following processing can be performed:

In the case of Figure 46, face0 is set as attribute information for 3D points that have connectivity between 3D points G and G. Similarly, face1 is set as attribute information for 3D points that have connectivity between 3D points G' and G'. Using this method, the decoding device can obtain the set plane information by decoding the attribute information of the 3D points. Then, using this plane information, it can obtain, for example, related projection information.

Furthermore, as shown in FIG. 47, a three-dimensional point in the mesh data, or a normal vector corresponding to the plane to which that three-dimensional point belongs, may be set as attribute information. For example, the device can set the normal vector (nx, ny, nz) for a three-dimensional point (x, y, z) as attribute information. It is also possible to set a three-dimensional point G or a normal vector (nx, ny, nz) corresponding to the plane to which that three-dimensional point G belongs as attribute information for G.

Note that the attribute information in mesh data is not limited to this. Attribute information can include any information, such as color information of a three-dimensional point, reflectance, or the group ID to which the three-dimensional point belongs. Attribute information may also include information with various dimensionalities, such as two-dimensional information such as UV coordinates, one-dimensional information such as plane information face0, and three-dimensional information such as normal vectors.

Furthermore, it is possible to assign multiple pieces of attribute information to a single 3D point. For example, the device can assign multiple pieces of attribute information, such as UV coordinates and a normal vector, to a 3D point (x, y, z). Also, there may be 3D points to which no attribute information has been assigned. In this way, by assigning various types of attribute information to 3D points, it is possible to generate mesh data with greater expressiveness.

In this embodiment, an example of a method for encoding or decoding attribute information by a device is shown. The example shown here describes the case of encoding or decoding one piece of attribute information assigned to a three-dimensional point, but is not limited to this. For example, if two or more pieces of attribute information are assigned to a three-dimensional point, it is also possible to use this embodiment to encode or decode each piece of attribute information individually. This makes it possible to efficiently encode or decode the attribute information even for three-dimensional points to which multiple pieces of attribute information are assigned.

In this embodiment, an example is shown in which UV coordinates are used as attribute information, but other attribute information can also be used. Figures 48 to 53 are diagrams for explaining specific examples of methods for encoding attribute information.

First, as shown in Figure 48, the device selects a first initial 3D point v0 and encodes its uv coordinates without prediction. For example, in the following case, the device may encode the value of (u0, v0) as is. Note that the order in which the 3D points for encoding attribute information are selected is arbitrary; for example, the device may select them in the same order as when encoding the position information of the 3D points. This allows the device to apply predictive encoding while reducing the amount of processing required to select 3D points.

Next, as shown in Figure 49, the device selects 3D point v1. If 3D point v1 has connectivity with an already-encoded 3D point v, i.e., if they have an edge, the device can use the UV coordinates of 3D point v0 as predicted values and differentially encode the UV coordinates of 3D point v1. For example, in the following case, the device can encode (u1 - u0, v1 - v0). This makes it possible to reduce the amount of coding when the UV coordinate values of 3D point v0 and 3D point v1 are close. Note that this method of predictive encoding using the UV coordinates of a single connected 3D point is called "coarse prediction."

Next, as shown in Figure 50, the device selects 3D point v2. If 3D point v2 has connectivity with already-encoded 3D points v0 and v1, i.e., if 3D point v2 has edges with 3D point v1 and v2 with 3D point v0, the device can generate a predicted value using the UV coordinates of 3D points v0 and v1, and differentially encode the UV coordinates of 3D point v2. For example, in the following case, the device can encode (u2 - predx, v2 - predy). This allows for highly accurate generation of a predicted value using 3D points v0 and v1, thereby reducing the amount of coding. Note that this method of predictive encoding using the UV coordinates of two or more connected 3D points is called "fine prediction."

Next, as shown in Figure 51, the device selects 3D point v3. If 3D point v3 has connectivity with already-encoded 3D points v1 and v2, i.e., if 3D point v3 has edges with 3D point v1 and 3D point v3 with 3D point v2, the device can generate predicted values using fine prediction using the UV coordinates of 3D points v1 and v2, and differentially encode the UV coordinates of 3D point v3.

Next, as shown in FIG. 52, the device selects 3D point v4. If 3D point v4 has connectivity with already-encoded 3D points v0, v1, and v3, i.e., if 3D point v4 and 3D point v3, and if 3D point v4 and 3D point v1 and 3D point v4 and 3D point v3 have edges, the device can generate a predicted value by fine prediction using the UV coordinates of 3D points v0, v1, and v3, and differentially encode the UV coordinates of 3D point v4. In this case, for example, the device may calculate fine prediction value 1 using the UV coordinates of 3D points v0 and v1, calculate fine prediction value 2 using the UV coordinates of 3D points v1 and v3, and use the average of fine prediction value 1 and fine prediction value 2 as the predicted value for the UV coordinate of 3D point v4. In this way, when two or more fine prediction values can be calculated, using their average as the predicted value can improve prediction accuracy and reduce the amount of coding.

Next, as shown in Figure 53, the device selects 3D point v5. If 3D point v5 has no connectivity with any previously encoded 3D points, the device selects 3D point v5 as the second initial 3D point and can encode the value of (u5, v5) directly without prediction, as in the first process. Similarly, the device can reduce the amount of coding for the uv coordinates by applying predictive coding using each process thereafter.

Next, we will explain examples of prediction modes for encoding attribute information.

First, we will explain the no-prediction mode. The no-prediction mode is an example of the first prediction mode. The no-prediction mode is applied when the 3D point to be coded is an initial 3D point. Here, the initial 3D point refers to a 3D point to be processed that has already been coded and for which there are no other coded 3D points that have connectivity with that 3D point.

In no-prediction mode, the device can encode the attribute information values as is. For example, if the uv coordinates are (100, 60), the device can encode the u value as 100 and the v value as 60. The device may also assume fixed-length encoding using n bits (n is an integer value greater than or equal to 0). For example, if the uv coordinates are (100, 60), the device can encode each component using fixed-length encoding using 7 bits.

If a device applies n-bit fixed-length coding, the device may add the value n to the header of the bitstream. This allows the decoding device to correctly decode the attribute information of the initial 3D point coded in n-bit fixed length by decoding the value n of the header.

The device may change the bit length used for fixed-length encoding for each component of the attribute information. For example, if the u and v coordinates are (100, 60), the device can reduce the amount of code by applying 7-bit fixed-length encoding to u and 6-bit fixed-length encoding to v. In this case, the device may add the bit lengths used for the fixed-length encoding of each component to the header of the bit stream. This allows the decoding device to correctly decode the attribute information of the initial 3D point that was encoded using different bit lengths for each component by decoding the bit length for each component in the header.

The device can set the bit length for each component to 0 bits. 0 bits may indicate that the component is not included in the bitstream. If the bit length added to the header is 0 bits, the decoding device can determine that the component is not included in the bitstream and estimate the value of the component to be 0. For example, if the attribute information includes a component whose value is always 0, the device can set the bit length of the fixed-length coding of that component to 0 bits and reduce the amount of code by not encoding the value 0. Furthermore, if the bit length added to the header is 0 bits, the decoding device can determine that the component is not included in the bitstream and estimate the value of the component to be 0.

For example, as explained using Figure 46, the device may project a three-dimensional point G(x, y, z) onto the xz plane, and use the plane face0 to which the two-dimensional coordinates (u, v) on the xy plane after projection belong as attribute information for G. In this case, the device can associate G with face0 as uv coordinates by setting (u, v) = (face0, 0). In this case, since v is always 0, the device can reduce the amount of code by setting the bit length of the fixed-length encoding of v to 0 bits and not including the value 0 for v in the bitstream.

The device may encode the attribute information of the initial three-dimensional points using variable-length coding such as unary coding or exponential golomb coding. The device can also reduce the amount of coding by assigning a context to each bit after binarization and applying arithmetic coding.

Note that it is not necessary to apply the no-prediction mode to the initial 3D points, and the device may apply other prediction methods. For example, the device may calculate a representative value for the initial 3D points and predictively encode the attribute information of the initial 3D points using that representative value as a predicted value. The representative value may be, for example, the average value of the initial 3D points. In this case, the device may add the representative value to the header, allowing the decoding device to decode the header, obtain the predicted value of the initial 3D points, and perform correct decoding. The device may also use the attribute information of the initial 3D points that was coded or decoded immediately before in the coding or decoding order as the predicted value of the attribute information of the initial 3D points. In this way, the device can reduce the amount of code by predictively encoding the attribute information of the initial 3D points.

Next, we will explain coarse prediction mode. Coarse prediction mode is an example of a second prediction mode. Coarse prediction mode is applied when, among the already-encoded 3D points, there is one 3D point that has connectivity with the 3D point to be encoded.

When applying coarse prediction, the device can use attribute information of a single 3D point that has already been coded and has connectivity as a predicted value, and encode the prediction residual. Note that, because the tendency of the prediction residual may differ from that in fine prediction, the device may use different methods for encoding or decoding the prediction residual in coarse prediction and fine prediction.

For example, a device may binarize the prediction residual using unary coding or exponential golomb coding, assign a context to each bit, and use different contexts for coarse prediction and fine prediction when applying arithmetic coding. This makes it possible to reduce the amount of coding even when the trends in the prediction residual values differ between coarse prediction and fine prediction. Furthermore, by using different binarization methods for coarse prediction and fine prediction, a device can further reduce the amount of coding.

Next, we will explain the fine prediction mode. The fine prediction mode is an example of the third prediction mode. The fine prediction mode is applied when, among the already-encoded 3D points, there are two or more 3D points that have connectivity with the 3D point to be encoded.

When applying fine prediction, the device can calculate a predicted value using attribute information of two or more 3D points that have already been coded and have connectivity, and encode the prediction residual. Note that when a 3D point with attribute information to be coded belongs to the vertices of two or more triangles made up of coded 3D points (such as 3D point v4 in Figure 52), the device may be able to generate two or more fine predicted values using two or more triangles. In such cases, the device may use the average value of two or more fine predicted values as the predicted value (average fine prediction).

For example, if the device is able to generate count fine predicted values, the device may calculate the sum of the count fine predicted values, divide this value by count to calculate an average fine predicted value, and use this average fine predicted value as the predicted value for the attribute information to be coded. This allows the device to improve the accuracy of the predicted values obtained by fine prediction and reduce the amount of coding.

The device may also limit the value of count to a power of 2. For example, by limiting the count of fine predictions used for average fine prediction to a power of 2, such as 1, 2, 4, or 8, the device can use a right bit shift instead of division when calculating the average value, thereby reducing the amount of calculations. For example, by applying a 1-bit right shift when count = 2 and a 2-bit right shift when count = 4, the average fine prediction value can be generated without using division.

In this case, average fine predictions using counts that are not powers of two may be prohibited, and the device may omit their calculation. This allows the device to reduce the amount of processing required to calculate the average fine prediction value. The device may also set a limit on the number of counts used to calculate the average fine prediction. For example, the device can set a fineMaxcount that indicates the maximum value of count, and limit the number of fine predictions used to calculate the average fine prediction value to no more than fineMaxcount. This allows the device to reduce the amount of processing required to calculate the average fine prediction.

The device may add fineMaxcount to the header of the bitstream. This allows the decoding device to decode the fineMaxcount in the header, calculate the average fine prediction value using the same maximum count value as the encoding device, and correctly decode the bitstream.

Furthermore, by setting fineMaxcount = 1, the device can limit the number of fine predictions used to calculate the average fine prediction value to 1, essentially eliminating the need to calculate the average value. This reduces the amount of processing required to calculate the average value and prevents decimal points from appearing when calculating the average value.

Furthermore, the device can set fineMaxcount = 0 so that fine prediction is not performed. In this case, the prediction mode used will be (1) no prediction mode or (2) coarse prediction mode. For example, if the attribute information of all 3D points following the initial 3D point is the same, the device can apply no prediction mode to the initial 3D point and apply coarse prediction to subsequent 3D points, which can keep the prediction residual at 0 and make it unnecessary to apply fine prediction. In such cases, the device can set fineMaxcount = 0 so that fine prediction is not applied, reducing the amount of processing required for encoding or decoding.

In addition, if additional information required for fine prediction exists, the device sets fineMaxcount = 0 and does not apply fine prediction, thereby reducing the amount of additional information and the amount of coding.

Note that the above predictive coding can also be applied to predictive decoding. In other words, by replacing "encoding" with "decoding" above, the above explanation of predictive coding can be used as an explanation of predictive decoding.

Figure 54 shows an example of a flowchart for encoding attribute information of three-dimensional points according to an embodiment.

The encoding device acquires multiple 3D points (vertices) of the 3D mesh (S1501).

The encoding device determines whether there are any 3D points for which attribute information has not been encoded (S1502). If there are any 3D points for which attribute information has not been encoded (Yes in S1502), the encoding device proceeds to step S1503; if not (No in S1502), the encoding device terminates processing.

The encoding device selects a 3D point to be encoded from among the multiple 3D points that have not yet been encoded (S1503).

The encoding device determines whether there are two or more encoded 3D points that have connectivity with the 3D point to be encoded (S1504). If there are two or more encoded 3D points that have connectivity with the 3D point to be encoded (Yes in S1504), the encoding device proceeds to step S1505; if not (No in S1504), the encoding device proceeds to step S1508.

The encoding device calculates a predicted value using attribute information of two or more three-dimensional points (S1505).

The encoding device determines whether count is greater than 0 (S1506). If count is greater than 0 (Yes in S1506), the encoding device proceeds to step S1507; if not (No in S1506), the encoding device proceeds to step S1508.

The encoding device encodes the attribute information of the 3D point to be encoded using the calculated predicted value (S1507). In other words, the encoding device calculates the residual between the attribute information of the 3D point to be encoded and a predicted value calculated using the attribute information of two or more 3D points, and encodes the calculated residual. Step S1507 is predictive encoding in the third prediction mode.

The encoding device determines whether there is one encoded 3D point that has connectivity with the 3D point to be encoded (S1508). If there is one encoded 3D point that has connectivity with the 3D point to be encoded (Yes in S1508), the encoding device proceeds to step S1509; if not (No in S1508), the encoding device proceeds to step S1510.

The encoding device encodes the attribute information of the 3D point to be encoded using the attribute information of one 3D point as a predicted value (S1509). In other words, the encoding device calculates the residual between the attribute information of the 3D point to be encoded and the attribute information of one 3D point, and encodes the calculated residual. Step S1509 is predictive encoding in the second prediction mode.

The encoding device encodes the attribute information of the 3D point to be encoded in a no-prediction mode (S1510). In other words, the encoding device encodes the attribute information of the 3D point to be encoded without using a predicted value. Step S1510 is predictive encoding in the first prediction mode.

FIG. 55 is a flowchart showing an example of a process for calculating a predicted value in the third prediction mode according to an embodiment.

The encoding device sets count to 0 (S1511).

The encoding device determines whether count is less than fineMaxcount (S1512). If count is less than fineMaxcount (Yes in S1512), the encoding device proceeds to step S1513; if not (No in S1512), the encoding device proceeds to step S1515. Note that the encoding device can turn off fine prediction by setting fineMaxcount=0. The encoding device may also add the set fineMaxcount value to the bitstream.

Furthermore, if the value of fineMaxcount decoded from the bitstream is 0, the decoder may decode the attribute information without applying fine prediction.

The encoding device calculates the count-th fine prediction value (S1513).

The encoding device adds 1 to count, updates it (S1514), and returns to step S1512.

The encoding device determines whether count is greater than 0 (S1515). If count is greater than 0 (Yes in S1515), the encoding device proceeds to step S1516; if not (No in S1515), the encoding device proceeds to step S1517.

The encoding device calculates the average fine prediction value (S1516). That is, the encoding device calculates the average of one or more fine prediction values as the average fine prediction value. Note that if there is one fine prediction value, the encoding device may calculate that one fine prediction value as the average fine prediction value. Note that the one or more fine prediction values are the count fine prediction values obtained in steps S1511 to S1514.

The encoding device outputs count (S1517).

Figure 56 shows an example of a flowchart for decoding attribute information of 3D points according to an embodiment.

The decoding device obtains multiple 3D points (vertices) of the 3D mesh (S1521).

The decoding device determines whether there are any 3D points for which attribute information has not been decoded (S1522). If there are any 3D points for which attribute information has not been decoded (Yes in S1522), the decoding device proceeds to step S1523; if not (No in S1522), the decoding device terminates processing.

The decoding device selects a 3D point to be decoded from among the multiple 3D points that have not yet been decoded (S1523).

The decoding device determines whether there are two or more decoded 3D points that have connectivity with the 3D point to be decoded (S1524). If there are two or more decoded 3D points that have connectivity with the 3D point to be decoded (Yes in S1524), the decoding device proceeds to step S1525; if not (No in S1524), the decoding device proceeds to step S1528.

The decoding device calculates a predicted value using attribute information for two or more three-dimensional points (S1525).

The decoding device determines whether count is greater than 0 (S1526). If count is greater than 0 (Yes in S1526), the decoding device proceeds to step S1527; if not (No in S1526), the decoding device proceeds to step S1528.

The decoding device decodes the attribute information of the 3D point to be decoded using the calculated predicted value (S1527). In other words, the decoding device calculates the residual between the attribute information of the 3D point to be decoded and a predicted value calculated using the attribute information of two or more 3D points, and decodes the calculated residual. Step S1527 is predictive decoding in the third prediction mode.

The decoding device determines whether there is one decoded 3D point that has connectivity with the 3D point to be decoded (S1528). If there is one decoded 3D point that has connectivity with the 3D point to be decoded (Yes in S1528), the decoding device proceeds to step S1529; if not (No in S1528), the decoding device proceeds to step S1530.

The decoding device decodes the attribute information of the 3D point to be decoded using the attribute information of one 3D point as a predicted value (S1529). In other words, the decoding device calculates the residual between the attribute information of the 3D point to be decoded and the attribute information of one 3D point, and decodes the calculated residual. Step S1529 is predictive decoding in the second prediction mode.

The decoding device decodes the attribute information of the 3D point to be decoded in a prediction-free mode (S1530). In other words, the decoding device decodes the attribute information of the 3D point to be decoded without using a predicted value. Step S1530 is predictive decoding in the first prediction mode.

Next, we will explain mesh_attribute_coding, which is a unit that encodes attribute information of three-dimensional points. Figure 57 is a diagram showing an example of the syntax of mesh_attribute_coding according to an embodiment. Figure 58 is a diagram showing an example of the syntax of mesh_attribute_header according to an embodiment.

mesh_attribute_coding indicates the unit for encoding attribute information of 3D points, mesh_attribute_header indicates header information related to the attribute information, and mesh_attribute_data indicates the encoded data related to the attribute information.

AttributeBitDepthMinus1 is information that indicates the bit precision of the attribute information encoded in the bitstream. The bit precision of the attribute information is calculated by adding 1 to AttributeBitDepthMinus1.

NumComponent is information that indicates the number of components that the attribute information has. For example, if the attribute information is UV coordinates, the value of NumComponent is 2, and if the attribute information is a normal vector, the value of NumComponent is 3.

AttributeStartDeltaBitDepth[j] is information that indicates the bit precision of the initial 3D point of the attribute information encoded in the bitstream. The bit precision of the jth component of the initial 3D point, AttributeStartBitDepth[j], is calculated as follows:

AttributeStartBitDepth[j] = AttributeBitDepthMinus1 + 1 - AttributeStartDeltaBitDepth[j]

For example, if AttributeBitDepthMinus1 is 9, AttributeStartDeltaBitDepth[0] is 0, and AttributeStartDeltaBitDepth[1] is 2:
AttributeStartBitDepth[0] = 9 + 1 - 0 = 10
AttributeStartBitDepth[1] = 9 + 1 - 2 = 8
It can be shown that the bit precision of the 0th component of the initial 3D point is 10-bit precision, and the bit precision of the 1st component is 8-bit precision.

In this way, by setting AttributeStartDeltaBitDepth for each component of the initial 3D point, it is possible to set different bit precisions for each component. Furthermore, by setting the bit precision of the initial 3D point separately from AttributeBitDepthMinus1, which is the bit precision of the attribute information encoded in the bitstream, if, for example, the range of values that each component of the initial 3D point can take is smaller than the range of values that each component of the attribute information encoded in the bitstream can take, the value of AttributeStartBitDepth may be smaller than the value obtained by adding 1 to AttributeBitDepthMinus1. In such cases, the amount of code can be reduced by encoding AttributeStart, the value of each component of the initial 3D point (described below), using the value of AttributeStartBitDepth rather than the value obtained by adding 1 to AttributeBitDepthMinus1.

Furthermore, in this embodiment, the difference between the value obtained by adding 1 to the bit precision AttributeBitDepthMinus1 of the attribute information encoded in the bit stream and the bit precision of the jth component of the initial 3D point may be encoded as AttributeStartDeltaBitDepth. As a result, if the bit precision of the attribute information and the bit precision of the initial 3D point are the same, the difference value added to the bit stream will be 0, thereby reducing the amount of coding.

Furthermore, the value of AttributeStartBitDepth[j] may be set to 0 by setting AttributeStartDeltaBitDepth[j] to the same value as AttributeBitDepthMinus1 plus 1. In this case, the j-th component for which AttributeStartBitDepth[j] is 0 does not need to be coded as 0 in the bitstream. The decoder can infer that the j-th component for which AttributeStartBitDepth[j] is 0 is 0. This allows the amount of coding to be reduced by not adding components for which AttributeStartBitDepth[j] is 0 to the bitstream.

In this embodiment, an example has been shown in which the differential value AttributeStartDeltaBitDepth[j] is added to the bitstream, but this is not necessarily limited to this. For example, AttributeStartBitDepth[j] may be directly coded into the bitstream. This method can reduce the amount of processing required to calculate the differential value.

Furthermore, the bit length of AttributeStartDeltaBitDepth[j] can be calculated based on the value of AttributeBitDepthMinus1. Specifically, the bit length of AttributeStartDeltaBitDepth[j] can be calculated as ceil(log2(AttributeBitDepthMinus1 + 2)). Here, ceil(x) is a function that outputs the integer value obtained by rounding up the value x. In this way, by determining the bit length of AttributeStartDeltaBitDepth[j] according to the value of AttributeBitDepthMinus1, the amount of code can be reduced.

AttributePredMode is information indicating the method for encoding the attribute information. For example, it may indicate a prediction method such as methodA, which includes fine prediction, coarse prediction, and no prediction mode, as shown in Figure 54. Note that the prediction method is not limited to methodA. For example, it may be a prediction method that combines two of the three prediction modes of methodA, or a prediction method that uses only one prediction mode. Specifically, a prediction method such as methodB, which includes coarse prediction and no prediction mode, is possible. By providing methodB as the prediction method, predicted values can be generated without applying division or arithmetic processing that requires decimal point precision, which reduces the amount of processing and enables lossless encoding.

fineMaxcount is information indicating the maximum number of fine predictions to use when calculating the average fine prediction value. If AttributePredMode includes fine prediction, for example, if AttributePredMode is methodA, fineMaxcount may be added to the bitstream.

Furthermore, by setting fineMaxcount to 1, the number of fine predictions used to calculate the average fine prediction value can be limited to 1, effectively eliminating the need to calculate the average value. This reduces the amount of processing required to calculate the average value. It also prevents decimal points from being generated when calculating the average value.

Fine prediction may also be prevented from being applied by setting fineMaxcount to 0. In this case, coarse prediction or no prediction mode will be used as the prediction mode. For example, if the attribute information of all 3D points following the initial 3D point is the same, the prediction residual can be set to 0 by applying no prediction mode to the initial 3D point and coarse prediction to subsequent 3D points, eliminating the need to apply fine prediction. In such cases, setting fineMaxcount to 0 prevents the application of fine prediction, thereby reducing the amount of processing required for encoding or decoding.

Furthermore, even if there is additional information required for fine prediction, setting fineMaxcount to 0 prevents fine prediction from being applied, thereby reducing the amount of additional information and the amount of coding.

AttributeBitDepthMinus1, AttributePredMode, and fineMaxcount may be binarized using Exponential Golomb coding or unary coding, and then variable-length coded. Coding efficiency can also be improved by assigning a context to each bit and coding while updating the probability of occurrence based on the frequency of occurrence of 0 and 1. This information may also be coded at a fixed length, which reduces the amount of processing.

FineMaxcount may also be set to a different value per sequence, per frame, per basemesh, per submesh, or per 3D point. This allows the amount of processing to be reduced by controlling the number used for the average fine prediction in appropriate units.

Figure 59 shows an example of mesh_attribute_data syntax related to an embodiment.

AttributeStartCount is information indicating the number of initial 3D points. AttributeStart[c][j] is information indicating the value of the jth component of the attribute information for the cth initial 3D point. For example, if the attribute information is UV coordinates, AttributeStart[c][0] may indicate the u value of the cth initial 3D point, and AttributeStart[c][1] may indicate the v value of the cth initial 3D point. AttributeStart[c][j] may also be fixed-length coded using the bit length calculated as AttributeStartBitDepth described in Figure 58. For example, if AttributeStartBitDepth[0] is 10 and AttributeStartBitDepth[1] is 8, this indicates that AttributeStart[c][0], the 0th component of the cth initial 3D point, is coded with 10-bit precision, and AttributeStart[c][1], the first component, is coded with 8-bit precision. The amount of coding can be reduced by coding the 0th component with 10 bits and the first component with 8 bits.

Furthermore, if the value of AttributeStartBitDepth[j] is 0, AttributeStart[c][j] does not need to be coded into the bitstream. By confirming that the value of AttributeStartBitDepth[j] is 0, the decoder can infer that the value of AttributeStart[c][j] is 0. In this way, by not including components with an AttributeStartBitDepth[j] value of 0 in the bitstream, the amount of coding can be reduced.

Furthermore, AttributeStart may apply variable-length coding such as unary coding or exponential golomb coding. It is also possible to reduce the amount of code by assigning a context to each bit after binarization and applying arithmetic coding.

Also, if the value of AttributeStartCount is 0, there is no need to encode AttributeStart. In this case, the decoder can confirm that AttributeStartCount is 0 and infer that the value of AttributeStart is 0. This method can reduce the amount of coding.

Furthermore, if it is guaranteed that there will always be at least one initial 3D point, AttributeStartCountMinus1 can be added to the bitstream instead of AttributeStartCount. In this case, the decoder can calculate AttributeStartCount by adding 1 to AttributeStartCountMinus1. This makes it possible to further reduce the amount of coding.

AttributeFineResiCount is information that indicates the number of prediction residuals of attribute information predictively coded using fine prediction. In particular, if the value of fineMaxcount is 0, it is expected that AttributeFineResiCount will be set to 0 as the most efficient case.

AttributeFineResiSize represents information indicating the code size of AttributeFineResi. For example, it may indicate the code size when AttributeFineResi is binarized and arithmetically coded. In this way, by notifying the decoder of the code size using AttributeFineResiSize before AttributeFineResi, the decoder can pre-read the bits related to AttributeFineResi from the bitstream. Specifically, while the decoder is performing arithmetic decoding on AttributeFineResi, it can perform decoding processing for the following AttributeCoarseResi in parallel, thereby reducing decoding time.

AttributeFineResi[c][j] is information indicating the prediction residual for the jth component of attribute information predictively coded by the cth fine prediction. AttributeFineResi[c][j] can be binarized using Exponential Golomb coding or unary coding, and then variable-length coded. In addition, coding efficiency can be improved by assigning a context to each bit and coding while updating the probability of occurrence based on the frequency of occurrence of 0s and 1s.

Furthermore, the context used for arithmetic coding of each bit may be shared with the context used for AttributeCoarseResi, which will be described later. This method allows the number of contexts to be reduced. On the other hand, the context used for arithmetic coding of each bit can also be set to be different from the context used for AttributeCoarseResi, which will be described later. By using different contexts in this way, coding efficiency can be improved even when the trends in prediction residuals differ between fine prediction and coarse prediction. Furthermore, using different contexts allows AttributeFineResi and AttributeCoarseResi to be decoded in parallel, further improving the efficiency of the decoding process.

AttributeCoarseResiCount is information that indicates the number of prediction residuals of attribute information predictively coded using coarse prediction. AttributeCoarseResiCount plays an important role in coding AttributeCoarseResi, which will be described later.

AttributeCoarseResiSize represents information indicating the code size of AttributeCoarseResi. For example, it may indicate the code size when AttributeCoarseResi is binarized and arithmetically coded. In this way, by informing the decoder of AttributeCoarseResiSize prior to coding AttributeCoarseResi, the decoder can pre-read the bits related to AttributeCoarseResi from the bitstream. Specifically, while the decoder is performing arithmetic decoding related to AttributeCoarseResi, it can perform decoding processing of the subsequent bitstream in parallel, thereby reducing decoding time.

AttributeCoarseResi[c][j] is information indicating the prediction residual for the jth component of attribute information predictively coded using the cth coarse prediction. AttributeCoarseResi[c][j] can be binarized using Exponential Golomb coding or unary coding, and then variable-length coded. Furthermore, coding efficiency can be improved by assigning a context to each bit and coding while updating the probability of occurrence based on the frequency of occurrence of 0s and 1s.

Furthermore, the context used for arithmetic coding of each bit may be shared with the context used by AttributeFineResi described above. This method allows the number of contexts to be reduced. On the other hand, the context used for arithmetic coding of each bit can also be set to be different from the context used by AttributeFineResi described above. By using different contexts in this way, coding efficiency can be improved even when the trends in prediction residuals differ between fine prediction and coarse prediction.

Furthermore, AttributeStartCount, AttributeFineResiCount, AttributeFineResiSize, AttributeFineResi, AttributeCoarseResiCount, AttributeCoarseResiSize, and AttributeCoarseResi may be binarized using Exponential Golomb coding or unary coding, and then variable-length coded. Coding efficiency can also be improved by assigning a context to each bit and coding while updating the probability of occurrence based on the frequency of occurrence of 0s and 1s. Furthermore, this information can also be coded at a fixed length, reducing the amount of processing.

In the description of this embodiment, examples relating mainly to the encoding or decoding of attribute information of three-dimensional points are shown, but this is not necessarily limited to this. For example, this embodiment may also be applied to the encoding or decoding of position information of three-dimensional points. An example of syntax when applied to position information of three-dimensional points will be described later.

In this embodiment, for example, as shown in FIG. 46, when the device projects a three-dimensional point G(x, y, z) onto the xz plane and the plane face0 to which the two-dimensional coordinates (u, v) on the xy plane after projection belong is set as attribute information for G, an example is shown in which G and face0 are associated as uv coordinates by setting (u, v) = (face0, 0), but this is not necessarily limited to this. For example, when encoding or decoding face0 as uv coordinates, the attribute information is uv coordinates, but by setting attribute information with NumComponent = 1, only u of the uv coordinates can be encoded. In this way, the amount of code can be reduced by encoding face0 as u and not encoding v.

Connected 3D points (connected components) can be projected onto a 2D image, the same plane information added as attribute information for the 3D points, and this attribute information can be encoded. This allows the decoder to obtain the plane information associated with each 3D point by decoding the attribute information of the 3D points, and this value can be used to obtain, for example, associated projection information. The method of generating UV coordinates for 3D points on the decoder side using this projection information can also be called an "ortho atlas."

When encoding plane information as UV coordinates using an ortho atlas, the attribute information of three-dimensional points with connectivity all have the same value (face0, 0), as shown in Figure 60. Figure 60 shows an example of attribute information related to a modified embodiment.

In this case, the device encodes the initial 3D points in no prediction mode, and applies coarse encoding to subsequent 3D points, thereby reducing the prediction residual to zero. Therefore, when using an ortho atlas, setting fineMaxcount=0 as described in this embodiment prevents the application of fine prediction, thereby reducing the amount of processing required for encoding or decoding. Furthermore, if additional information required for fine prediction exists, setting fineMaxcount=0 prevents the application of fine prediction, thereby reducing the amount of additional information and the amount of coding.

Furthermore, in the uv coordinates, v is always 0. Therefore, when using an ortho atlas, AttributeStartBitDepth for the v component of the initial 3D point can be set to 0 using AttributeBitDepthMinus1 and AttributeStartDeltaBitDepth described in this embodiment, and the bit length for the fixed-length encoding of the v component can be set to 0 bits, thereby reducing the amount of code by not encoding the v value of 0.

Furthermore, when using an ortho atlas, the prediction residual can be set to 0 by applying coarse prediction or fine prediction, so the amount of code can be reduced by not encoding at least one of the information about the prediction residual: AttributeFineResiCount, AttributeFineResiSize, AttributeFineResi, AttributeCoarseResiCount, AttributeCoarseResiSize, and AttributeCoarseResi. Information that is not encoded can be estimated to have a value of 0 on the decoder side.

Furthermore, if fine prediction is not applied when using an ortho atlas, the amount of coding can be reduced by not encoding the additional information related to fine prediction.

Figure 61 shows an example of mesh_attribute_data syntax for a modified embodiment.

If AttributeFineResiCount or AttributeCoarseResiCount is not coded, the number of 3D points that have connectivity with the initial 3D point, OtherAttributeCount, may be added to the bitstream in units of the number of initial 3D points. Figure 60 shows an example of this syntax. In this case, the decoder can correctly decode the UV coordinates by decoding the UV coordinates of the initial 3D point c and copying them to the UV coordinates of OtherAttributeCount[c] 3D points that have connectivity with the initial 3D point c.

Also, OtherAttributeCount may be binarized using Exponential Golomb coding or unary coding, and then variable-length coded. Coding efficiency can be improved by assigning a context to each bit and coding while updating the probability of occurrence based on the frequency of occurrence of 0 and 1. Alternatively, it may be coded at a fixed length, which reduces the amount of processing.

Furthermore, an AttributePredMode for ortho atlas may be defined, and when the encoder encodes attribute information using the method described above, it may add information to the bitstream indicating that the AttributePredMode is a prediction method for ortho atlas. This allows the decoder to determine whether the bitstream was encoded using the above method by decoding the AttributePredMode. If the determination result is Yes, the decoder can decode the attribute information using the above method.

In this embodiment, we have shown a method for encoding or decoding attribute information corresponding to three-dimensional points such as UV coordinates, but this is not necessarily limited to this. For example, the encoding or decoding method described in this embodiment may also be applied to attribute information corresponding to the surfaces of a polygon made up of three-dimensional points.

Specifically, face_id, which is face information corresponding to the faces of a polygon, can be encoded or decoded as attribute information with NumComponent=1. By doing so, this embodiment can be used to reduce the amount of code required for attribute information corresponding to three-dimensional points or attribute information corresponding to polygons made up of three-dimensional points.

FIG. 62 is a diagram showing an example of a geometry map according to an embodiment. FIG. 63 is a diagram showing an example of a texture map according to an embodiment.

Texture mapping is a common technique used to add visual appearance by projecting images onto the surfaces of a three-dimensional model. This technique allows for the visual details to be changed without directly modifying the model itself, thereby improving performance when rendering is required. The information stored in a two-dimensional texture image may include color, smoothness, transparency, etc. The relationship between the three-dimensional mesh and the two-dimensional texture image is established using a UV map. A UV map is a deformation of the three-dimensional mesh onto a two-dimensional plane. Attribute values from the image are then projected onto the three-dimensional model at rendering time by overlaying the texture image on the UV map. This necessitates encoding and decoding techniques that can efficiently store and stream UV maps.

Different prediction schemes are available for encoding texture coordinates in UV maps. These prediction schemes include those based on the parametrization techniques used when projecting a 3D mesh onto a 2D map, or those based on the relationship between the mesh's geometric data and texture coordinates. In the case of conformal parametrization, triangle edge lengths maintain approximately the same ratio in both 3D space and UV coordinates. Therefore, texture coordinates on a UV map can be predicted using the ratio obtained from the corresponding edges in 3D space.

As shown in Figure 62, texture coordinates A2, B2, and C2 correspond to A1, B1, and C1 on the geometry map, respectively. The ratio between edges A1B1 and A2B2 in 3D space is calculated as (A1C1)/(C1B1)=R. Finally, the coordinates of A2 in UV space are predicted as A2C2=R*C2B2 and A2=C2+A2C2.

As shown in Figure 63, the device can calculate a predicted value for the UV coordinate A2 when a three-dimensional point A1 is projected onto a two-dimensional image (texture map). The device may calculate this predicted value using C2 and B2 that have already been encoded or decoded, and the positional information of the three-dimensional points A1, B1, and C1 that correspond to A2, B2, and C2.

More specifically, the device assumes that triangles A1B1C1 and A2B2C2 are similar to each other, and can calculate A2C2 by multiplying C2B2, whose value has already been determined, by the similarity ratio R, calculated as (A1C1)/(C1B1) = R. The device can also predict the position of A2 using C2 + A2C2. As a result, if triangles made up of corresponding 3D points in the geometry map and texture map are similar to each other, the device can use the 3D point information in the encoded or decoded geometry map and the UV coordinates in the encoded or decoded texture map to highly accurately estimate the predicted value of the attribute information of the 3D point to be encoded or decoded, thereby reducing the amount of coding.

In this embodiment, the method in which the device calculates predicted values for attribute information of the three-dimensional point to be encoded or decoded using three-dimensional point information in an encoded or decoded geometry map and UV coordinates in an encoded or decoded texture map may also be referred to as fine prediction.

Next, we will explain mesh_position_coding, which is a unit that encodes the position information of three-dimensional points. Figure 64 is a diagram showing an example of the syntax of mesh_position_coding according to an embodiment. Figure 65 is a diagram showing an example of the syntax of mesh_position_header according to an embodiment.

mesh_position_coding is a unit for encoding the position information of three-dimensional points, and performs processing related to the encoding and decoding of position information. mesh_position_header indicates header information related to position information, and mesh_position_data indicates encoded data related to position information.

PositionBitDepthMinus1 is information that indicates the bit precision of the position information encoded in the bitstream. The bit precision of the position information can be calculated by adding 1 to PositionBitDepthMinus1.

NumComponent is information that indicates the number of components contained in the position information. For example, if the position information is xyz coordinates, the value of NumComponent can be set to 3.

PositionStartDeltaBitDepth[j] is information that indicates the bit precision of the initial 3D point of the position information encoded in the bitstream. The bit precision of the jth component of the initial 3D point, PositionStartBitDepth[j], can be calculated as follows:

PositionStartBitDepth[j] = This can be calculated by adding 1 to PositionBitDepthMinus1 and then subtracting PositionStartDeltaBitDepth[j]. For example, if PositionBitDepthMinus1 is 9, PositionStartDeltaBitDepth[0] is 0, PositionStartDeltaBitDepth[1] is 2, and PositionStartDeltaBitDepth[2] is 4, then the value of PositionStartBitDepth[0] is 9 + 1 - 0 = 10, the value of PositionStartBitDepth[1] is 9 + 1 - 2 = 8, and the value of PositionStartBitDepth[2] is 9 + 1 - 4 = 6. This indicates that the 0th component of the initial 3D point has 10-bit precision, the 1st component has 8-bit precision, and the second component has 6-bit precision.

In this way, by setting PositionStartDeltaBitDepth for each component of the initial 3D point, it is possible to set different bit precision for each component. Furthermore, by being able to set the bit precision of the initial 3D point separately from PositionBitDepthMinus1, which is the bit precision of the position information encoded in the bitstream, it is possible that the value of PositionStartBitDepth will be smaller than the value obtained by adding 1 to PositionBitDepthMinus1, for example, if the range of values that each component of the initial 3D point can take is smaller than the range that each component of the position information encoded in the bitstream can take.

In such cases, the amount of coding can be reduced by encoding PositionStart, the value of each component of the initial 3D point (described below), using the value of PositionStartBitDepth rather than the value obtained by adding 1 to PositionBitDepthMinus1. This setting also improves coding efficiency by avoiding the specification of unnecessary bit precision.

In this embodiment, the difference between the value obtained by adding 1 to PositionBitDepthMinus1, which is the bit precision of the position information encoded in the bit stream, and the bit precision of the jth component of the initial 3D point can be encoded as PositionStartDeltaBitDepth. With this method, if the value obtained by adding 1 to PositionBitDepthMinus1, which is the bit precision of the position information, is the same as the bit precision of the jth component of the initial 3D point, the difference value added to the bit stream will be 0, making it possible to reduce the amount of coding.

Furthermore, by setting PositionStartDeltaBitDepth[j] to the same value as PositionBitDepthMinus1 plus 1, the value of PositionStartBitDepth[j] for the jth component can be set to 0. In this case, the jth component whose PositionStartBitDepth[j] is 0 does not need to be coded as 0 in the bitstream. The decoder can infer that the jth component whose PositionStartBitDepth[j] is 0 has a value of 0. This allows the amount of coding to be reduced by not adding components whose PositionStartBitDepth[j] is 0 to the bitstream.

In this embodiment, a method is shown in which the differential value PositionStartDeltaBitDepth[j] is added to the bitstream, but this method is not necessarily limited to this. For example, a method can be used in which PositionStartBitDepth[j] is directly encoded into the bitstream. This method can reduce the amount of processing required to calculate the differential value.

Furthermore, the bit length of PositionStartDeltaBitDepth[j] can be determined based on the value of PositionBitDepthMinus1. Specifically, the bit length of PositionStartDeltaBitDepth[j] can be calculated as ceil(log2(PositionBitDepthMinus1+2)). Here, ceil(x) is a function that outputs the integer value obtained by rounding up the value x. In this way, by determining the bit length of PositionStartDeltaBitDepth[j] according to the value of PositionBitDepthMinus1, the amount of code can be efficiently reduced.

PositionPredMode is information indicating the method for encoding position information. For example, methodA may be used as a prediction method including fine prediction, coarse prediction, and no prediction mode, as shown in the encoding flowchart. The prediction method is not limited to methodA, and a prediction method combining two of the three prediction modes, or a prediction method consisting of one prediction mode, may also be used. A specific example is methodB, a prediction method combining coarse prediction and no prediction mode. By using methodB, predicted values can be generated without applying division or arithmetic processing requiring decimal point precision, thereby reducing the amount of processing and enabling lossless encoding. Furthermore, in the case of position information, methods such as parallelogram prediction may be included as fine prediction.

fineMaxcount is information indicating the maximum number of fine predictions used when calculating the average fine prediction value. When PositionPredMode includes fine prediction, for example when PositionPredMode is methodA, fineMaxcount can be added to the bitstream. By setting fineMaxcount to 1, the number of fine predictions used to calculate the average fine prediction value can be limited to one, essentially making average value calculation unnecessary. This reduces the amount of processing required to calculate the average value and prevents the occurrence of numbers with less than decimal precision due to average value calculation.

Fine prediction can also be prevented from being applied by setting fineMaxcount to 0. In this case, coarse prediction or no prediction mode can be used as the prediction mode. For example, if the position information of all 3D points following an initial 3D point is the same, the no prediction mode can be applied to the initial 3D point, and coarse prediction can be applied to subsequent 3D points, making it possible to always keep the prediction residual at 0. In such cases, setting fineMaxcount to 0 prevents the application of fine prediction, thereby reducing the amount of processing required for encoding or decoding.

Furthermore, if there is additional information required for fine prediction, the amount of coding can be reduced by setting fineMaxcount to 0, which will not apply fine prediction and will not encode the additional information.

Figure 66 shows an example of mesh_position_data syntax related to an embodiment.

PositionStartCount is information indicating the number of initial 3D points. PositionStart[c][j] is information indicating the value of the jth component of the position information for the cth initial 3D point. For example, if the position information is in xyz coordinates, PositionStart[c][0] may indicate the x value of the cth initial 3D point, PositionStart[c][1] may indicate the y value of the cth initial 3D point, and PositionStart[c][2] may indicate the z value of the cth initial 3D point.

Note that PositionStart[c][j] may be fixed-length coded using the bit length calculated using the PositionStartBitDepth described above. For example, if PositionStartBitDepth[0] is 10, PositionStartBitDepth[1] is 8, and PositionStartBitDepth[2] is 6, then PositionStart[c][0], the 0th component of the cth initial 3D point, has 10-bit precision, PositionStart[c][1], the 1st component, has 8-bit precision, and PositionStart[c][2], the 2nd component, has 6-bit precision. The 0th component may be fixed-length coded using 10 bits, the 1st component using 8 bits, and the 2nd component using 6 bits. In this way, the amount of code can be reduced by setting an appropriate bit length for each component of the initial 3D point and encoding it using that.

Also, if PositionStartBitDepth[j] is 0, PositionStart[c][j] does not need to be coded into the bitstream, and the decoder may infer that the value of PositionStart[c][j] is 0 when PositionStartBitDepth[j] is 0. This method reduces the amount of coding by not adding the jth component for which PositionStartBitDepth[j] is 0 to the bitstream.

Note that PositionStart may use variable-length coding such as unary coding or Exponential Golomb coding. It is also possible to assign a context to each bit after binarization and apply arithmetic coding. This reduces the amount of code.

Also, if PositionStartCount is 0, PositionStart does not need to be coded, and the decoder may infer that the value of PositionStart is 0 when PositionStartCount is 0. This reduces the amount of coding required.

Furthermore, if there is always at least one initial 3D point, PositionStartCountMinus1 may be added to the bitstream instead of PositionStartCount. In this case, the decoder may calculate PositionStartCount by adding 1 to PositionStartCountMinus1.

PositionFineResiCount is information indicating the number of prediction residuals of position information predictively coded using fine prediction. PositionFineResiSize is information indicating the code size of PositionFineResi, and may indicate, for example, the code size when PositionFineResi is binarized and arithmetically coded.

In this way, by informing the decoder of the code size using PositionFineResiSize before PositionFineResi, the decoder can pre-read the bits related to PositionFineResi from the bitstream. For example, while arithmetic decoding related to PositionFineResi is in progress, the decoding process for the following PositionCoarseResi can be performed in parallel, reducing the decoding time.

PositionFineResi[c][j] is information indicating the prediction residual of the jth component of the position information predictively coded using the cth fine prediction. PositionFineResi[c][j] may be binarized using Exponential Golomb coding or unary coding, and then variable-length coded. Alternatively, a context may be assigned to each bit, and the coding may be performed while updating the probability of occurrence based on the frequency of occurrence of 0s and 1s. This can improve coding efficiency.

Furthermore, the context used for arithmetic coding of each bit may be shared with the context used by PositionCoarseResi, which will be described later. This method allows the number of contexts to be reduced. Also, the context used for arithmetic coding of each bit may be different from the context used by PositionCoarseResi, which will be described later. This improves coding efficiency when the trends in prediction residuals differ between fine prediction and coarse prediction. Furthermore, using different contexts makes it possible to decode PositionFineResi and PositionCoarseResi in parallel.

PositionCoarseResiCount is information indicating the number of prediction residuals of position information predictively coded using coarse prediction. PositionCoarseResiSize is information indicating the code size of PositionCoarseResi, and may indicate, for example, the code size when PositionCoarseResi is binarized and arithmetically coded.

In this way, by informing the decoder of the code size using PositionCoarseResiSize before PositionCoarseResi, the decoder can pre-read the bits related to PositionCoarseResi from the bitstream. For example, while arithmetic decoding related to PositionCoarseResi is in progress, the decoding process for the following bitstream can be performed in parallel, reducing the decoding time.

PositionCoarseResi[c][j] is information indicating the prediction residual of the jth component of the position information predictively coded using the cth coarse prediction. PositionCoarseResi[c][j] may be binarized using Exponential Golomb coding or unary coding, and then variable-length coded. Alternatively, a context may be assigned to each bit, and the coding may be performed while updating the probability of occurrence based on the frequency of occurrence of 0s and 1s. This can improve coding efficiency.

Furthermore, the context used for arithmetic coding of each bit may be shared with the context used by PositionFineResi described above. This method allows the number of contexts to be reduced. Furthermore, the context used for arithmetic coding of each bit may be different from the context used by PositionFineResi described above. This can improve coding efficiency when the trends in prediction residuals differ between fine prediction and coarse prediction.

Furthermore, it is possible to decode PositionFineResi and PositionCoarseResi in parallel by using different contexts.

Note that PositionStartCount, PositionFineResiCount, PositionFineResiSize, PositionFineResi, PositionCoarseResiCount, PositionCoarseResiSize, and PositionCoarseResi may be binarized using Exponential Golomb coding or unary coding, and then variable-length coded. A context may also be assigned to each bit, and coding may be performed while updating the probability of occurrence based on the frequency of occurrence of 0 and 1. This can improve coding efficiency. It is also possible to code this information at a fixed length, reducing the amount of processing required.

(Configuration example)
Fig. 67 is a diagram showing an example of the configuration of an encoding device in an embodiment. Fig. 68 is a flowchart showing an example of an encoding method by the encoding device in an embodiment.

The encoding device 1530 includes a circuit 1531 and a memory 1532 connected to the circuit 1531. The encoding device 1530 may perform the processing described in Figures 54 and 55.

Circuit 1531 performs the following operations:

Circuit 1531 acquires multiple vertices included in a three-dimensional mesh (S1541). Circuit 1531 encodes attribute information of one of the multiple vertices using one of at least three prediction modes (S1542). The three prediction modes include a first prediction mode that does not make a prediction, a second prediction mode that performs predictive coding based on one processed vertex, and a third prediction mode that performs predictive coding based on two or more processed vertices. If the number of components of the attribute information is set to 1, during encoding (S1542), the attribute information is encoded using a prediction mode different from the third prediction mode.

This reduces the load on the calculation process by not applying the third prediction mode (predictive coding based on two or more vertices that have already been processed) when the number of components is set to 1. Furthermore, eliminating redundant prediction calculations improves the efficiency of the coding process.

For example, if a vertex has connectivity with another vertex among multiple vertices, the number of components in the attribute information is set to 1.

By setting the condition that connectivity exists when the number of components is 1, it becomes possible to apply an efficient encoding method that assumes connectivity. This improves the compression efficiency of encoded data.

For example, attribute information is specific attribute information.

This makes it easy to clarify what is to be coded and optimize it to improve coding efficiency by coding specific attribute information.

For example, the specific attribute information is plane information that indicates the area to which a vertex belongs when the vertex is projected onto a two-dimensional plane.

This allows for efficient representation of three-dimensional information by encoding it based on projection onto a two-dimensional plane. Furthermore, encoding planar information as attribute information simplifies data and improves encoding efficiency.

For example, the attribute information of one vertex is common to the attribute information of another vertex.

As a result, when multiple vertices with connectivity share the same attribute information, redundant data can be omitted, reducing the amount of coding required.

For example, when one vertex is coded in the second prediction mode, the prediction residual is 0.

As a result, the prediction residual is 0 when predictions made in the second prediction mode are a perfect match, further improving coding efficiency.

Figure 69 is a diagram showing an example of the configuration of a decoding device in an embodiment. Figure 70 is a flowchart showing an example of a decoding method performed by a decoding device in an embodiment.

The decoding device 1540 includes a circuit 1541 and a memory 1542 connected to the circuit 1541. The decoding device 1540 may perform the processing described in FIG. 56.

Circuit 1541 performs the following operations:

Circuit 1541 acquires multiple vertices included in a three-dimensional mesh (S1551). Circuit 1541 decodes attribute information of one of the multiple vertices using one of at least three prediction modes (S1552). The three prediction modes include a first prediction mode that does not make a prediction, a second prediction mode that performs predictive decoding based on one processed vertex, and a third prediction mode that performs predictive decoding based on two or more processed vertices. If the number of components of the attribute information is set to 1, during decoding (S1552), the attribute information is decoded using a prediction mode different from the third prediction mode.

This improves the efficiency of the decoding process by not using the third prediction mode when the number of components is 1 during decoding. Furthermore, by omitting redundant processing, the computational load of the decoding process can be reduced.

For example, if a vertex has connectivity with another vertex among multiple vertices, the number of components in the attribute information is set to 1.

By defining the number of components as 1 when connectivity exists, efficient decoding processing based on connectivity becomes possible, reducing the processing load.

For example, attribute information is specific attribute information.

By performing decoding processing on specific attribute information, it is possible to clarify the decoding target and improve decoding efficiency.

For example, the specific attribute information is plane information that indicates the area to which a vertex belongs when the vertex is projected onto a two-dimensional plane.

This makes it possible to perform decoding processing based on two-dimensional plane information, thereby enabling efficient decoding of attribute information for three-dimensional meshes.

For example, the attribute information of one vertex is common to the attribute information of another vertex.

As a result, when using common attribute information, redundant decoding of attribute information for multiple vertices can be omitted, improving decoding efficiency.

For example, when one vertex is decoded in the second prediction mode, the prediction residual is 0.

This reduces the load on the decoding process and improves the efficiency of data decoding.

Furthermore, the encoding device and decoding device of the present disclosure may be implemented by combining at least a portion of other aspects. Furthermore, the configuration or processing of the encoding device and decoding device of the present disclosure may be realized by combining part of the processing shown in any of the flowcharts according to this aspect, part of the configuration of any of the devices, part of the syntax, etc. with other aspects.

Furthermore, the processing performed by the decoding device of this disclosure may also be performed in an encoding device in a similar manner.

Note that the circuit 1531 of the encoding device 1530 may also perform the following operation. This operation example is the second example.

The circuit 1531 acquires multiple vertices included in a three-dimensional mesh (S1541). The circuit 1531 arithmetically encodes one of the multiple vertices using the first prediction mode (the second prediction mode in the above embodiment) or the second prediction mode (the third prediction mode in the above embodiment) (S1542). The first context used in the arithmetic encoding of the first prediction mode and the second context used in the arithmetic encoding of the second prediction mode are the same.

This allows for more efficient context management by using a common context for different prediction modes (first prediction mode and second prediction mode). This reduces the processing load required for initializing or updating the context, and improves the efficiency of the encoding process. Furthermore, by sharing a context, there is a possibility that encoding efficiency will improve even if the data has different parameters, as long as they have similar tendencies in arithmetic encoding. Sharing a context reduces the use of redundant context memory and improves encoding efficiency.

For example, the first prediction mode is a prediction mode that performs predictive coding based on one processed vertex. The second prediction mode is a prediction mode that performs predictive coding based on two or more processed vertices.

This allows for improved prediction accuracy depending on the situation by using different prediction modes: a first prediction mode (prediction based on one processed vertex) and a second prediction mode (prediction based on two or more processed vertices). In particular, prediction based on two or more processed vertices can achieve more accurate coding. Furthermore, by sharing contexts for the residuals of attribute information generated by separate predictions, the number of contexts can be reduced while improving the coding efficiency of arithmetic coding.

For example, arithmetic coding of one vertex includes arithmetic coding of the position information of one vertex.

As a result, by encoding position information, it is possible to encode the structure or vertex arrangement of a 3D mesh with high efficiency. Applying context sharing to position information can further improve the efficiency of arithmetic coding. In particular, sharing context for the residuals of position information generated using different prediction modes enables efficient encoding.

For example, arithmetic coding of one vertex includes arithmetic coding of the attribute information of one vertex.

This can contribute to improving the rendering or display quality of 3D meshes by including coding for attribute information (e.g., color, normal vectors, etc.). Applying a common context to attribute information can also improve coding efficiency. Furthermore, sharing a context for the residuals of attribute information generated by separate predictions can further improve coding efficiency.

For example, multiple vertices are included in one group based on a three-dimensional mesh. In other words, multiple vertices are included in a group of multiple vertices that make up the same three-dimensional mesh, or in a group of multiple vertices that make up the same sub-mesh.

This allows for the use of a common context when encoding vertices in the same group, thereby improving the prediction accuracy of related vertices. Furthermore, by sharing contexts on a mesh or sub-mesh basis, the number of contexts can be reduced, further improving encoding efficiency.

Furthermore, the circuit 1541 of the decoding device 1540 may perform the following operation. This operation example is the second example.

The circuit 1541 acquires multiple vertices included in a three-dimensional mesh (S1551). The circuit 1541 arithmetically decodes one of the multiple vertices using the first prediction mode (the second prediction mode in the above embodiment) or the second prediction mode (the third prediction mode in the above embodiment) (S1552). The first context used in the arithmetic decoding of the first prediction mode and the second context used in the arithmetic decoding of the second prediction mode are the same.

This allows for the use of a common context across different prediction modes, making the decoding process more efficient. Sharing a context reduces memory usage and processing load during arithmetic decoding. Furthermore, using a common context can improve decoding efficiency even when different parameters have the same tendency.

For example, the first prediction mode is a prediction mode that performs predictive decoding based on one processed vertex. The second prediction mode is a prediction mode that performs predictive decoding based on two or more processed vertices.

This allows for efficient decoding that corresponds to the prediction method used during encoding by using the first prediction mode (predictive decoding based on one processed vertex) and the second prediction mode (predictive decoding based on two or more processed vertices). Furthermore, by supporting multiple prediction methods, the accuracy of the decoding process can be improved. By sharing context, the amount of calculation and memory consumption during the decoding process can be reduced.

For example, arithmetic decoding of one vertex includes arithmetic decoding of the position information of one vertex.

This allows for cases where position information is coded during encoding, and allows the position information to be efficiently restored during decoding. Applying a common context to position information can improve decoding efficiency. Furthermore, using a common context for different prediction modes for position information can reduce processing load.

For example, arithmetic decoding of one vertex includes arithmetic decoding of the attribute information of one vertex.

This allows for cases where attribute information is coded during encoding, and allows the attribute information to be efficiently restored during decoding. By applying a common context to the attribute information, decoding efficiency can be improved. Furthermore, by sharing a context for the residuals of attribute information generated in different prediction modes, efficient decoding becomes possible.

For example, multiple vertices are included in one group based on a three-dimensional mesh. In other words, multiple vertices are included in a group of multiple vertices that make up the same three-dimensional mesh, or in a group of multiple vertices that make up the same sub-mesh.

This allows for improved processing efficiency during decoding by using a common context on a mesh or sub-mesh basis. This is particularly effective when processing multiple vertices as a single group, as it allows for efficient decoding while reducing redundant context.

FIG. 71 is a flowchart showing another example of the process for calculating predicted values in the third prediction mode according to an embodiment. In other words, it is a modified example of the flowchart in FIG. 55.

The encoding device sets count to 0 (S1561).

The encoding device determines whether count is less than fineMaxcount (S1562). If count is less than fineMaxcount (Yes in S1562), the encoding device proceeds to step S1563; if not (No in S1562), the encoding device proceeds to step S1565. Note that the encoding device can turn off fine prediction by setting fineMaxcount=0. The encoding device may also add the set fineMaxcount value to the bitstream. If the value of fineMaxcount decoded from the bitstream is 0, the decoding device may decode the attribute information without applying fine prediction.

The encoding device calculates the count-th fine prediction value with decimal precision (S1563).

The encoding device adds 1 to count, updates it (S1564), and returns to step S1562.

The encoding device determines whether count is greater than 0 (S1565). If count is greater than 0 (Yes in S1565), the encoding device proceeds to step S1566; if not (No in S1565), the encoding device proceeds to step S1568.

The encoding device calculates the average fine prediction value with decimal precision (S1566). That is, the encoding device calculates the average of one or more fine prediction values as the average fine prediction value. Note that the one or more fine prediction values are the count fine prediction values obtained in steps S1561 to S1564. Note that if there is one fine prediction value, this one fine prediction value may be calculated as the average fine prediction value. Note that in S1566, it is sufficient to combine two or more fine prediction values to calculate one fine prediction value, and calculation of the average fine prediction value is not limited to this.

The encoding device adjusts the number of bits in the decimal part of the average fine prediction value (integrated value) so that it is smaller (S1567). The encoding device, for example, adjusts the number of bits in the decimal part to 0. In other words, the encoding device converts the average fine prediction value (integrated value) to integer precision (integerization). Note that when adjusting the number of bits, the encoding device may perform a process to round decimal values, such as by rounding off.

The encoding device outputs count (S1568).

Next, we will explain an example of using information from three-dimensional space to calculate fine prediction values in the UV coordinates of a two-dimensional plane.

FIG. 72 is a diagram showing an example of vectors in three-dimensional space when performing fine prediction according to an embodiment. FIG. 73 is a diagram showing an example of vectors on a two-dimensional plane when performing fine prediction according to an embodiment.

As shown in Figure 72, two encoded 3D points G(next) and G(prev) are connected to the 3D point G(curr). Furthermore, as shown in Figure 73, the points obtained by projecting these 3D points G(curr), G(next), and G(prev) onto the UV coordinate system are UV(curr), UV(next), and UV(prev), respectively. Below, we explain an example of how to calculate the predicted value (predUV) when fine-predicting a vector for UV(curr).

Here, let gCurr, gNext, and gPrev be the three-dimensional vectors corresponding to the three-dimensional points G(curr), G(next), and G(prev), respectively, and let uvCurr, uvNext, and uvPrev be the two-dimensional vectors corresponding to the two-dimensional points UV(curr), UV(next), and UV(prev), respectively. Also, let gNgP be the vector from gNext to gPrev, gNgC be the vector from gNext to gCurr, and uvNuvP be the vector from UV(next) to UV(prev). Furthermore, let dot(gNgP, gNgC) be the dot product of gNgP and gNgC, and dot(gNgP, gNgP) be the dot product of two gNgPs.

Let the projection points of the 3D point G(curr) and the 2D point UV(curr) onto the opposite sides be the 3D point G(proj) and the 2D point UV(proj), respectively. The vectors gProj and uvProj can be found by assuming that the triangle formed by the 3D points G(curr), G(next), and G(prev) is similar to the triangle formed by the 2D points UV(curr), UV(next), and UV(prev). Specifically, the vectors gProj and uvProj are expressed as follows:

uvProj = uvNext + uvNuvP × (dot(gNgP, gNgC) / dot(gNgP, gNgP))
gProj = gNext + gNgP × (dot(gNgP, gNgC) / dot(gNgP, gNgP))

Furthermore, the dot product of the vectors from the projection point G(proj) to G(curr) is dot(gCurr - gProj, gCurr - gProj). Here, the vector orthogonal to uvNuvP is defined as orthogonal(uvNuvP).

In this case, the distance between the projection points G(proj) and G(curr) corresponds to the distance between the projection points UV(proj) and UV(curr) under the assumption that the triangles are similar. Therefore, the vector uvProjuvCurr from the projection points UV(proj) to UV(curr) can be expressed as follows:

uvProjuvCurr = orthogonal(uvNuvP) * sqrt(dot(gCurr - gProj, gCurr - gProj) / dot(gNgP, gNgP))

where sqrt is the square root.

Using this result, the predicted vector predUV can be calculated as follows:

predUV = uvProj + uvProjuvCur

Alternatively, the predicted vector can be calculated as follows, assuming predUV1:

predUV1 = uvProj - uvProjuvCur

In this way, by considering prediction vectors in the opposite direction to predUV, prediction accuracy can be improved and the amount of coding can be reduced.

The above shows a specific example of a fine prediction calculation, but if the range of vector components is extremely wide, for example, setting a large number of decimal points to ensure calculation accuracy may result in overflow. On the other hand, reducing the number of decimal points or performing calculations with integer precision to avoid overflow may result in a decrease in calculation accuracy.

To solve this problem, the number of bits in the fractional part can be adjusted based on the magnitude of the dot product between vector gNgP and vector gNgC, or the magnitude of the absolute value of the dot product.

The device may determine whether the magnitude of the absolute value of dot(gNgP, gNgC) exceeds the threshold value TH, and perform calculations according to the determination result. Figure 74A is a diagram showing an example of calculation of a predicted value when the magnitude of the absolute value of dot(gNgP, gNgC) exceeds the threshold value TH in accordance with an embodiment. Figure 74B is a diagram showing an example of calculation of a predicted value when the magnitude of the absolute value of dot(gNgP, gNgC) is equal to or less than the threshold value TH in accordance with an embodiment.

As a specific example, if the magnitude of the absolute value of dot(gNgP, gNgC) exceeds the threshold TH = (1 << (9 * 2)) - 1, the device may perform integer arithmetic without preserving the fractional part, as shown in Figure 74A.

On the other hand, if the magnitude of the absolute value of dot(gNgP, gNgC) is TH = (1 << (9 * 2)) - 1 or less, as shown in Figure 74B, the device may perform calculations to reserve a specified number of bits (for example, 9 bits) for the decimal part. This allows the calculation precision to be lowered when the vector is large. Conversely, the calculation precision can be increased when the vector is small. In this way, the device can adjust the number of bits for the decimal part to widen the range of vector values that it can support.

The above describes a method for switching the number of bits in the fractional part in two stages, but the device may also determine the number of bits in the fractional part depending on the number of bits in the magnitude of the absolute value of dot(gNgP, gNgC). For example, the device may determine the number of bits in the fractional part using a logarithmic value with base 2 based on the magnitude of the absolute value of dot(gNgP, gNgC). This allows the device to adjust the number of bits in the fractional part more accurately. Alternatively, the number of bits in the fractional part may be set to only one fixed value. This allows the device to reduce the amount of processing.

Furthermore, the device may determine the number of bits in the fractional part based on a value other than the magnitude of the absolute value of dot(gNgP, gNgC). For example, a method may be adopted in which the determination is based on the magnitude of the vector itself. This allows the device to reduce the amount of processing.

In the above example, the number of bits in the decimal part is set to 9 bits, but it is not limited to being set to 9 bits. For example, the device may set the number of bits in the decimal part based on the number of bits in the hardware register or arithmetic unit. The number of bits may also be set based on the performance of the software or system. Furthermore, the optimal number of bits may be set by taking into account the trade-off between accuracy and processing volume. In this way, the device can improve calculation accuracy and reduce the amount of code by setting an appropriate number of bits depending on the execution environment.

The device may also add the threshold value TH to the header of the bitstream. By adding the threshold value TH used by the encoding device to the header and encoding it, the decoding device can obtain the threshold value TH from the header and properly decode the bitstream using the same value as the number of bits in the fractional part used by the encoding device.

Figure 75 shows another example of calculating a predicted value according to an embodiment.

After calculating the predicted value, for example, when calculating the average value of the fine predicted values, the device may adjust the number of bits in the decimal part to 8 bits to ensure accuracy in the division process, as shown in FIG. 75. This adjustment may be performed regardless of the magnitude of the absolute value of dot(gNgP, gNgC). The device may also convert the average value of the fine predicted values to integer precision, or may adjust the number of bits in the decimal part. In this case, the device may perform rounding processing such as rounding off. The rounding processing may round positive numbers and negative numbers in the same direction, or may round them in different directions. In some cases, the device may not perform rounding processing.

FIG. 76 is a flowchart showing an example of a process for calculating a predicted value according to an embodiment.

The device calculates the dot product dot(gNgP, gNgC) of vector gNgP and vector gNgC (S1571).

The device determines whether the magnitude of the absolute value of the dot product dot(gNgP, gNgC) exceeds the threshold value TH (S1572). If the magnitude of the absolute value of the dot product dot(gNgP, gNgC) exceeds the threshold value TH (Yes in S1572), the device proceeds to step S1573; if not (No in S1572), the device proceeds to step S1575.

The device calculates the predicted value (S1573). Specific examples of calculating the predicted value are as described in Figures 72 and 73.

The device adjusts the decimal part of the predicted value to 8 bits (S1574). In other words, the device converts the decimal part of the predicted value to 8 bits.

The device adjusts the decimal part of the predicted value to 9 bits (S1575). In other words, the device makes the decimal part of the predicted value 9 bits.

The device calculates the predicted value (S1576). Specific examples of calculating the predicted value are as described in Figures 72 and 73.

The device adjusts the decimal part of the predicted value to 8 bits (S1577). In other words, the device converts the decimal part of the predicted value to 8 bits.

Once step S1574 or S1577 is executed, the process of calculating the predicted value ends.

Next, we will provide a more specific example of how to calculate predicted values.

The device can calculate the dot product of vectors gNgP and gNgC as dot(gNgP, gNgC), with the x, y, and z components being .x, .y, and .z, respectively, as follows:

dot(gNgP, gNgC) = gNgP.x * gNgC.x + gNgP.y * gNgC.y + gNgP.z * gNgC.z

Similarly, the device can calculate the dot product of vectors gNgP as dot(gNgP, gNgP) as follows:

dot(gNgP, gNgP) = gNgP.x * gNgP.x + gNgP.y * gNgP.y + gNgP.z * gNgP.z

Next, the device determines the magnitude of the absolute value of the dot product dot(gNgP, gNgC) as | dot(gNgP, gNgC) | and determines the number of decimal points, shiftBits, which represents the bit shift amount, as follows:

if (| dot(gNgP, gNgC) | > (1 << (9 * 2)) - 1) {
shiftBits = 0
} else {
shiftBits = 9
}
}

The device can use this bit shift amount, shiftBits, to calculate the vector to the projection point as follows:

uvProj = (uvNext << shiftBits) + (uvNuvP * (dot(gNgP, gNgC) << shiftBits))
/ dot(gNgP, gNgP)
gProj = (gNext << shiftBits) + (gNgP * (dot(gNgP, gNgC) << shiftBits))
/ dot(gNgP, gNgP)

Furthermore, the device calculates the dot product of the vectors from the projection point G(proj) to the point G(curr) as dot(gCurr - gProj, gCurr - gProj) as follows:

dot(gCurr - gProj, gCurr - gProj)
= ((gCurr.x << shiftBits) - gProj.x) * ((gCurr.x << shiftBits) - gProj.x)
+ ((gCurr.y << shiftBits) - gProj.y) * ((gCurr.y << shiftBits) - gProj.y)
+ ((gCurr.z << shiftBits) - gProj.z) * ((gCurr.z << shiftBits) - gProj.z)

The device also defines the vector orthogonal to the two-dimensional vector uvNuvP as orthogonal(uvNuvP), and calculates the vector uvProjuvCurr as follows:

uvProjuvCurr
= ((orthogonal(uvNuvP) << shiftBits) * sqrt(dot(gCurr - gProj, gCurr - gProj)))
/ sqrt(dot(gNgP, gNgP) << (shiftBits * 2))

where sqrt is the square root. The device may also use an integer approximation of the square root for sqrt. This allows operations to be performed in integer format.

The device can calculate the candidate predicted vector predUV0 or predUV1 as follows:

predUV0 = uvProj + uvProjuvCurr

Also, if the direction of uvProjuvCurr is reversed, it can be calculated as follows:

predUV1 = uvProj - uvProjuvCurr

If a device wants to set the number of bits in the fractional part to 8, it can do the following:

if (shiftBits >= 8) {
predUV0 >>= (shiftBits - 8)
predUV1 >>= (shiftBits - 8)
} else {
predUV0 <<= (8 - shiftBits)
predUV1 <<= (8 - shiftBits)
}

By performing the above process, the device can change the number of bits in the decimal part from shiftBits to 8 bits. Note that while this embodiment shows an example of changing to 8 bits, this is not necessarily limited to this, and the device can also change the number of bits in the decimal part to any n bits (n is an integer greater than or equal to 0).

If the device changes the number of bits in the fractional part from shiftBits to n bits, it may perform the following processing.

if (shiftBits >= n) {
predUV0 >>= (shiftBits - n)
predUV1 >>= (shiftBits - n)
} else {
predUV0 <<= (n - shiftBits)
predUV1 <<= (n - shiftBits)
}

This process allows the device to change the number of bits in the fractional part from shiftBits to any n bits.

Either of the candidate prediction vectors predUV0 or predUV1 is selected, and the prediction vector predUV is set as follows:

(1) When predUV0 is selected:
predUV = predUV0
(2) When predUV1 is selected:
predUV = predUV1

Furthermore, when calculating predUV from predUV0 or predUV1, some offset value may be added or subtracted, taking into account that the decimal point is 8 bits. An example is shown below.

predUV = (offset0 << 8) + predUV0
predUV = (offset1 << 8) - predUV1

This process adjusts the predicted vector using the offset.

Furthermore, several predicted vectors may be added together to calculate the average value. For example, count predicted vectors may be cumulatively added together to obtain a predicted vector predUVcount, which may then be rounded and averaged.

if(predUVcount.x >= 0) {
rounding.x = 1 << (8 - 1)
} else {
rounding.x = -1 * (1 << (8 - 1))
}
if(predUVcount.y >= 0) {
rounding.y = 1 << (8 - 1)
} else {
rounding.y = -1 * (1 << (8 - 1))
}
predUV = (predUVcount / count + rounding) >> 8

In this process, the direction of rounding and adding is adjusted depending on the sign of predUVcount. This improves the accuracy of the predicted vector while increasing encoding efficiency.

In the above example, the direction of rounding and adding is adjusted depending on the sign of predUVcount, but you can also use a method that rounds in the same direction, or a method that does not perform rounding and adding.

Also, the above formula "predUV = (predUVcount / count + rounding) >> 8" indicates the following individual processing:

predUV.x = (predUVcount.x / count + rounding.x) >> 8
predUV.y = (predUVcount.y / count + rounding.y) >> 8

This ensures that each component is processed appropriately.

Next, we will explain an example of calculating fine prediction values in UV coordinates using information from three-dimensional space.

FIG. 77 is a diagram showing an example of vectors in three-dimensional space when performing fine prediction according to a modified example of the embodiment. FIG. 78 is a diagram showing an example of vectors on a two-dimensional plane when performing fine prediction according to a modified example of the embodiment.

As shown in Figure 77, the two encoded 3D points connected to the 3D point G(Curr) are designated as G(Next) and G(Prev). As shown in Figure 78, the points obtained by projecting these 3D points G(curr), G(next), and G(prev) onto the UV coordinates are designated as UV(Next) and UV(Prev), respectively. The point obtained by projecting G(Curr) onto the UV coordinates is designated as UV(Curr). Here, an example of fine-predicting a vector for UV(Curr) (deriving predUV) is shown.

Here, the three-dimensional vectors corresponding to the known three-dimensional points G(Curr), G(Next), and G(Prev) are respectively the three-dimensional vector gCurr, the three-dimensional vector gNext, and the three-dimensional vector gPrev, and the two-dimensional vectors corresponding to the known two-dimensional points UV(Next) and UV(Prev) are respectively the two-dimensional vector uvNext and the two-dimensional vector uvPrev. Furthermore, the vector from the three-dimensional vector gNext to the three-dimensional vector gPrev is the three-dimensional vector gNgP, the vector from the three-dimensional vector gNext to the three-dimensional vector gCurr is the three-dimensional vector gNgC, and the vector from the two-dimensional vector uvNext to the two-dimensional vector uvPrev is the two-dimensional vector uvNuvP.

Let the dot product of three-dimensional vectors gNgP and gNgC be dot(gNgP, gNgC), and the dot product of two three-dimensional vectors gNgP together be dot(gNgP, gNgP). Here, let the projection points of the three-dimensional point G(Curr) and the two-dimensional point UV(Curr) onto their respective opposite sides be the three-dimensional point G(Proj) and the two-dimensional point UV(Proj).

The three-dimensional vector gProj for the three-dimensional point G(Proj) and the two-dimensional vector uvProj for the two-dimensional point UV(Proj) can be expressed as follows using the dot product ratio, assuming that the triangle formed by the three-dimensional point G(Curr), three-dimensional point G(Next), and three-dimensional point G(Prev) is similar to the triangle formed by the two-dimensional point UV(Curr), two-dimensional point UV(Next), and two-dimensional point UV(Prev).

uvProj = uvNext + uvNuvP * (dot(gNgP, gNgC) / dot(gNgP, gNgP))
gProj = gNext + gNgP * (dot(gNgP, gNgC) / dot(gNgP, gNgP))

Furthermore, let the dot product of the vectors from the projection point 3D point G(Proj) to the 3D point G(Curr) be dot(gCurr - gProj, gCurr - gProj). Let the vector orthogonal to the 2D vector uvNuvP be orthogonal(uvNuvP), and similarly use triangle similarity to express the vector from the projection point 2D point UV(Proj) to the 2D point UV(Curr), the 2D vector uvProjuvCurr, as follows:

uvProjuvCurr = orthogonal(uvNuvP) * sqrt(dot(gCurr - gProj, gCurr - gProj)
/ dot(gNgP, gNgP))

where sqrt represents the square root.

Therefore, the predicted vector two-dimensional vector predUV can be calculated, for example, as follows:

predUV = uvProj + uvProjuvCur

Depending on the direction of the two-dimensional vector uvProjuvCurr, the predicted vector can be set to predUV0 or its opposite, predUV1, and calculated as follows:

predUV0 = uvProj + uvProjuvCur
predUV1 = uvProj - uvProjuvCur

In this way, by considering predUV as a bidirectional prediction vector, prediction accuracy can be improved and the amount of coding can be reduced.

Figure 79 shows another example of calculating a predicted value according to an embodiment.

In this embodiment, a method of ensuring a fixed number of decimal points, N, in order to ensure integer arithmetic accuracy when calculating fine prediction in three-dimensional space will be described. For example, as shown in FIG. 79, the decimal point may be set to N bits. N bits may be set, for example, as N=1, N=2, N=3, ..., N=5, N=6, N=7, N=8. N=16 or N=0 (treated as an integer with no decimal point) may also be used.

The number of bits N in the fractional part may be changed depending on the processing content of the fine prediction. For example, when calculating the square root, the number of bits in the fractional part is halved, so by doubling the number of bits in the fractional part to 2N bits before the operation, the fractional part of the calculation result can be maintained at N bits.

Furthermore, to prevent values from overflowing, the number of bits N in the fractional part can be decreased by shifting it to the right. Conversely, to prevent underflow, the number of bits N in the fractional part can be increased by shifting it to the left in advance. Furthermore, to improve the accuracy of calculations such as integer division, the number of bits can be increased by shifting the fractional part to the left before the calculation, and then decreased by shifting the fractional part to the right after the calculation. In this case, calculation errors can be suppressed by performing rounding processing. This makes it possible to accommodate a wide range of input values.

The number of bits N in the decimal part is not limited to the above setting value. For example, it can be determined based on the number of bits in the hardware register or arithmetic unit. It can also be set based on the performance of the software or system, or the optimal number of bits can be determined by taking into account the trade-off with calculation accuracy. In this way, by setting an appropriate number of bits based on the execution environment, calculation accuracy can be improved and the amount of code can be reduced.

The number of bits N in the fractional part may also be added to the header of the bitstream. Using this method, the number of bits N used by the encoding device is included in the header and encoded, allowing the decoding device to obtain the number of bits N from the header. This allows the decoding device to perform decoding processing using the same number of bits as the number of bits in the fractional part used by the encoding device, making it possible to properly decode the bitstream.

Furthermore, the number of bits N in the fractional part can be switched to a different value depending on the profile or level of the standard. For example, profile A, which targets high image quality, can use a large value (e.g., N=16) to increase calculation accuracy, while profile B, which targets low processing loads, can use a small value (e.g., N=5) to reduce the number of bits required. In this way, by specifying the number of bits N in the fractional part depending on the profile or level of the standard, the decoding device can appropriately switch the number of bits N based on the profile or level information contained in the stream and correctly decode the bitstream.

The number of bits N in the decimal part can also be determined based on the input value or conditions. For example, in the example of calculating the count-th fine prediction value, one possible method is to determine the number of bits N in the decimal part based on the number of bits of the magnitude of the absolute value of dot(gNgP, gNgC). More specifically, the number of bits N in the decimal part can be determined using the number of bits of dot(gNgP, gNgC) or the logarithm to the base 2 of the magnitude of the absolute value. This ensures an appropriate number of bits N in the decimal part based on the range of input values, improving calculation accuracy.

Figure 80 is a flowchart showing another example of the process for calculating predicted values according to an embodiment.

The device converts the decimal part of the predicted value to N bits (S1581). In other words, the device calculates the predicted value with decimal precision.

The device calculates the vector dot product value and dot product ratio (S1582).

The device converts the decimal part into 2*N bits (S1583).

The device calculates the square root (S1584). In this way, the decimal part may be converted to 2*N bits in step S1583 before calculating the square root.

The device converts the decimal part to N bits (S1585). Step S1585 may be incorporated into step S1584. In other words, the device may convert the decimal part to N bits when calculating the square root.

The device calculates the predicted value (vector) (S1586).

As shown in this flowchart, when calculating the square root, the number of bits in the fractional part is also halved, so by doubling the number of bits N in the fractional part in advance, it is possible to maintain the same precision as the number of bits N in the fractional part on the output side.

In addition, the number of bits may remain N as is as input to the square root calculation process, or the decimal part may be converted to N bits after the square root calculation process is output.

Next, we will show a specific example of how to calculate predicted values.

First, let the dot product of vectors gNgP and gNgC be dot(gNgP, gNgC), and let the symbols representing the x, y, and z components be .x, .y, and .z, respectively, and it can be calculated as follows:

dot(gNgP, gNgC) = gNgP.x * gNgC.x + gNgP.y * gNgC.y + gNgP.z * gNgC.z

Similarly, the device can calculate the dot product of vectors gNgP as dot(gNgP, gNgP) as follows:

dot(gNgP, gNgP) = gNgP.x * gNgP.x + gNgP.y * gNgP.y + gNgP.z * gNgP.z

Here, we will assume that the number of bits in the decimal part is N = 8, and that shiftBits = N = 8. Note that shiftBits can also be set to values such as 5, 6, or 7.

Next, we will explain how to find the vector to the projection point. In this case, the "/" in the calculation means integer division that truncates to the decimal point. The vector to the projection point is calculated using the following formula.

uvProj = (uvNext << shiftBits) + uvNuvP * ((dot(gNgP, gNgC) << shiftBits)
/ dot(gNgP, gNgP))
gProj = (gNext << shiftBits) + gNgP * ((dot(gNgP, gNgC) << shiftBits)
/ dot(gNgP, gNgP))

Next, the dot product of the vectors from the projection point G(Proj) to G(Curr) is taken as dot(gCurr - gProj, gCurr - gProj) and calculated as follows:

dot(gCurr - gProj, gCurr - gProj)
= ((gCurr.x << shiftBits) - gProj.x) * ((gCurr.x << shiftBits) - gProj.x)
+ ((gCurr.y << shiftBits) - gProj.y) * ((gCurr.y << shiftBits) - gProj.y)
+ ((gCurr.z << shiftBits) - gProj.z) * ((gCurr.z << shiftBits) - gProj.z)

Furthermore, if the vector orthogonal to the two-dimensional vector uvNuvP is defined as orthogonal(uvNuvP), the vector uvProjuvCurr can be expressed as follows:

uvProjuvCurr = orthogonal(uvNuvP) * sqrt(dot(gCurr - gProj, gCurr - gProj)
/ dot(gNgP, gNgP))

However, sqrt() means square root operation.

In this case, since the number of bits in the fractional part of dot(gCurr - gProj, gCurr - gProj) is 2N (2 * shiftBits), the number of bits in the fractional part of the input to sqrt() is also 2N (2 * shiftBits).

Therefore, the predicted vector candidate predUV0 or predUV1 is calculated as follows:

predUV0 = uvProj + uvProjuvCurr
predUV1 = uvProj - uvProjuvCurr (if uvProjuvCurr is facing in the opposite direction)

Either predUV0 or predUV1 obtained in this way can be used as the prediction vector.

Either the predicted vector candidate predUV0 or predUV1 is selected, and the predicted vector predUV is set as follows:

(1) When predUV0 is selected:
predUV = predUV0
(2) When predUV1 is selected:
predUV = predUV1

Furthermore, when calculating predUV from predUV0 or predUV1, some offset value may be added or subtracted, taking into account that the decimal part is shiftBits. An example is shown below.

predUV = (offset0 << shiftBits) + predUV0
predUV = (offset1 << shiftBits) - predUV1

Furthermore, several predicted vectors may be added together to calculate the average value. For example, count predicted vectors may be cumulatively added together to obtain a predicted vector predUVcount, which may then be rounded and averaged.

if (predUVcount.x >= 0) {
rounding.x = 1 << (shiftBits - 1);
} else {
rounding.x = -1 * (1 << (shiftBits - 1));
}
if (predUVcount.y >= 0) {
rounding.y = 1 << (shiftBits - 1);
} else {
rounding.y = -1 * (1 << (shiftBits - 1));
}
predUV = (predUVcount / count + rounding) >>shiftBits;

In this way, by setting the rounding direction to differ based on the sign of predUVcount, rounding errors can be reduced. Note that it is also possible to use a method that rounds in the same direction, or to avoid rounding addition.

Also, the above formula predUV = (predUVcount / count + rounding) >> shiftBits indicates that processing is performed separately for each x and y component, as follows:

predUV.x = (predUVcount.x / count + rounding.x) >> shiftBits
predUV.y = (predUVcount.y / count + rounding.y) >> shiftBits

Next, we will show a specific example of the square root calculation process (sqrt). Let a be the input value for the square root calculation, and the square root of a, root, be expressed as follows:

The square root calculation process may be performed using Newton's method using the following recurrence formula:

After convergence, root=x _n .

More specifically, it is calculated as follows:

The device may also switch the number of iterations k of the recurrence formula to a different value depending on the profile or level of the standard. For example, in the case of profile A, which targets high image quality, the device may use a large value for the number of iterations k of the recurrence formula in order to increase calculation accuracy. Specifically, it may be possible to use a large value such as k=5.

On the other hand, for Profile B, which targets low processing load, the device can use a small value for the number of iterations k of the recurrence formula in order to reduce processing load. For example, a small value such as k = 2 could be used.

In this way, by specifying the value of the number of iterations k of the recurrence formula according to the profile or level of the standard, the decoding device can correctly decode the bitstream by appropriately switching the number of iterations of the recurrence formula based on the profile or level information contained in the stream.

In addition, for a certain profile C, the device can also set the number of iterations k of the recurrence formula to 1. In this case, the device can perform one iteration by right bit shifting, eliminating the need to perform division and allowing the device to specify a profile with reduced processing volume.

The device may also add the number of iterations k of the recurrence formula to the header of the bitstream, etc. This allows the device (encoding device side) to encode by adding the number of iterations k used to the header. Furthermore, the decoding device can obtain the number of iterations k from the header and perform decoding processing using the same value as the number of iterations used by the encoding device, thereby properly decoding the bitstream.

In the processing of equation 3, "≪" indicates a left shift, "≫" indicates a right shift, "/" indicates integer division that truncates to the nearest digit, "++" indicates an increment, and "≫=" indicates that the shifted value is assigned.

The initial value _x0 of the recurrence formula can be calculated using the bit count value bits(a) of the input value a as an approximation of the square root, which is the true value. Specifically, _x0 = 2 ^{((bits(a) >> 1))} may be used. For example, when the input value a = 8, the bit count value bits(a) = 4, and the initial value _x0 can be set to _x0 = 4. Furthermore, when the input value a = 40, the bit count value bits(a) = 6, and the initial value _x0 can be set to _x0 = 8.

In this way, the initial approximate value can be easily calculated without performing complex calculations such as lookup tables or division. Furthermore, by using an approximate value of the square root (the true value) as the initial value, it is possible to find a highly accurate square root while minimizing the number of iterations required until the recurrence formula converges. Furthermore, by minimizing the number of iterations, the square root can be found quickly.

For example, by using an approximation of the true square root as the initial value, it is possible to obtain a highly accurate square root even if the recurrence formula is repeated only twice, thereby achieving low processing load.

Furthermore, by setting the initial value _x0 of the recurrence formula to a value that is a power of 2, such as _x0 = 2 ^{((bits(a) >> 1))} , the division process in the first iteration of the recurrence formula can be performed by a right shift. This reduces the number of division processes, which are computationally expensive.

For example, if the number of iterations of a recurrence formula is set to two, the initial value of the recurrence formula can be set to a power of two, so that the division process in the first iteration is performed by a right shift, and the division process is performed in the second iteration. In this way, the number of division processes can be reduced by one.

Furthermore, since the input value a has a 2*N-bit fractional part, sufficient precision can be obtained by simply performing integer division in the division in the second iteration using the recurrence formula. Therefore, there is no need to reserve the fractional part separately. This reduces the amount of processing.

If it is not necessary to find a square root with high accuracy, the first iteration using the recurrence formula may be terminated, and the value representing the square root may be set to root = x _1. On the other hand, if it is necessary to find a square root with even higher accuracy, the iterative process may be performed k times, and the second and subsequent iterative processes using the recurrence formula may be performed (k - 1) times, and the value representing the square root may be set to root = x _n .

In this case, the second and subsequent iterations of the recurrence formula can be repeated as follows:

Alternatively, the first iteration using the recurrence formula with shifts may not be performed, but may be included in the second and subsequent iterations, with all iterations calculated in the same way.

Next, we will show another specific example of the square root calculation process (sqrt). Let the input value for the square root calculation be a, and express the square root of a, root, as follows:

The square root calculation process may be performed using Newton's method using the following recurrence formula to find the reciprocal of the square root:

After convergence, root=1/x _n .

More specifically, it is calculated as follows:

In the above process, "≪" means left shift, "≫" means right shift, "++" means increment, and "≫=" means assigning the shifted value.

The initial value _x0 of the recurrence formula may be calculated using the bit count value bits(a) of the input value a as an approximation of the reciprocal of the square root, which is the true value. In this case, _x0 may be calculated as _x0 = 2 ^{- ((bits(a) >> 1))} .

This allows the device to easily calculate approximate values without having to perform complex calculations such as lookup tables or division. Furthermore, by using an approximate value of the reciprocal of the square root (the true value) as the initial value, the device can calculate the square root with high accuracy while minimizing the number of iterations required until the recurrence formula converges. Furthermore, by minimizing the number of iterations, the device can quickly calculate the square root.

For example, by using an approximation to the reciprocal of the true square root as the initial value, the device can calculate the square root with high accuracy even when the recurrence formula is repeated twice, thereby achieving low processing load.

Furthermore, since _x0 is a power of 2, the device can perform the multiplication process by right shifting in the first iteration of the recurrence formula. In this way, there is no need to multiply values with fractional parts, and overflow due to insufficient bits can be easily avoided. Furthermore, since it can be achieved by a shift operation, the device can be easily implemented.

Next, we will explain in more detail other specific examples of the square root calculation process (sqrt), including their effects. Figures 81 to 85 are figures for explaining other specific examples of the square root calculation process (sqrt) according to the embodiment.

As shown in Figure 81, the input a for square root calculation is assumed to have a maximum total of 48 bits, with the integer part being 32 bits and the decimal part being N=8, 8*2=16 bits.

The recurrence formula for the first iterative process is shown in Equation 6. If the initial value is _x0 = 2 ^{- ((bits(a) >> 1))} , it is necessary to multiply the input a, which has a large number of bits, by _x0 , which has a large fractional part. Furthermore, since it is necessary to multiply _x0 three times, a considerable number of bits is required.

Here, we add a new fractional part of (N × 2 + (bits(a)≫1)) bits to input a, that is, (16 + (bits(a)≫1)) bits when N = 8. In this case, the (N × 2) part is a fixed fractional part that does not depend on the number of bits of input a, and the (bits(a)≫1) part is a variable fractional part that depends on the number of bits of input a (Figure 82).

As described above, simply adding a fractional part requires more than 80 bits, which could result in overflow in a system with a 64-bit register or arithmetic unit. Therefore, taking advantage of the fact that _x0 is a negative power of 2, we add a variable fractional part (left shift) and process the multiplication by _x0 as a truncation operation with a right shift of (bits(a) >> 1). Here, the addition of the variable fractional part is performed as a left shift operation, and the reduction of the number of bits associated with the multiplication by _x0 is processed as a right shift, thereby canceling out the left and right shifts. In this way, the process of adding a variable fractional part with a left shift and the right shift associated with the multiplication by _x0 simultaneously prevent an increase in the number of bits. Therefore, the device can implement _x0 as a process of adding a fixed fractional part to input a with a left shift operation (Figure 83). This process allows us to add a fractional part necessary and sufficient for square root calculation while avoiding overflow in a 64-bit system.

Next, in the recurrence formula in the first iteration, the device can perform the multiplication by _x0 by a right shift of (bits(a)>>1), as shown in Figures 84 and 85.

This process allows _x1 to maintain 16-bit precision even after the first iteration is complete.

Next, the second iteration of the square root calculation process (sqrt) will be explained in more detail, including its effects. Figures 86 to 93 are figures explaining other specific examples of the square root calculation process (sqrt) according to the embodiment.

First, when the operation a·x ₁ is performed, the operation result becomes 64 bits as shown in Figure 86. However, if the operation a·x ₁ ² is then performed, there is a possibility of overflow. Therefore, by truncating (N×2=16) bits by a right shift operation as shown in Figure 87 and then performing the operation a·x ₁ ² (Figure 88), overflow can be prevented.

Next, when the operation a x ₁ ² is performed, the result becomes 64 bits again. However, if the operation x ₁ × (3 - a x ₁ ² ) is then performed, there is a possibility of overflow. For this reason, the (bits(a) >> 1) bit is truncated by a right shift operation as shown in Figure 89, and then the operation x ₁ × (3 - a x ₁ ² ) is performed (Figure 90). Note that because (N × 2 = 16) bits were previously truncated by a right shift operation, only the (bits(a) >> 1) bit is truncated by the right shift operation in this process. This enables accurate calculations while preventing overflow without reducing the number of bits more than necessary.

Furthermore, when _x1 × (3 - a· _x12 ⁾ is calculated, the result is 56 bits. Finally, to calculate the square root root from the reciprocal of the square root _x2 , multiplication by the input a (48 bits) may result in an overflow. For this reason, as shown in Figure 91, it is necessary to discard (16 + (bits(a) >> 1)) bits by a right shift operation. Note that in this process, when calculating ( _x1 /2) × (3 - a ^· _x12 ), (16 + (bits(a) >> 1) + 1) bits are discarded by a right shift.

When the second iterative process is completed and the converged reciprocal _x2 of the square root is found, the reciprocal _x2 is multiplied by the input a to calculate the square root, as shown in FIG.

Finally, as shown in Figure 93, the final square root output, root, can be obtained by truncating the fractional part (16 + (bits(a) >> 1)) bits added during the square root process using a right shift operation.

By performing this type of processing, it is possible to calculate the square root without using division, which is computationally expensive. Furthermore, even if the range of input a is wide and the input value is large, the square root can be calculated without causing an overflow, even with registers and arithmetic units with a limited number of bits.

In this explanation, an example was shown in which N=8 and 16 bits were added as the fixed fraction, but other numbers of bits are also acceptable, such as N=7, N=6, N=5, etc., and the fixed fraction may be 14 bits, 12 bits, 10 bits, etc. Furthermore, if the system has registers or arithmetic units that exceed 64 bits, such as 128 bits, N=9 (18-bit fixed fraction) or more may be used. The variable fraction is also not limited to (bits(a)≫1) bits, and may be of other numbers of bits.

Furthermore, if the number of bits in the input a is small and repeated multiplications do not result in an overflow, the number of shift operations can be reduced by not performing a right shift operation after each multiplication, but performing a shift operation only at the end of the process. In this case, the fractional part added by the left shift can be reduced, or it can be omitted altogether.

Furthermore, the lower bits of input a or the decimal point originally contained in input a may be used as part or all of the fixed decimal point shown in another specific example of the square root calculation process (sqrt). This allows processing to be performed using registers and arithmetic units with a smaller number of bits, reducing the number of shift operations.

If a highly accurate square root cannot be obtained, the process can be terminated at the first iteration using the recurrence formula, and the square root can be calculated as root = (a x1) >> (16 + (bits(a) >> 1)). This allows the square root to be obtained without performing additional multiplications, reducing the amount of processing.

Conversely, to find a square root with even higher precision, the iterative process can be performed k times, with the second and subsequent iterative processes using the recurrence formula being performed (k-1) times, and the final square root being root=(a・xn)≫(16+(bits(a)≫1)). In that case, a highly precise square root can be found by repeating the second iterative process using the recurrence formula in another specific example of the square root calculation process (sqrt) as follows:

Furthermore, the number of iterations k of the recurrence formula may be switched to a different value depending on the profile or level of the standard. For example, in the case of profile A, which targets high image quality, the device may use a large value for the number of iterations of the recurrence formula to improve calculation accuracy. Specifically, a large value such as k=5 may be used.

On the other hand, for Profile B, which targets low processing loads, the device may use a small value for the number of iterations in the recurrence formula to reduce processing load. For example, a small value such as k=2 may be used. In this way, by specifying the number of iterations in the recurrence formula according to the profile or level of the standard, the decoding device can correctly decode the bitstream by selecting an appropriate number of iterations based on the profile or level information contained in the bitstream.

In addition, for a certain profile C, the number of iterations may be set to k=1. In this case, one iteration can be performed by a right bit shift, reducing the amount of processing without requiring multiplication.

Furthermore, the number of iterations k of the recurrence formula may be added to the header of the bitstream. In this way, by adding the number of iterations k used by the encoding device to the header and encoding it, the decoding device can obtain the number of iterations k from the header. The decoding device can then perform decoding processing using the same value as the number of iterations used by the encoding device, thereby properly decodeing the bitstream.

The specific example of the square root calculation process (SQRT) described in this embodiment and other specific examples may be applied not only to the calculation of fine prediction values, but also to other processes that require square root calculations. In this case, as shown in these examples of square root calculations, by setting the initial value of the recurrence formula to a value that is a power of 2, it becomes possible to implement the division process or multiplication process in the first iteration process using the recurrence formula with a shift operation, thereby reducing the amount of processing.

FIG. 94 is a diagram showing another example of the configuration of an encoding device according to an embodiment. FIG. 95 is a flowchart showing another example of an encoding method performed by an encoding device according to an embodiment.

The encoding device 1550 includes a circuit 1551 and a memory 1552 connected to the circuit 1551. The encoding device 1550 may perform the processing described in Figures 71 to 93.

Circuit 1551 performs the following operations:

Circuit 1551 acquires multiple vertices included in a three-dimensional mesh (S1561). Circuit 1551 predictively encodes one of the multiple vertices (S1562). In the predictive encoding, a first predicted value for the one vertex is calculated with decimal precision, the number of bits in the decimal part of the first predicted value is reduced to calculate a second predicted value, and the one vertex is predictively encoded using the second predicted value.

By reducing the number of bits in the decimal part of the predicted value calculated with decimal precision, it is possible to reduce the amount of data while controlling the accuracy during calculation. Reducing the number of bits reduces the amount of encoding processing, thereby improving processing efficiency.

For example, when calculating the first predicted value, two or more third predicted values are calculated with decimal precision based on each of the two or more processed vertices, and the first predicted value is calculated by integrating the two or more third predicted values.

By combining multiple predicted values, it is possible to obtain a highly accurate predicted value. This reduces prediction errors and improves coding efficiency. Furthermore, combining multiple predicted values allows for greater flexibility in the method for selecting an appropriate predicted value.

For example, when calculating the first predicted value, two or more third predicted values are integrated with decimal precision.

By performing the integration process with decimal precision and then converting to integers later, it is possible to perform the encoding process while maintaining the precision of the integration process. By performing high-precision integration and then converting to integers, it is possible to suppress an increase in prediction error.

For example, the two or more third predicted values and the first predicted value each have a decimal part with a predetermined number of bits. The second predicted value is calculated by reducing the decimal part of the first predicted value by a predetermined number of bits.

This allows for easy integer conversion by reducing the decimal portion by a predetermined number of bits. Reducing the number of bits reduces the amount of code required, improving the efficiency of calculation processing.

For example, the first predicted value is the average of two or more third predicted values.

This allows the overall prediction error to be reduced by averaging multiple predicted values. Averaging can potentially cancel out noise components and prediction errors, improving coding accuracy.

For example, in predictive coding, the specified number of bits is not reduced until the second predicted value is calculated.

This allows the accuracy of the predicted value to be maintained by not reducing the number of bits before converting to integers. Since there is no loss of accuracy during calculation, high-precision encoding can be achieved.

For example, when predictively encoding one vertex using the second predicted value, the second predicted value is rounded to calculate a fourth predicted value, and the fourth predicted value is used to predictively encode the one vertex.

This allows rounding to be performed to reduce decimal point errors. This improves data accuracy after encoding and minimizes errors when reducing the number of bits.

For example, in the rounding process, the second predicted value is rounded in a different direction depending on whether the second predicted value is a positive number or a negative number.

This allows for improved encoding accuracy by switching the rounding direction depending on whether the predicted value is positive or negative. In particular, it allows for optimal rounding processing based on the magnitude and sign of the predicted value.

Figure 96 is a diagram showing another example of the configuration of a decoding device according to an embodiment. Figure 97 is a flowchart showing another example of a decoding method performed by a decoding device according to an embodiment.

The decoding device 1560 includes a circuit 1561 and a memory 1562 connected to the circuit 1561. The decoding device 1560 may perform the processing described in Figures 71 to 93.

Circuit 1561 performs the following operations:

Circuit 1561 acquires multiple vertices included in a three-dimensional mesh (S1571). Circuit 1561 predictively decodes one of the multiple vertices (S1572). In the predictive decoding, a first predicted value for the one vertex is calculated with decimal precision, the number of bits in the decimal part of the first predicted value is reduced to calculate a second predicted value, and the one vertex is predictively decoded using the second predicted value.

By reducing the number of bits in the decimal part of the predicted value calculated with decimal precision, it is possible to reduce the amount of data while controlling the accuracy during calculation. Reducing the number of bits reduces the amount of decoding processing, thereby improving processing efficiency.

For example, when calculating the first predicted value, two or more third predicted values are calculated with decimal precision based on each of the two or more processed vertices, and the first predicted value is calculated by integrating the two or more third predicted values.

By combining multiple predicted values, a highly accurate predicted value can be obtained. This reduces decoding errors and improves decoding accuracy.

For example, when calculating the first predicted value, two or more third predicted values are integrated with decimal precision.

This allows for more accurate predictions to be calculated by integrating with decimal precision. By converting the values to integers later, the decoding process can be performed efficiently.

For example, the two or more third predicted values and the first predicted value each have a decimal part with a predetermined number of bits. The second predicted value is calculated by reducing the decimal part of the first predicted value by a predetermined number of bits.

This reduces the number of bits in the decimal part and converts it to an integer, thereby reducing the burden of the decoding process and enabling the use of highly accurate prediction values.

For example, the first predicted value is the average of two or more third predicted values.

This allows prediction errors to be reduced by averaging multiple predicted values. The averaging process reduces the variance in predicted values, enabling highly accurate decoding.

For example, in predictive decoding, the specified number of bits is not reduced before the second predicted value is calculated.

This means that there is no loss of accuracy due to bit reduction until integer conversion occurs during the decoding process, allowing for high-precision decoding.

For example, when predictively decoding one vertex using the second predicted value, the second predicted value is rounded to calculate a fourth predicted value, and the fourth predicted value is used to predictively decode the one vertex.

By applying rounding, errors can be minimized when reducing the number of bits, allowing highly accurate prediction values to be obtained during the decoding process.

For example, in the rounding process, the second predicted value is rounded in a different direction depending on whether the second predicted value is a positive number or a negative number.

This allows the accuracy of the predicted value to be improved by adjusting the direction of the rounding process depending on whether the predicted value is positive or negative. It also reduces the occurrence of rounding errors.

Furthermore, not all of the components described in this embodiment are always required, and only some of the components of the encoding device and decoding device disclosed herein may be included.

This disclosure is useful, for example, for encoding devices, decoding devices, transmitting devices, and receiving devices related to three-dimensional meshes, and is applicable to computer graphics systems and three-dimensional data display systems, etc.

100 Encoding device 101, 121, 144 Vertex information encoder 102, 145 Connection information encoder 103, 122 Attribute information encoder 104, 204, 1103 Preprocessor 105, 205, 2106 Postprocessor 110 Three-dimensional data encoding system 111, 211 Controller 112, 212 Input/output processor 113 Three-dimensional data encoder 114 System multiplexer 115 Three-dimensional data generator 123 Metadata encoder 124 Multiplexer 131 Vertex image generator 132 Attribute image generator 133 Metadata generator 134 Video encoder 141 Two-dimensional data encoder 142 Mesh data encoder 143 Texture encoder 148 Description encoder 151, 251 Circuit 152, 252 Memory 200 Decoding device 201, 221, 244 Vertex information decoder 202, 245 Connection information decoder 203, 222 Attribute information decoder 210 3D data decoding system 213 3D data decoder 214 System demultiplexer 215, 247 Presenter 216 User interface 223 Metadata decoder 224 Demultiplexer 231 Vertex information generator 232 Attribute information generator 234 Video decoder 241 2D data decoder 242 Mesh data decoder 243 Texture decoder 246 Mesh reconstructor 248 Description decoder 300 Network 310 External connector 511 Volumetric capturer 512 Projector 513 Base mesh encoder 514 Displacement encoder 515 Attribute encoder 516 Other Type Encoder 613 Base Mesh Decoder 614 Displacement Decoder 615 Attribute Decoder 616 Other Type Decoder 617 3D Reconstructor 1101 Input Mesh 1102 Attribute Map 1104 Base Mesh 1105 Displacement Data 1106 Compressor 1107 Bitstream 1108 Metadata 1206 Subdivider 1207 Displacement Vector Calculator 1231 Demultiplexer 1232 Switch 1233 Static Mesh Decoder 1234 Mesh Buffer 1235 Motion Decoder 1236 Base Mesh Reconstructor 1237 Inverse Quantizer 1238, 1243 Video Decoder 1239 Image Unpacker 1240 Inverse Quantizer 1241 Inverse Wavelet Transformer 1242 Reconstructor 1244 Color Transformer 1251 Decoded base mesh 1252 Subdivider 1253 Subdivided mesh 1254 Decoded displacement data 1255 Displacer 1256 Decoded 3D mesh 1301 Mesh frame 1302 Base mesh frame 1303 Displacement information 1304 Bitstream 1501 Set of vertex coordinates 1502 Set of 2D coordinates 1530, 1550 Encoder 1531, 1541, 1551 Circuitry 1532, 1542, 1552 Memory 1540, 1560 Decoder 2101 Bitstream 2102 Decompressor 2103 Base mesh 2104 Displacement data 2105 Metadata 2106 Postprocessor 2107 Output mesh 2108 Attribute map 2204 Subdivider 2209 Reconstructor 2301 Base mesh frame 2302 Base mesh frame 2303 Displacement information 2304 Mesh frame

Claims (18)

1. An encoding method performed by an encoding device, comprising:
Obtain multiple vertices contained in a three-dimensional mesh,
predictively encoding one vertex of the plurality of vertices;
In the predictive coding,
calculating a first predicted value of the one vertex with decimal precision;
calculating a second predicted value by reducing the number of bits of the fractional part of the first predicted value;
and predictively encoding the one vertex using the second predicted value. In calculating the first predicted value,
calculating two or more third predicted values with decimal precision based on each of the two or more processed vertices;
The encoding method according to claim 1 , further comprising: calculating a first predicted value by integrating the two or more third predicted values. The encoding method according to claim 2 , wherein the calculation of the first predicted value comprises integrating the two or more third predicted values with decimal precision. the two or more third predicted values and the first predicted value have a decimal part with a predetermined number of bits;
The encoding method according to claim 2 , wherein the calculation of the second predicted value comprises calculating the second predicted value by reducing the predetermined number of bits of the fractional part of the first predicted value. The encoding method according to claim 2 , wherein the first predicted value is an average of the two or more third predicted values. The encoding method according to claim 4 , wherein in the predictive encoding, the predetermined number of bits is not reduced until the second predicted value is calculated. In the predictive coding of the one vertex using the second predicted value,
calculating a fourth predicted value by rounding the second predicted value;
The encoding method according to claim 2 , further comprising predictively encoding the one vertex using the fourth predicted value. The encoding method according to claim 7 , wherein the rounding process rounds the second predicted value in a different direction depending on whether the second predicted value is a positive number or a negative number. A decoding method performed by a decoding device, comprising:
Obtain multiple vertices contained in a three-dimensional mesh,
predictively decoding one vertex of the plurality of vertices;
In the predictive decoding,
calculating a first predicted value of the one vertex with decimal precision;
calculating a second predicted value by reducing the number of bits of the fractional part of the first predicted value;
a decoding method for predictively decoding the one vertex using the second predicted value. In calculating the first predicted value,
calculating two or more third predicted values with decimal precision based on each of the two or more processed vertices;
The decoding method according to claim 9 , further comprising calculating the first predicted value by integrating the two or more third predicted values. The decoding method according to claim 10 , wherein the calculation of the first predicted value comprises integrating the two or more third predicted values with decimal precision. the two or more third predicted values and the first predicted value have a decimal part with a predetermined number of bits;
The decoding method according to claim 10 , wherein the calculation of the second predicted value comprises calculating the second predicted value by reducing the predetermined number of bits of the fractional part of the first predicted value. The decoding method according to claim 10 , wherein the first predicted value is an average of the two or more third predicted values. The decoding method according to claim 12 , wherein in the predictive decoding, the predetermined number of bits is not reduced until the second predicted value is calculated. In the predictive decoding of the one vertex using the second predicted value,
calculating a fourth predicted value by rounding the second predicted value;
The decoding method according to claim 10 , further comprising predictively decoding the one vertex using the fourth predicted value. The decoding method according to claim 15 , wherein the rounding process rounds the second predicted value in a different direction depending on whether the second predicted value is a positive number or a negative number. The circuit and
a memory connected to the circuit;
The circuit, in operation,
Obtain multiple vertices contained in a three-dimensional mesh,
predictively encoding one vertex of the plurality of vertices;
In the predictive coding,
calculating a first predicted value of the one vertex with decimal precision;
calculating a second predicted value by reducing the number of bits of the fractional part of the first predicted value;
and a coding device that predictively codes the one vertex using the second predicted value. The circuit and
a memory connected to the circuit;
The circuit, in operation,
Obtain multiple vertices contained in a three-dimensional mesh,
predictively decoding one vertex of the plurality of vertices;
In the predictive decoding,
calculating a first predicted value of the one vertex with decimal precision;
calculating a second predicted value by reducing the number of bits of the fractional part of the first predicted value;
A decoding device that predictively decodes the one vertex using the second predicted value.