WO2024232339A1 - Transmission device, decoding device, transmission system, transmission method, and transmission program - Google Patents

Abstract

The present invention makes it possible to reduce a cost and a reception period of point group data. In the present invention, a server includes: an acquisition unit that acquires environmental information on an environment around the server; a determination unit that determines, on the basis of the environmental information acquired by the acquisition unit, a transmission target from point group data including at least one of first point group data and second point group data; and a communication unit that transmits, to a decoding device, the transmission target determined by the determination unit.

Description

TRANSMITTING APPARATUS, DECODING APPARATUS, TRANSMITTING SYSTEM, TRANSMITTING METHOD, AND TRANSMITTING PROGRAM

The present invention relates to a transmitting device, a decoding device, a transmitting system, a transmitting method, and a transmitting program.

Technologies for transmitting point cloud data are known (see, for example, Patent Documents 1 and 2). Such technologies include a technique in which point cloud data is divided based on space and resolution, and the resulting encoded divided point cloud data is transmitted to a decoding device.

One example is a technology that divides point cloud data into uncompressed divided point cloud data that includes coordinate information such as xyz and color information such as RGB (Red-Green-Blue), transmits the encoded uncompressed divided point cloud data to a decoding device, and then causes the decoding device to decompress and display the decoded point cloud data. Another example is a technology that compresses the divided point cloud data using GPCC (Geometry based Point Cloud Compression), which is a method of compressing point cloud data, saves it as compressed compressed divided point cloud data, transmits the encoded compressed divided point cloud data to a decoding device, and then causes the decoding device to decompress and display the decoded point cloud data.

International Publication No. 2020/071116 International Publication No. 2019/078000

However, the above-mentioned technology may not be able to reduce the reception time or cost of point cloud data. For example, if the uncompressed divided point cloud data divided from point cloud data based on resolution is low resolution, the number of point cloud data (number of points) is small and the size (file size) is small. On the other hand, if this uncompressed divided point cloud data is high resolution, the number of points is large and the size is large. In the above-mentioned situation, if the uncompressed divided point cloud data is compressed by GPCC for each resolution level and transmitted to a decoding device and decoded by the decoding device, it may take a long time to receive the point cloud data by the decoding device and may cost money to store, decode, etc.

For example, if uncompressed divided point cloud data is small-sized, low-resolution point cloud data, but is compressed by GPCC and transmitted to a decoding device, there is no benefit of reducing the amount of data through GPCC compression, and the decoding cost by the decoding device is high. In an environment where the transmission rate is high (communication speed is fast) and the performance of the decoding device is low, it becomes clear that the decoding cost exceeds the reception time. In an environment where the transmission rate is low (communication speed is slow), it becomes clear that the reception time exceeds the decoding cost. On the other hand, if uncompressed divided point cloud data is large-sized, high-resolution point cloud data, but is not compressed by GPCC and transmitted to a decoding device, it takes a long time to receive. One aspect of the present disclosure makes it possible to reduce the reception period and cost of point cloud data.

A transmitting device according to one aspect of the present disclosure is a transmitting device having an acquisition unit that acquires environmental information related to the environment surrounding the transmitting device, a determination unit that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit, and a transmitting unit that transmits the transmission target determined by the determination unit to a decoding device.

A decoding device according to one aspect of the present disclosure includes an acquisition unit that acquires environmental information related to the surrounding environment of the decoding device, a determination unit that determines a transmission target from point cloud data that includes at least one of first point cloud data and second point cloud data based on the environmental information, and a receiving unit that receives the transmission target determined by the determination unit from a transmitting device.

A transmission system according to one aspect of the present disclosure includes a transmission device and a decoding device, the transmission device includes an acquisition unit that acquires environmental information related to the environment surrounding the transmission device, a determination unit that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit, and a transmission unit that transmits the transmission target determined by the determination unit to a decoding device, and the decoding device includes a receiving unit that receives the transmission target determined by the determination unit from the transmission device.

A transmission method according to one aspect of the present disclosure is a transmission method executed by a transmission device, and includes an acquisition step of acquiring environmental information related to the environment surrounding the transmission device, a determination step of determining a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition step, and a transmission step of transmitting the transmission target determined by the determination step to a decoding device.

A transmission program according to one aspect of the present disclosure causes a computer mounted on a transmitting device to execute an acquisition process for acquiring environmental information relating to the environment surrounding the transmitting device, a determination process for determining a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition process, and a transmission process for transmitting the transmission target determined by the determination process to a decoding device.

FIG. 1 is a diagram illustrating an example of a configuration of a transmission system according to a first embodiment. FIG. 2 is a diagram for explaining point cloud data. FIG. 2 is a diagram for explaining point cloud data. FIG. 1 is a diagram for explaining a transmission system according to a first embodiment. FIG. 1 is a diagram for explaining a transmission system according to a first embodiment. FIG. 1 is a diagram for explaining a transmission system according to a first embodiment. FIG. 4 is a flow chart showing an example of processing by a server according to the first embodiment; FIG. 4 is a flow chart showing an example of processing by a server according to the first embodiment; FIG. 13 is a diagram for explaining a transmission system according to a modified example. FIG. 11 is a diagram illustrating an example of a configuration of a transmission system according to a second embodiment. FIG. 11 is a diagram for explaining a transmission system according to a second embodiment. FIG. 11 is a diagram for explaining a transmission system according to a second embodiment. FIG. 11 is a flowchart illustrating an example of a process of a client according to the second embodiment. FIG. 2 illustrates an example of a hardware configuration of the apparatus. FIG. 1 is a diagram for explaining a transmission system according to a reference technique. FIG. 1 is a diagram for explaining a transmission system according to a reference technique. FIG. 1 is a diagram for explaining a transmission system according to a reference technique.

The following describes in detail the embodiments of the present disclosure with reference to the drawings. In the following embodiments, the same elements are given the same reference numerals and duplicate descriptions may be omitted.

The present disclosure will be described in the following order.
0. Introduction 1. First embodiment 2. Modification 3. Second embodiment 4. Example of hardware configuration 5. Example of effects

0. Introduction As described above, a technique for transmitting point cloud data is known in which point cloud data is divided based on space and resolution, and the encoded divided point cloud data is transmitted to a decoding device. For example, in such a technique, the decoding device decodes the transmitted encoded uncompressed divided point cloud data into uncompressed divided point cloud data and displays it on a PC (Personal Computer) or the like. As an example, when the decoding device superimposes and displays uncompressed divided point cloud data related to a predetermined object on a satellite image, the decoding device selects a box required for display from boxes (voxel data) prepared in advance, thereby achieving a high-speed response with a short display response until the uncompressed divided point cloud data is displayed. Before describing the embodiments of the present disclosure, a reference technique will be described as an example of such a technique using Figs. 15 to 17. Figs. 15 to 17 are diagrams for describing a transmission system related to the reference technique.

In the example shown in FIG. 15, a server of a transmission system according to the reference technology transmits to a client only encoded RAW files including encoded RAW data, which is encoded uncompressed point cloud data obtained by encoding uncompressed point cloud data such as uncompressed 3D (Dimensions) point cloud data. The transmission system according to the reference technology causes the server to prepare and causes the client to display. For example, a point cloud spatial division resolution division encoding processing unit (Point-cloud-converter) of the server generates an encoded RAW file group consisting of multiple encoded RAW files in which an uncompressed point cloud file including uncompressed point cloud data is divided by space and resolution.

The client identifies a file from the group of encoded RAW files that corresponds to the gaze point, which is the ROI (Region of Interest) area, over the network (communication path), and converts this file transferred from the server from 3D to 2D and displays it. Specifically, the client performs the following processing on the web browser. First, the client displays HTML. Next, the client reads out a program. For example, the client reads out and executes JavaScript (registered trademark). Next, the client acquires and displays a satellite photo as the background. Next, the client displays the transmission target to be displayed from the group of encoded RAW files. For example, the client requests the transmission target from the server, receives the transmission target from the server, decodes the received transmission target, renders the decoded transmission target, and repeats the above processing until there are no more transmission targets to display.

In this case, the client receives and displays the transmission target, which includes encoded and uncompressed 3D point cloud data transferred from the server over the network. For this reason, the network communication bandwidth has a large impact on the display response by the client, and depending on the communication bandwidth, it may not be possible to reduce the time and cost required to receive the point cloud data. For example, in environments with low communication bandwidth or transmission rates, the long transmission times become apparent. Furthermore, even if the encoded RAW file is a large, high-resolution file, transmission times are particularly long if compression processing is not performed before transmission to the client.

In the example shown in FIG. 16, a server of a transmission system according to the reference technology transmits to a client only encoded GPCC files including encoded GPCC data, which is encoded compressed point cloud data obtained by encoding compressed point cloud data such as uncompressed 3D (Dimensions) point cloud data. The transmission system according to the reference technology causes the server to prepare and causes the client to display. For example, a point cloud spatial division resolution division encoding processing unit (Point-cloud-converter) of the server generates an encoded GPCC file group consisting of multiple encoded GPCC files in which a compressed point cloud file including compressed point cloud data is divided by space and resolution.

The client identifies a file corresponding to the gaze point from the group of encoded GPCC files over the network, decodes and expands this file transferred from the server to uncompressed data, converts it from 3D to 2D, and displays it. Specifically, the client performs the following process on the web browser. First, the client displays HTML. Next, the client reads a program. For example, the client reads and executes JavaScript. Next, the client acquires and displays a satellite photo as the background. Next, the client assembles software for a decoder such as an LLVM (Low Level Virtual Machine). Next, the client displays the transmission target to be displayed from the group of encoded GPCC files. For example, the client requests the transmission target from the server, receives the transmission target from the server, decodes and expands the received transmission target to uncompressed data, renders the expanded transmission target, and repeats the above process until there are no more transmission targets to display. Note that GPCC compression reduces the number of point cloud data (number of point clouds) and degrades color information due to quantization of coordinates and color information according to compression parameters, so even if the client decodes and expands the data to be sent back to pre-compression data, it is not necessarily expanded to the original size.

In this case, the client receives the transmission target including the encoded and compressed 3D point cloud data transferred from the server over the network, expands it, and displays it. For this reason, the network communication bandwidth has a small impact on the display response, but the client's performance, such as decoding ability, has a large impact on the display response. For this reason, depending on the client's performance, it may not be possible to reduce the reception period or cost of point cloud data. For example, in an environment where the communication bandwidth or transmission rate is high and the client's performance is low, it becomes clear that the decoding cost exceeds the transmission time. Also, if an encoded GPCC file is transmitted to a client despite its small size and low resolution, there is no benefit to reducing the amount of data through compression, and the decoding cost is particularly high.

Next, an example of execution of the transmission system according to the reference technology will be described with reference to FIG. 17. In the example shown in FIG. 17, an example of execution in which the client of the transmission system according to the reference technology displays only RAW files and an example of execution in which the client displays only GPCC files are compared side by side. As shown in FIG. 17, the display response of RAW files is good for about one second after the client starts displaying the transmission target, but in the latter half, as the number of point clouds increases, the display response of GPCC files improves. Thus, in the example of execution shown in FIG. 17, in the transmission system that displays GPCC, the cost of the client decoding the GPCC files increases in the latter half as the number of point clouds increases.

In contrast, a transmission system according to an aspect of the present disclosure determines a combination of first point cloud data such as a RAW file and second point cloud data such as a GPCC file based on environmental information related to the environment, thereby enabling a reduction in the reception period and cost of point cloud data. For example, a transmission system according to an aspect of the present disclosure determines (selects) a transmission target to display a RAW file in the first half and a GPCC file in the second half, thereby speeding up the display speed of the transmission target and shortening responses such as display responses. In addition, while the optimal combination of a RAW file and a GPCC file changes even for the same server and client depending on environmental information related to the communication bandwidth during transmission, etc., a transmission system according to an aspect of the present disclosure can change to an appropriate combination according to the environmental information. Specific techniques will be described in the following embodiments.

1. First embodiment Fig. 1 is a diagram showing an example of the configuration of a transmission system according to a first embodiment. The transmission system 1 includes a server (transmission device) 10 and a client (decoding device) 20. The transmission system 1 acquires environmental information using the server 10, determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information, and transmits the transmission target to the client 20.

Environmental information refers to information about the environment surrounding the server 10 and the client 20. Details of environmental information will be described later. The first point cloud data is realized by, but is not limited to, encoded uncompressed point cloud data in which uncompressed point cloud data is encoded, such as encoded RAW data. The second point cloud data is realized by, but is not limited to, encoded compressed point cloud data in which compressed point cloud data is encoded, such as encoded GPCC data.

At least one of the first point cloud data and the second point cloud data may include 3D point cloud data and at least one of a plurality of divided point cloud data divided based on at least one of space and resolution. The divided point cloud data is not particularly limited as long as at least one of the first point cloud data and the second point cloud data is divided. The divided point cloud data may be, for example, first point cloud data such as uncompressed point cloud data that has been compressed after division, or first point cloud data such as divided uncompressed point cloud data that has been combined and then compressed.

Below, an example of point cloud data will be described with reference to FIG. 2 and FIG. 3. FIG. 2 and FIG. 3 are diagrams for explaining point cloud data. In the example shown in FIG. 2, an uncompressed point cloud file including uncompressed point cloud data as first point cloud data is divided into a plurality of RAW files, which are a plurality of uncompressed divided point cloud files, by performing space division for each space and resolution division (frequency decomposition) for each resolution level by the server 10. In addition, the uncompressed point cloud file is divided into resolution levels from Level 0 to Level 30 by the server 10, and is divided into eight RAW files as the resolution level increases by 1. Then, the server 10 performs compression processing by GPCC on half of the plurality of RAW files so that the ratio of RAW files to GPCC files, which are compressed divided point cloud files, is 1:1, and the RAW files and GPCC files are encoded.

In the above example, the data structure of the point cloud data is a mixture of RAW files and GPCC files for each file (file unit), but for example, RAW data and GPCC data may be mixed for each box in each RAW file. Also, in the above example, the data structure of the point cloud data is a 1:1 ratio of RAW files to GPCC files, but for example, the entire data structure may be GPCC files. Also, the compression parameters of the RAW files and GPCC files may be the same or different. For example, the data structure of the point cloud data may be a mixture of GPCC files with multiple different compression parameters such as GPCC 1/GPCC 2/.../GPCC 2/RAW and RAW files with a single compression parameter. Also, in the above example, the uncompressed point cloud file is divided by the server 10 into resolution levels from Level 0 to Level 30, but the resolution division level does not have to be from 0 to 30.

In the above example, the RAW file may be several thousand files in the ".xyz" format, which contains coordinate information such as xyz and color information such as RGB (Red-Green-Blue), has a small number of points, and has a hierarchical structure realized by Dir (Directory). GPCC files may be files in the ".bin (binary)" format, which has an internal octree structure in addition to the characteristics of the ".xyz" format file, in which the coordinate information such as xyz and color information contained in the RAW file are compressed by a compression process using GPCC. An example of a RAW file and a GPCC file will be described in detail below with reference to Figure 3.

In the example shown in FIG. 3, the RAW file and GPCC file use a data format for point cloud space division resolution division encoding processing. The RAW file includes a header for point cloud space division resolution division encoding processing, coordinate information such as xyz, and color information such as RGB. Transmission processing and display processing are mainly performed on the RAW file. In the transmission processing, the time it takes for the server 10 to transmit the RAW file to the client 20 is proportional to the file size, and therefore directly proportional to the number of point clouds. In the display processing of the RAW file, 3D to 2D rendering is performed by the client 20.

On the other hand, the GPCC file includes a header for point cloud spatial division resolution division encoding processing, a GPCC header, a slice header, etc. For example, in addition to transmission processing and display processing, preparation processing and decoding processing are mainly performed on the GPCC file. In the transmission processing, the transmission time of the GPCC from the server 10 to the client 20 depends on the compression rate and is therefore proportional to the file size. In the preparation processing, the client 20 receives the encoded GPCC file transmitted (transferred) from the server 10, compiles the LLVM, and initializes the execution binary. In the decoding processing, the client 20 performs a decoding and decompression processing in which the data is decoded so as to be decompressed. In the display processing, the client 20 performs rendering from 3D to 2D.

As shown in the relationship between the number of point clouds and size in Figure 3, the file size of a RAW file increases in direct proportion to the increase in the number of point clouds from the initial size of the header for the point cloud spatial division resolution division encoding process. On the other hand, the file size of a GPCC increases rapidly from the initial size of the GPCC header in proportion to the increase in the number of point clouds until halfway through, and then increases more slowly. As shown in the relationship between the number of point clouds and decompression processing time in Figure 3, the decompression processing time of a RAW file increases gradually in direct proportion to the increase in the number of point clouds. On the other hand, the decompression processing time of a GPCC file increases rapidly in response to the preparation processing, then increases significantly in proportion to the number of point clouds until halfway through, and then increases more slowly.

The transmission system 1 makes it possible to reduce the time and cost required to receive point cloud data by varying the ratio of the RAW files and GPCC files in accordance with environmental information relating to their data structures and attributes. Next, returning to FIG. 1, the configuration of the server 10 and client 20 of the transmission system 1 will be described in detail.

The server 10 acquires environmental information, determines a transmission target from point cloud data that includes at least one of the first point cloud data and the second point cloud data based on the environmental information, and transmits the transmission target to the client 20. In the example shown in FIG. 1, the server 10 has a control unit 11, a communication unit (transmission unit) 12, and a storage unit 13.

The control unit 11 controls the entire server 10. For example, the control unit 11 is configured with one or more processors having programs that define each processing procedure and internal memory that stores control data, and the processor executes each process using the programs and internal memory. Examples of the control unit 11 include electronic circuits such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit), as well as integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array). In the example shown in FIG. 1, the control unit 11 has a division unit 111, an encoding unit 112, an acquisition unit 113, and a determination unit 114.

The division unit 111 divides at least one of the first point cloud data and the second point cloud data into a plurality of divided point cloud data based on at least one of space and resolution. For example, the division unit 111 divides a point cloud file including first point cloud data such as uncompressed point cloud data into a plurality of RAW files including RAW data for each space and each resolution level.

The encoding unit 112 encodes each of the multiple divided point cloud data divided by the division unit 111. For example, the encoding unit 112 encodes the multiple uncompressed divided point cloud data into multiple encoded uncompressed divided point cloud data, and compresses and encodes the multiple uncompressed divided point cloud data into multiple encoded compressed divided point cloud data. The encoding unit 122 may combine the multiple uncompressed divided point cloud data and encode the combined uncompressed point cloud data, or may encode compressed point cloud data that has been compressed in advance. As an example, the encoding unit 112 encodes a RAW file including multiple RAW data into an encoded RAW file, and encodes a GPCC file including multiple GPCC data into an encoded GPCC file.

The acquisition unit 113 acquires environmental information related to the environment around the server 10. For example, the acquisition unit 113 acquires environmental information including at least one of communication environment information related to communication between the server 10 and the client 20, server environment information (transmission device environment information) related to the server 10, client environment information (decoding device environment information) related to the client 20, and point cloud data environment information related to the point cloud data. The acquisition unit 113 may acquire environmental information including communication environment information related to the performance of the communication path between the server 10 and the client 20. The acquisition unit 113 may acquire environmental information including communication environment information related to at least one of the communication speed and communication bandwidth of the communication path. The acquisition unit 113 may acquire environmental information including client environment information related to at least one of the performance, load, decoding period, reception status, and display status of the client 20. The acquisition unit 113 may acquire environmental information including client environment information related to the decoding speed of the client 20. The acquisition unit 113 may acquire environmental information including client environment information related to the reception status of the encoded compressed point cloud data by the client 20.

The acquisition unit 113 may acquire environmental information including server environment information related to at least one of the performance, load, and transmission period of the server 10. The acquisition unit 113 may acquire environmental information including server environment information related to the service capacity of the server 10. The acquisition unit 113 may acquire environmental information including point cloud data environmental information related to the attributes of the point cloud data requested by the client 20. The acquisition unit 113 may acquire point cloud data environmental information related to at least one of the accuracy, size, resolution, number, space, and ROI area of the point cloud data. Information related to the accuracy of the point cloud data refers to, for example, coordinate accuracy related to the coordinates of the point cloud data, color accuracy related to the color of the point cloud data, etc. The number of point cloud data (point cloud number) refers to, for example, the number of point cloud data in a specified space (i.e., the density of the point cloud data), and information on the distribution of the point cloud data, etc. The acquisition unit 113 may acquire environmental information including point cloud data environmental information related to the point cloud data requested by the client 20 as a transmission target.

The acquisition unit 113 may further acquire the acquisition target determined by the determination unit 114 as point cloud data. As one example, the acquisition unit 113 acquires point cloud data including only first point cloud data such as encoded uncompressed point cloud data. As another example, the acquisition unit 113 acquires point cloud data including only second point cloud data such as encoded compressed point cloud data. As another example, the acquisition unit 113 acquires point cloud data including both first point cloud data such as encoded uncompressed point cloud data and second point cloud data such as encoded compressed point cloud data.

In this case, the acquisition unit 113 may acquire the acquisition target from at least one of the first point cloud data and the second point cloud data, which are prepared in advance by a means such as generation and stored in the server 10. As an example, the acquisition unit 113 acquires the acquisition target from one of the first point cloud data such as encoded non-compressed point cloud data and the second point cloud data such as encoded compressed point cloud data, which are prepared in advance. As another example, the acquisition unit 113 acquires the acquisition target from both the first point cloud data such as encoded non-compressed point cloud data and the second point cloud data such as encoded compressed point cloud data, which are prepared in advance. The acquisition unit 113 may acquire the acquisition target from at least one of the first point cloud data such as encoded non-compressed point cloud data and the second point cloud data such as encoded compressed point cloud data, which are generated on demand after the acquisition target is determined. The acquisition unit 113 may acquire point cloud data including at least one of the first point cloud data such as encoded non-compressed point cloud data and the second point cloud data such as encoded compressed point cloud data, as the transmission target, in parallel with the display of the transmission target by the client 20. That is, the acquisition unit 113 may acquire the transmission target by an on-the-fly method.

The acquisition unit 113 may further acquire point cloud data. For example, the acquisition unit 113 acquires an uncompressed point cloud file that includes first point cloud data, such as uncompressed point cloud data that has not been divided or encoded. In the example shown in FIG. 1, the acquisition unit 113 is included in the control unit 11, but when acquiring point cloud data that has not been divided or encoded, the acquisition unit 113 may function as an input receiving unit that exists outside the control unit 11. Examples of such an acquisition unit 113 include a UI (User Interface) such as a button displayed on a display unit such as the desktop of a PC or a touch panel.

The determination unit 114 determines a transmission target from point cloud data including at least one of the first point cloud data and the second point cloud data, based on the environmental information acquired by the acquisition unit 113. An example of a determination by the determination unit 114 will be described below with reference to Figs. 4 to 6. Figs. 4 to 6 are diagrams for explaining the transmission system according to the first embodiment.

In the example shown in FIG. 4, the determination unit 114 determines a transmission target including both first point cloud data such as encoded non-compressed point cloud data and second point cloud data such as encoded compressed point cloud data from point cloud data including both first point cloud data such as encoded non-compressed point cloud data and second point cloud data such as encoded compressed point cloud data, which are generated in advance by spatially dividing and resolution dividing a point cloud file. Specifically, the determination unit 114 determines a transmission target including a mixture of RAW files and GPCC files from a mixed file including a mixture of RAW files and GPCC files. The determination unit 114 may or may not determine a transmission target including a mixture of RAW files and GPCC files from a mixed file including a mixture of RAW files and GPCC files acquired in advance by the acquisition unit 113. In a case other than the above, for example, the determination unit 114 determines an acquisition target to be acquired by the acquisition unit 113 based on the environmental information acquired by the acquisition unit 113.

Below, an example will be described with reference to FIG. 5 in which the determination unit 114 determines a transmission target from a mixed file containing a mixture of RAW files and GPCC files previously acquired by the acquisition unit 113. In the example shown in FIG. 5, the division unit 111 performs resolution division from Level 0 to Level 30 on the point cloud file, and then performs spatial division. The encoding unit 112 encodes the multiple divided point cloud files divided by the division unit 111 into an encoded RAW file, an encoded GPCC 1 file, and an encoded GPCC n file. The determination unit 114 determines a transmission target from these encoded files based on the environmental information acquired by the acquisition unit 113. Note that the resolution division level does not have to be from 0 to 30.

The determination unit 114 determines the transmission target based on information included in the environmental information acquired by the acquisition unit 113. As an example, the determination unit 114 determines the transmission target based on communication environment information included in the environmental information acquired by the acquisition unit 113. For example, the determination unit 114 determines the transmission target based on environmental information including communication environment information related to the communication bandwidth of the communication path between the server 10 and the client 20. When the communication bandwidth width is within a predetermined range, the determination unit 114 determines a mixed file containing a mixture of an encoded RAW file and an encoded GPCC file as the transmission target since the display response can be maintained. As another example, the determination unit 114 determines the transmission target based on client environment information included in the environmental information acquired by the acquisition unit 113. For example, the determination unit 114 determines the transmission target based on client environment information related to the load and decoding period of the client 20.

In this case, the determination unit 114 estimates the load related to decoding and rendering of the client 20, and the load related to rendering and the decoding period based on the number of point groups, the size of the Bounding (B) box, the encoding period of the encoding process, etc. As an example, the determination unit 114 estimates a value obtained by dividing the added value (total value) of the following (1) to (3) by the number of remaining threads that can be simultaneously processed by the CPU at the current time as the load related to decoding of the client 20. The determination unit 114 makes an estimate by appropriately correcting it according to the actual example, for example, assuming that the load related to compilation is 0 when a compiled LLVM is used as the decoding unit 221 of the client 20.
(1) The load associated with compiling the client 20, such as LLVM (for example, the product of the code size of the client 20 and the coefficient A)
(2) The load associated with parsing the encoded GPCC file (e.g., the product of the size of the encoded GPCC file and the coefficient B, etc.)
(3) Load related to decoding of the encoded GPCC file (e.g., the multiplication value of the number of points in the encoded GPCC file and the coefficient C, etc.)

The determination unit 114 may calculate the load related to the decoding of the client 20 based further on the software of the decoding unit 221, such as the state of the decoding software for the encoded GPCC file, etc. For example, the determination unit 114 adds a load corresponding to the startup time and initialization time of the decoding software of the decoding unit 221 to the load related to the decoding of the client 20. Specifically, when the decoding software is Web Assembly, the determination unit 114 adds a load based on the compilation of the LLVM and the time corresponding to the client 20 (the execution time of the decoding process by the decoding unit 221, the generation time of an image, etc.) to the load related to the client 20. The determination unit 114 also adds a load corresponding to the time required to initialize the decoding software to the load.

The determination unit 114 may calculate the load related to decoding of the client 20 based further on a load according to time that is directly or indirectly correlated with the load of the decoding process by the decoding unit 221. For example, the determination unit 114 adds a load according to time that is highly correlated with the load of the decoding process, such as the time related to the decoding process estimated from the stream size or the number of point clouds, to the load related to decoding of the client 20. As one example, the determination unit 114 adds a load according to the parsing process time based on the stream size to the load related to decoding of the client 20. Furthermore, the determination unit 114 adds a load according to the conversion process time of the transmission target based on the number of point clouds to the load related to decoding of the client 20.

The determination unit 114 may estimate the load of the client 20 by adding the load of the rendering process by the decryption unit 221 of the client 20 to the load of the decryption of the client 20 calculated as described above. In this case, the determination unit 114 estimates the load of the rendering process as the load of the rendering process corresponding to the rendering process time by the decryption unit 221 of the client 20. The determination unit 114 can determine an appropriate transmission target by determining the transmission target so that each of the processing times, such as the above-mentioned decoding process time and rendering process time, is minimized.

As another example, the determination unit 114 determines the transmission target based on the server environment information included in the environment information acquired by the acquisition unit 113. For example, the determination unit 114 determines the transmission target based on the server environment information related to the load of the server 10. In this case, when a transmission target prepared (generated) in advance on the server 10 side is transmitted to the client 20 via the communication unit 12, the communication unit 21, and the network, the determination unit 114 estimates the transmission rate of the server 10 weighted by the most recent transmission rate of the server 10. For each candidate transmission target, the determination unit 114 estimates the transmission time based on the data size of the candidate transmission target, and calculates the load of the server 10 according to each estimated transfer time. The determination unit 114 determines the candidate transmission target that will have the smallest load on the server 10 as the transmission target.

As another example, the determination unit 114 determines the transmission target based on the point cloud data environment information included in the environment information acquired by the acquisition unit 113. For example, the determination unit 114 may determine the transmission target based on the point cloud data environment information related to the size of the point cloud data included in the environment information. In this case, the determination unit 114 simply determines the small-sized point cloud data as the transmission target. Specifically, the determination unit 114 estimates the size of the encoded RAW file based on the attribute of the RAW file that the size of the RAW file is directly proportional to the number of point clouds. In addition, the determination unit 114 estimates the size of the encoded GPCC file based on the attribute of the GPCC file that the size of the GPCC file is the sum of the header value and the Geometry attribute and the Attribute attribute values proportional to the number of point clouds. The determination unit 114 may actually compare the sizes of the encoded RAW file and the encoded GPCC file and determine the small-sized file as the transmission target, or may determine the transmission target by estimating the size of the above-mentioned file from the size of the B box and the number of point clouds. For example, if the size of a mixed file containing a mixture of an encoded RAW file and an encoded GPCC file is small, the determination unit 114 determines this mixed file as the file to be sent. This reduces the size of the file to be sent (amount of data to be sent) sent from the communication unit 12, allowing the determination unit 14 to shorten the display response. Note that if point cloud data has been generated in advance, the determination unit 114 does not need to estimate the size of the encoded RAW file or the encoded GPCC file, and may determine the file to be sent by comparing the actual file sizes.

As another example, the determination unit 114 may determine the transmission target based on point cloud data environment information related to the ROI region included in the environment information. When the point cloud data is an ROI region, the determination unit 114 determines the transmission target to be an encoded RAW file in order to maintain the accuracy of the original point cloud data. When the point cloud data is a non-ROI region, the determination unit 114 determines the transmission target to be an encoded GPCC file in order to shorten the transmission time and response. For example, when the point cloud data includes both an ROI region and a non-ROI region, the determination unit 114 determines a mixed file in which an encoded RAW file and an encoded GPCC file are mixed as the transmission target. As another example, the determination unit 114 determines the transmission target based on multiple pieces of information included in the above-mentioned environment information. For example, the determination unit 114 determines the transmission target each time by comprehensively considering point cloud data environment information related to the size of the point cloud data included in the environment information, client environment information related to the load of the client 20, and the like. The determination unit 114 determines that the RAW/GPCC mixed file is to be sent, and outputs the RAW/GPCC mixed file to the communication unit 12.

Below, an example will be described with reference to FIG. 6 in which the determination unit 114 determines the acquisition target to be acquired by the acquisition unit 113 to be a mixed file containing a mixture of a RAW file and a GPCC file based on environmental information, and then determines the mixed file containing a mixture of a RAW file and a GPCC file as a transmission target. In the example shown in FIG. 6, similar to the example shown in FIG. 5, the division unit 111 performs resolution division and space division on the point cloud file. Thereafter, before encoding is performed by the encoding unit 112, the determination unit 114 determines the acquisition target to be acquired by the acquisition unit 113 based on the environmental information acquired by the acquisition unit 113. For example, the determination unit 114 determines the acquisition target in the same manner as the determination of the transmission target by the determination unit 114 based on the environmental information shown in FIG. 5. In the example shown in FIG. 6, the determination unit 114 determines the acquisition target to be an encoded RAW file and an encoded GPCC file.

After the determination unit 114 determines the acquisition target, the encoding unit 112 encodes the RAW file and the GPCC file acquired by the acquisition unit 113 into an encoded RAW file and an encoded GPCC file. The acquisition unit 113 acquires the encoded RAW file and the encoded GPCC file, and the determination unit 114 determines these files as the transmission target and outputs the transmission target to the communication unit 12.

Returning to the explanation of FIG. 1, the communication unit 12 communicates with the communication unit 21 of the client 20 via a telecommunications line such as a LAN (Local Area Network) or the Internet. The communication unit 12 transmits the transmission target determined by the determination unit 114 to the client 20. An example of the communication unit 12 is a NIC (Network Interface Card).

The storage unit 13 stores environmental information, the first point cloud data, the second point cloud data, transmission targets, the OS, various programs, etc. Examples of the storage unit 13 include storage devices such as HDDs (Hard Disk Drives), SSDs (Solid State Drives), and optical disks, as well as data-rewritable semiconductor memories such as RAMs (Random Access Memory), flash memories, and NVSRAMs (Non Volatile Static Random Access Memory).

The client 20 receives the transmission target from the server 10. In the example shown in FIG. 1, the client 20 has a communication unit (receiving unit) 21, a control unit 22, a display unit 23, and a storage unit 24.

The communication unit 21 communicates with the communication unit 12 of the server 10 via an electric communication line such as a LAN or the Internet. The communication unit 21 receives the transmission target determined by the determination unit 114 from the server 10. An example of the communication unit 32 is a NIC.

The control unit 22 controls the entire client 20. For example, the control unit 22 is configured with one or more processors having an internal memory in which programs that define each processing procedure and control data are stored, and the processor executes each process using the programs and internal memory. Examples of the control unit 22 include electronic circuits such as a CPU, MPU, and GPU, as well as integrated circuits such as an ASIC and FPGA. In the example shown in FIG. 1, the control unit 22 has a decoding unit 221 and a display control unit 222.

The decoding unit 221 decodes the transmission target received by the communication unit 21. For example, the decoding unit 221 identifies a file corresponding to the ROI region over the network, and decodes this file, which is the transmission target transferred from the server 10, according to the file type.

The display control unit 222 causes the display unit 23 to display the transmission object decoded by the decoding unit 221. For example, the display control unit 222 causes the display unit 23 to display the transmission object decoded by the decoding unit 221 and converted from 3D to 2D.

The display unit 23 displays the transmission target decoded by the decoding unit 221. An example of the display unit 23 is the desktop of the terminal.

The storage unit 24 stores the transmission target, the OS, various programs, etc. Examples of the storage unit 24 include storage devices such as HDDs, SSDs, and optical disks, as well as data-rewritable semiconductor memories such as RAMs, flash memories, and NVSRAMs.

Next, an example of each process executed by the server 10 according to the first embodiment will be described with reference to Figs. 7 and 8. Figs. 7 and 8 are flow diagrams showing an example of a process of the server according to the first embodiment. First, an example of a process of the server 10 according to the first embodiment will be described with reference to Fig. 7.

In step S1, the acquisition unit 113 acquires environmental information about the environment surrounding the server 10.

In step S2, the determination unit 114 determines a transmission target from point cloud data that includes at least one of the first point cloud data and the second point cloud data, based on the environmental information acquired by the acquisition unit 113.

In step S3, the communication unit 12 transmits the transmission target determined by the determination unit 114 to the client 20.

Next, an example of the decision process in step S2 of FIG. 7 will be described in detail with reference to FIG. 8.

In step S21, the determination unit 114 determines whether point cloud data environment information regarding the ROI region has been acquired by the acquisition unit 113.

If the determination unit 114 determines that the acquisition unit 113 has acquired point cloud data environmental information regarding the ROI region (Yes in step S21), the determination unit 114 determines in step S22 whether the transmission target is an ROI region. If the determination unit 114 determines that the acquisition unit 113 has not acquired point cloud data environmental information regarding the ROI region (No in step S21), the determination unit 114 proceeds to step S24.

If the determination unit 114 determines that the transmission target is an ROI region (Yes in step S22), it determines in step S23 that the transmission target is an encoded RAW file. If the determination unit 114 determines that the transmission target is not an ROI region (No in step S22), it proceeds to step S24.

In step S24, the determination unit 114 determines whether the acquisition unit 113 has acquired only point cloud data environment information related to the size of the point cloud data.

If the determination unit 114 determines that only point cloud data environmental information related to the size of the point cloud data has been acquired by the acquisition unit 113 (Yes in step S24), then in step S25, the determination unit 114 determines whether the size indicated by the point cloud data environmental information is an encoded RAW file < an encoded GPCC file. If the determination unit 114 determines that only point cloud data environmental information related to the size of the point cloud data has not been acquired by the acquisition unit 113 (No in step S24), then the determination unit 114 proceeds to step S27.

If the determination unit 114 determines that the size indicated by the point cloud data environmental information is the encoded RAW file < the encoded GPCC file (Yes in step S25), it determines in step S23 that the file to be sent is the encoded RAW file. If the determination unit 114 determines that the size indicated by the point cloud data environmental information is the RAW file >= the encoded GPCC file (No in step S25), it determines in step S26 that the file to be sent is the encoded GPCC file.

In step S27, the determination unit 114 determines whether the acquisition unit 113 has acquired only client environment information related to the load of the client 20.

If the determination unit 114 determines that only the client environment information related to the load on the client 20 has been acquired (Yes in step S27), it determines in step S28 whether the load on the client 20 is below a predetermined level. For example, the determination unit 114 determines whether the load on the client 20 is below a predetermined level by, for example, calculating the load related to decoding and rendering on the client 20. Note that since the load on the client 20 is also related to the number of point clouds, the determination unit 114 does not always determine the encoded RAW file as the file to be sent, regardless of the load on the client 20. If the determination unit 114 determines that only the client environment information related to the load on the client 20 has not been acquired (No in step S27), it proceeds to step S29.

If the determination unit 114 determines that the load on the client 20 is equal to or lower than a predetermined level (Yes in step S28), it determines in step S26 that the file to be sent is the encoded GPCC file. If the determination unit 114 determines that the load on the client 20 exceeds a predetermined level (No in step S28), it determines in step S23 that the file to be sent is the encoded RAW file.

In step S29, the determination unit 114 determines whether the transmission time plus the decoding time is within a predetermined time. In this case, for example, the determination unit 114 calculates the transmission time based on the point cloud data environment information related to the size of the point cloud data acquired by the acquisition unit 113 and the communication environment information related to the transmission rate of the communication path. The determination unit 114 also estimates the decoding time based on the point cloud data environment information related to the attributes of the point cloud data, such as the size of the point cloud data and the type of transmission stream, acquired by the acquisition unit 113. The determination unit 114 may correct the calculated value or estimated value based on the most recent transmission time or decoding time. Furthermore, when the server environment information related to the transmission time and the client environment information related to the decoding time are directly acquired by the acquisition unit 113, the determination unit 114 may make the above-mentioned determination based on the transmission time and the decoding time indicated by this environmental information.

If the determination unit 114 determines that the transmission time plus the decoding time is within the predetermined time (Yes in step S29), it determines in step S26 that the file to be sent is the encoded GPCC file. If the determination unit 114 determines that the transmission time plus the decoding time exceeds the predetermined time (No in step S29), it determines in step S23 that the file to be sent is the encoded RAW file.

2. Modification In the example shown in FIG. 4, the acquisition unit 113 of the server 10 of the transmission system 1 acquires point cloud data in which the compression parameters of the RAW file and the GPCC file are the same as the acquisition target. In addition, the determination unit 114 determines a transmission target in which the encoded RAW file and the encoded GPCC file are mixed from the point cloud data in which the compression parameters of the RAW file and the GPCC file are the same as the transmission target. However, the configuration of the transmission system 1 is not limited to the configuration shown in FIG. 4, and may be, for example, the configuration shown in FIG. 9. Hereinafter, an example of the configuration of the transmission system 1 and point cloud data according to the modification will be described with reference to FIG. 9. FIG. 9 is a diagram for explaining the transmission system according to the modification.

In the example shown in FIG. 9, point cloud data is generated (prepared) that is a mixture of GPCC files with different compression parameters (GPCC files High Reso in which point cloud data with a higher resolution than normal GPCC files is compressed, GPCC files Lossy in which point cloud data is irreversibly compressed, and GPCC files Low Reso Lossy in which point cloud data with a lower resolution than normal GPCC files is irreversibly compressed) and RAW files with a single compression parameter. The acquisition unit 113 acquires these point cloud data as acquisition targets based on the environmental information. Furthermore, the determination unit 114 determines the transmission target from these point cloud data based on the environmental information. In other words, the determination unit 114 selectively switches between the encoded uncompressed point cloud data and the encoded compressed point cloud data selected as the transmission target from these point cloud data based on the environmental information. In the example shown in FIG. 9, in addition to or instead of the GPCC files listed as GPCC files with multiple different compression parameters, GPCC files Loses less may be generated that have less reduction in the number of point clouds and less degradation of color information compared to normal GPCC files.

In addition, in the above example, the first point cloud data is encoded RAW data, and the second point cloud data is encoded GPCC data, but the first point cloud data and the second point cloud data are not limited to such a configuration. Each of the first point cloud data and the second point cloud data may be uncompressed point cloud data or compressed point cloud data. In other words, the combination of the first point cloud data and the second point cloud data is not limited to uncompressed/compressed combinations, and may be compressed/uncompressed, compressed/compressed, and uncompressed/uncompressed combinations. For example, if the first point cloud data is compressed encoded RAW data, the communication unit 12 of the server 10 may transmit compressed encoded RAW data, in which the encoded EAW data is compressed in gzip format or the like, to the client 20 by an on-the-fly method or the like.

In the above example, the compressed point cloud data is GPCC data, but the compressed point cloud data is not limited to GPCC data and may be ZIP data, laszip data, draco data, etc. In the above example, the compressed point cloud data is uncompressed divided point cloud data including coordinate information such as xyz after spatial division and resolution division and color information such as RGB, which is compressed as is, but may be compressed by changing the decomposition unit (division). In this case, for example, the compressed point cloud data is compressed by changing the decomposition unit by the division unit 111 and the encoding unit 112 functioning as part of the point cloud space division resolution division encoding processing unit (Point-cloud-converter) in FIG. 4. In the above example, the first point cloud data and the second point cloud data are encoded point cloud data, but may be unencoded point cloud data. In the above example, the client 20 decodes the transmission target by the decoding unit 221, but it is not necessary to decode it, and the transmission target may be stored (stored) as encoded compressed point cloud data, etc.

3. Second embodiment In a transmission system according to one aspect of the present disclosure, a decoding device may acquire environmental information instead of a transmission device and determine a transmission target. Hereinafter, a transmission system 1X according to a second embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of a configuration of a transmission system according to the second embodiment.

In the example shown in FIG. 10, the transmission system 1X has a server 10X and a client 20X instead of the server 10 and the client 20 according to the first embodiment. Other than this, the transmission system 1X is similar to the transmission system 1 according to the first embodiment. The server 10X has a control unit 11X instead of the control unit 11 in the first embodiment. Other than this, the server 10X is similar to the server 10 according to the first embodiment. The control unit 11X does not have the acquisition unit 113 and the determination unit 114 in the first embodiment. Other than this, the control unit 11X is similar to the control unit 11 in the first embodiment. The client 20X has a control unit 22X and a communication unit (receiving unit) 21X instead of the control unit 22 and the communication unit 21 in the first embodiment. Other than this, the client 20X is similar to the client 20 according to the first embodiment. The control unit 22X has an acquisition unit 223 and a determination unit 224. Other than this, the control unit 22X is similar to the control unit 22 in the first embodiment.

The acquisition unit 223 acquires environmental information related to the environment surrounding the client 20X. For example, similar to the acquisition unit 113 in the first embodiment, the acquisition unit 223 acquires environmental information including at least one of communication environment information related to communication between the server 10 and the client 20X, server environment information related to the server 10, client environment information related to the client 20X, and point cloud data environment information related to the point cloud data. For example, the acquisition unit 223 acquires communication environment information related to the transmission rate (communication path) and the network bandwidth (communication bandwidth) by monitoring the network (communication path).

The determination unit 224 determines a transmission target from point cloud data including at least one of the first point cloud data and the second point cloud data, based on the environmental information acquired by the acquisition unit 223. An example of a determination by the determination unit 224 will be described below with reference to Figs. 11 and 12. Figs. 11 and 12 are diagrams for explaining a transmission system according to the second embodiment.

In the example shown in FIG. 11, the determination unit 224 appropriately determines the transmission target when displaying point cloud data that includes both first point cloud data, such as encoded uncompressed point cloud data, and second point cloud data, such as encoded compressed point cloud data, which have been generated in advance by spatially dividing and resolution dividing a point cloud file. Specifically, the determination unit 224 appropriately selects a RAW file and a GPCC file from a mixed file that contains a mixture of RAW files and GPCC files, and determines the transmission target when displaying.

As an example, the determination unit 224 determines the transmission target based on communication environment information regarding the transmission rate included in the environment information acquired by the acquisition unit 223. When the transmission rate most recently acquired by the acquisition unit 223 is low, such as when the transmission rate is lower than a threshold, the determination unit 224 determines a small-sized file, such as the smallest-sized encoded GPCC file, from the file group to be the transmission target. When the transmission rate most recently acquired by the acquisition unit 223 is high, such as when the transmission rate is equal to or higher than a threshold, the determination unit 224 determines an encoded RAW file from the file group to be the transmission target. In this case, in order to refer to the size list of the file group, the determination unit 224 may cause the server 10X to transmit the size list in advance via the communication unit 12 and the communication unit 21X and cause the acquisition unit 223 to acquire it. The determination unit 224 may determine the size list to be requested from the server 10X depending on whether image quality or size is to be prioritized, and then cause this size list to be transmitted from the server 10X via the communication unit 12 and the communication unit 21X, and to be acquired by the acquisition unit 223.

As another example, the determination unit 224 determines a transmission target based on client environment information related to the load of the client 20X included in the environment information acquired by the acquisition unit 223 and other environment information. In this case, the determination unit 224 calculates a value related to the CPU margin, such as the CPU usage rate, from the current load average of the client 20X, and estimates the load of decoding, rendering, etc. on the client 20X when the transmission target is transmitted, from the CPU margin and the compression format, etc. The determination unit 224 determines candidates for the transmission target from a file group based on other environment information, and determines the transmission target with the smallest estimated load on the client 20X from among the candidates for the transmission target.

As another example, the determination unit 224 determines the transmission target based on client environment information related to the performance of the client 20X included in the environment information acquired by the acquisition unit 223. In this case, when the performance of the client 20X, such as the decoding ability, most recently acquired by the acquisition unit 223 is lower than a threshold, the determination unit 224 determines the encoded RAW file from among the file group to be the transmission target when the performance is low. When the performance of the client 20X, such as the decoding ability, most recently acquired by the acquisition unit 223 is equal to or higher than a threshold, the determination unit 224 determines the encoded GPCC file from among the file group to be the transmission target. When the performance of the client 20X, such as the decoding ability, most recently acquired by the acquisition unit 223 is equal to or higher than a threshold, the determination unit 224 determines the encoded RAW file from among the file group to be the transmission target when the performance is high, such as the decoding ability, most recently acquired by the acquisition unit 223 is equal to or higher than a threshold, and when the transmission rate most recently acquired by the acquisition unit 223 is equal to or higher than a threshold, the determination unit 224 determines the encoded RAW file from among the file group to be the transmission target. The decision unit 224 may change a threshold value for switching the ratio of encoded uncompressed point cloud data, such as encoded RAW files, to encoded compressed point cloud data, such as encoded GPCC files, in addition to the data to be transmitted, based on the performance of the client 20X, such as the decoding capability.

As another example, the determination unit 224 may determine the transmission target based on point cloud data environment information related to the ROI region included in the environment information. When the point cloud data is an ROI region, the determination unit 224 determines the transmission target to be an encoded RAW file in order to maintain the accuracy of the original point cloud data. When the point cloud data is a non-ROI region, the determination unit 224 determines the transmission target to be an encoded GPCC file in order to shorten the transmission time and response.

As another example, the determination unit 224 determines the transmission target based on multiple pieces of information included in the above-mentioned environmental information. For example, the determination unit 114 determines the transmission target each time by taking into consideration a combination of point cloud data environmental information related to the size of the point cloud data included in the environmental information, and client environmental information related to the load on the client 20X, etc.

As another example, the determination unit 224 determines the transmission target in parallel with the display of the transmission target by the client 20X. In the example shown in FIG. 12, the determination unit 224 (File Selector) determines the transmission target in parallel with the display of the transmission target on the display unit 23 by the display control unit 222 of the client 20X. For example, at the start of rough drawing on the display unit 23 by the display control unit 222, the determination unit 224 determines the files included in the transmission target based on the size of each file such as a RAW file and a GPCC file at the same spatial position, issues a transmission request for the transmission target to the server 10X via the communication unit 21X and the communication unit 12, and causes the display control unit 222 to proceed with drawing the file transmitted from the server 10X. As the display control unit 222 reads the Level value of the resolution level and the display of the transmission target on the display unit 23 progresses to a detailed level, the determination unit 224 calculates the transmission time from the file size based on the most recent transmission rate of the server 10. Furthermore, the determination unit 224 estimates the decoding time required by the decoding unit 221 of the client 20X when the encoded GPCC file is further selected as the file to be sent, based on the stream size and the most recent decoding time. The determination unit 224 selects the type of file to be sent and determines the file to be sent so that the total value of the transmission time and the decoding time is the smallest.

The communication unit 21X receives the transmission target determined by the determination unit 224 from the server 10X to which the transmission target is to be transmitted.

Next, an example of the decision processing by the decision unit 224 of the client 20X will be described with reference to FIG. 13. FIG. 13 is a flow diagram showing an example of the processing by the client according to the second embodiment. Steps S21 to S28 in FIG. 13 are similar to steps S21 to S28 in FIG. 8. That is, each process by the decision unit 224 other than step S29X in FIG. 13 is similar to each process by the decision unit 114 in the first embodiment.

In step S29X, the determination unit 224 determines whether the transmission time plus the decoding time is within a predetermined time. In the client 20X, the determination unit 224 may determine the transmission target in parallel with the display of the transmission target by the client 20X. For example, the determination unit 224 determines the transmission target based on the environmental information acquired by the acquisition unit 223 while the display control unit 222 of the client 20X is drawing the transmission target on the display unit 23. In this case, the determination unit 224 determines the type of data included in the transmission target based on the response indicated by the environmental information (such as the amount of communication bandwidth and the margin of the drawing process by the CPU of the control unit 22X, etc., which includes the display control unit 222). The determination unit 224 adaptively changes the type of data included in the transmission target acquired by the acquisition unit 223 and requested to be transmitted according to the response, thereby determining an appropriate transmission target and achieving a balance between an appropriate response and drawing image quality.

In the above example, the determination unit 224 of the client 20X determines the transmission target in parallel with the display of the transmission target by the client 20X. However, if the server 10X has the determination unit 224, the determination unit 224 of the server 10X may determine the transmission target in parallel with the display of the transmission target by the client 20X.

4. An Example of Hardware Configuration The various devices, such as the server 10, the server 10X, the client 20, and the client 20X, described above, may be configured to include a computer. An example will be described with reference to FIG.

FIG. 14 is a diagram showing an example of the hardware configuration of the device. The illustrated computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD 1400, a communication interface 1500, and an input/output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

The CPU 1100 operates based on the programs stored in the ROM 1300 or the HDD 1400, and controls each component. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200, and executes processes corresponding to the various programs.

The ROM 1300 stores boot programs such as the Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records generation programs for executing the operations related to the present disclosure, which are an example of program data 1450.

The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input/output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker or a printer via the input/output interface 1600. The input/output interface 1600 may also function as a media interface for reading programs and the like recorded on a specific recording medium. The media may be optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase change rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memories.

At least some of the functions of the server 10, server 10X, client 20, client 20X, etc. described above may be realized, for example, by the CPU 1100 of the computer 1000 executing a program loaded onto the RAM 1200. In addition, the HDD 1400 stores the programs and the like related to the present disclosure. Note that the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be obtained from other devices via the external network 1550.

5. Example of Effects As described with reference to Fig. 1 to Fig. 13 etc., one of the disclosed technologies is a server 10 etc. The server 10 includes an acquisition unit 113 that acquires environmental information related to the environment around the server 10, a determination unit 114 that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit 113, and a communication unit 12 that transmits the transmission target determined by the determination unit 114 to a decoding device.

The server 10 selects, according to the environmental information, either or both of the first point cloud data and the second point cloud data included in the point cloud data as the transmission target, and selectively switches between the first point cloud data and the second point cloud data selected as the transmission target. This allows the server 10 to determine an appropriate transmission target according to the environmental information. As a result, the server 10 can reduce the reception period and cost of the point cloud data.

As described with reference to Figures 1 to 13, the acquisition unit 113 acquires environmental information including at least one of communication environment information regarding communication between the server 10 and the client 20, server environment information regarding the server 10, client environment information regarding the client 20, and point cloud data environment information regarding the point cloud data, and the determination unit 114 may determine the transmission target based on the information included in the environmental information acquired by the acquisition unit 113.

Therefore, the server 10 can determine an appropriate transmission target according to the communication environment information, the server environment information, the client environment information, and the point cloud data environment information. As an example, when the communication bandwidth indicated by the communication environment information is narrow and the performance of the client 20 indicated by the client environment information is high, the server 10 determines point cloud data including second point cloud data such as a highly compressed transmission stream as the transmission target. As a result, the server 10 can reduce the transmission period and the subsequent display response. As another example, when the communication speed indicated by the communication environment information is fast but the performance of the client 20 indicated by the client environment information is poor, the server 10 selects point cloud data including first point cloud data such as encoded non-compressed point cloud data as the transmission target. As a result, the server 10 can reduce the cost of decoding by the client 20. As another example, the server 10 determines the transmission target with the smallest response, which is the sum of the transmission period indicated by the server environment information and the decoding period indicated by the client environment information. As a result, the server 10 can shorten the display response until the transmission target is displayed by the client 20.

As described with reference to Figures 1 to 13, the acquisition unit 113 may acquire environmental information including communication environment information related to the performance of the communication path between the server 10 and the client 20, and the determination unit 114 may determine the transmission target based on the communication environment information included in the environmental information acquired by the acquisition unit 113. This allows the server 10 to determine an appropriate transmission target according to the communication environment information related to the performance of the communication path between the server 10 and the client 20. For example, the server 10 can estimate the communication period based on this communication environment information, and determine an appropriate transmission target that will shorten the communication period.

As described with reference to Figures 1 to 13, the acquisition unit 113 may acquire environmental information that includes communication environment information related to at least one of the communication speed and communication bandwidth of the communication path. This allows the server 10 to determine an appropriate transmission target according to the communication environment information related to the communication speed and communication bandwidth of the communication path. For example, the server 10 can estimate the communication period based on the communication speed, and determine an appropriate transmission target that will shorten the communication period.

As described with reference to Figures 1 to 13, the acquisition unit 113 acquires environmental information including client environmental information related to at least one of the performance, load, decoding period, reception status, and display status of the client 20, and the determination unit 114 may determine the transmission target based on the client environmental information included in the environmental information acquired by the acquisition unit 113. This allows the server 10 to determine an appropriate transmission target according to the client environmental information related to at least one of the performance, load, decoding period, reception status, and display status of the client 20. For example, the server 10 can estimate the decoding period based on the performance of the client 20 and determine an appropriate transmission target that will shorten the decoding period.

As described with reference to Figures 1 to 13, the acquisition unit 113 may acquire environmental information including client environmental information related to the decoding speed of the client 20. This allows the server 10 to determine an appropriate transmission target according to the client environmental information related to the decoding speed of the client 20. For example, the server 10 can estimate the decoding period based on the decoding speed of the client 20 and determine an appropriate transmission target that will shorten the decoding period.

As described with reference to Figures 1 to 13, the acquisition unit 113 may acquire environmental information that includes client environmental information related to the reception status of the point cloud data by the client 20. This allows the server 10 to determine an appropriate transmission target according to the reception status of the client 20.

As described with reference to Figures 1 to 13, the acquisition unit 113 may acquire environmental information including server environmental information related to at least one of the performance, load, and transmission period of the server 10, and the determination unit 114 may determine the transmission target based on the server environmental information included in the environmental information acquired by the acquisition unit 113. This allows the server 10 to determine an appropriate transmission target according to the server environmental information related to at least one of the performance, load, and transmission period of the server 10. For example, the server 10 can determine a transmission target that reduces (alleviates) the load related to the transmission of the transmission target of the server 10 and the storage costs of the server 10, based on the server environmental information related to the load of the server 10.

As described with reference to Figures 1 to 13, the acquisition unit 113 may acquire environmental information that includes server environment information related to the service capabilities of the server 10. This allows the server 10 to determine an appropriate transmission target according to the server environment information related to the service capabilities of the server 10.

As described with reference to Figures 1 to 13, the acquisition unit 113 acquires environmental information including point cloud data environmental information related to attributes of point cloud data requested by the client 20, and the determination unit 114 may determine the transmission target based on the point cloud data environmental information included in the environmental information acquired by the acquisition unit 113. This allows the server 10 to determine an appropriate transmission target according to the point cloud data environmental information.

As described with reference to Figures 1 to 13, the acquisition unit 113 may acquire point cloud data environment information related to at least one of the accuracy, size, resolution, number, space, and ROI area of the point cloud data.

As a result, the server 10 can determine an appropriate transmission target according to the point cloud data environmental information related to at least one of the accuracy, size, resolution, number, space, and ROI area of the point cloud data. As an example, the server 10 determines high-precision point cloud data as the transmission target based on the point cloud data environmental information related to the accuracy of the point cloud data. As a result, the server 10 can determine an appropriate transmission target according to the point cloud data environmental information related to the accuracy of the point cloud data. As another example, the server 10 determines the transmission target depending on whether the transmission target is an ROI area based on the point cloud data environmental information related to the ROI area of the point cloud data. In this case, the server 10 determines the first point cloud data such as encoded uncompressed point cloud data as the transmission target when the transmission target is an ROI area, and determines the second point cloud data such as encoded compressed point cloud data as the transmission target when the transmission target is a non-ROI area. As a result, it is possible to determine an appropriate transmission target according to the point cloud data environmental information related to the ROI area of the point cloud data.

As described with reference to Figures 1 to 13, the acquisition unit 113 acquires environmental information including point cloud data environmental information related to the point cloud data requested by the client 20 as a transmission target, and the determination unit 114 may determine the transmission target based on the point cloud data environmental information included in the environmental information acquired by the acquisition unit 113.

As a result, the server 10 can determine an appropriate transmission target according to the point cloud data environment information related to the point cloud data requested by the client 20 as a transmission target. As an example, when the client 20 requests data almost as is as a transmission target based on the point cloud data environment information, the server 10 determines the transmission target to be first point cloud data such as encoded uncompressed point cloud data, or mixed data in which the proportion of the second point cloud data such as encoded compressed point cloud data is smaller than the proportion of the first point cloud data such as encoded uncompressed point cloud data. As another example, when the client 20 requests high-precision point cloud data or point cloud data that provides a faster display response based on the point cloud data environment information, the server 10 adjusts (selects) the compression degree of the compressed point cloud data. In this case, the server 10 adjusts the display response by, for example, adjusting the proportion of the first point cloud data such as encoded uncompressed point cloud data and the second point cloud data such as encoded compressed point cloud data. The server 10 also adjusts the compression parameters of the encoded uncompressed point cloud data. This allows the server 10 to determine an appropriate transmission target in response to a request from the client 20.

As described with reference to Figs. 1 to 13, the determination unit 114 determines the acquisition target to be acquired by the acquisition unit 113 based on the environmental information acquired by the acquisition unit 113, and the acquisition unit 113 may further acquire the acquisition target determined by the determination unit 114 as point cloud data. This allows the server 10 to reduce costs since it is necessary to prepare only the point cloud data determined as the transmission target as the acquisition target.

As described with reference to Figures 1 to 13, each of the first point cloud data and the second point cloud data may be uncompressed point cloud data or compressed point cloud data. The server 10 can advantageously reduce the reception period and cost of point cloud data with such a data structure.

As described with reference to Figures 1 to 13, at least one of the first point cloud data and the second point cloud data may include at least one of 3D point cloud data and a plurality of divided point cloud data divided based on at least one of space and resolution. The server 10 can advantageously reduce the reception period and cost of point cloud data with such a data structure.

The client 20X described with reference to Figures 1 to 13 etc. is also one of the disclosed technologies. The client 20X has an acquisition unit 223 that acquires environmental information related to the environment surrounding the client 20X, a determination unit 224 that determines a transmission target from point cloud data including first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit 223, and a communication unit 21X that receives the transmission target determined by the determination unit 224 from a transmitting device. As described above, such a client 20X can also reduce the reception period and cost of point cloud data.

The transmission system 1 described with reference to Figures 1 to 13 is also one of the disclosed technologies. The transmission system 1 has a server 10 and a client 20. The server 10 has an acquisition unit 113 that acquires environmental information related to the environment around the server 10, a determination unit 114 that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit 113, and a communication unit 12 that transmits the transmission target determined by the determination unit 114 to the client 20. The client 20 has a communication unit 21 that receives the transmission target determined by the determination unit 114 from the server 10. With such a transmission system 1, the reception period and cost of point cloud data can be reduced, as described above.

The transmission method described with reference to Figures 1 to 13 etc. is also one of the disclosed technologies. The transmission method is executed by the server 10 and includes an acquisition step of acquiring environmental information related to the environment around the server 10, a determination step of determining a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition step, and a transmission step of transmitting the transmission target determined by the determination step to a decoding device. As explained above, such a communication method can also reduce the reception period and cost of point cloud data.

The transmission program described with reference to Figures 1 to 13 is also one of the disclosed technologies. The transmission program causes a computer mounted on the server 10 to execute an acquisition process for acquiring environmental information related to the environment around the server 10, a determination process for determining a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition process, and a transmission process for transmitting the transmission target determined by the determination process to a decoding device. As described above, such a communication program can also reduce the reception period and cost of point cloud data.

The effects described in this disclosure are merely examples and are not limited to the disclosed contents. Other effects may also exist.

The above describes the embodiments of the present disclosure, but the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present disclosure, and different components in the modified examples may be appropriately combined.

This technology may be related to Goal 9 "Industry, innovation, infrastructure" of the Sustainable Development Goals (SDGs) adopted at the United Nations Summit in 2015.

The present technology can also be configured as follows.
(1)
A transmitting device,
an acquisition unit that acquires environmental information relating to a surrounding environment of the transmitting device;
a determination unit that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit;
a transmission unit that transmits the transmission target determined by the determination unit to a decoding device;
A transmitting device comprising:
(2)
the acquisition unit acquires the environment information including at least one of communication environment information related to communication between the transmission device and the decoding device, transmission device environment information related to the transmission device, decoding device environment information related to the decoding device, and point cloud data environment information related to the point cloud data;
The determination unit determines the transmission target based on information included in the environmental information acquired by the acquisition unit.
A transmitting device as described in (1).
(3)
the acquisition unit acquires the environment information including the communication environment information related to performance of a communication path between the transmission device and the decoding device;
The determination unit determines the transmission target based on the communication environment information included in the environment information acquired by the acquisition unit.
A transmitting device as described in (2).
(4)
the acquiring unit acquires the environment information including the communication environment information related to at least one of a communication speed and a communication band of the communication path.
A transmitting device as described in (3).
(5)
the acquiring unit acquires the environmental information including the decoding device environmental information relating to at least one of a performance, a load, a decoding period, a reception status, and a display status of the decoding device;
The determination unit determines the transmission target based on the decoding device environment information included in the environment information acquired by the acquisition unit.
A transmitting device as described in (2).
(6)
the acquiring unit acquires the environmental information including the decoding device environmental information related to a decoding speed of the decoding device.
A transmitting device as described in (5).
(7)
the acquiring unit acquires the environmental information including the decoding device environmental information relating to a receiving state of the point cloud data by the decoding device.
A transmitting device as described in (5).
(8)
the acquiring unit acquires the environmental information including the transmitting device environmental information related to at least one of a performance, a load, and a transmission period of the transmitting device;
The determination unit determines the transmission target based on the transmission device environment information included in the environment information acquired by the acquisition unit.
A transmitting device as described in (2).
(9)
The acquisition unit acquires the environmental information including the transmission device environmental information related to the service capability of the transmission device.
A transmitting device as described in (8).
(10)
the acquisition unit acquires the environmental information including point cloud data environmental information related to an attribute of the point cloud data requested by the decoding device;
The determination unit determines the transmission target based on the point cloud data environment information included in the environment information acquired by the acquisition unit.
A transmitting device as described in (2).
(11)
The acquisition unit acquires the point cloud data environment information related to at least one of accuracy, size, resolution, number, space, and ROI (Region of Interest) of the point cloud data.
A transmitting device as described in (10).
(12)
the acquisition unit acquires the environmental information including the point cloud data environmental information related to the point cloud data requested as the transmission target from the decoding device;
The determination unit determines the transmission target based on the point cloud data environment information included in the environment information acquired by the acquisition unit.
A transmitting device as described in (2).
(13)
The determination unit determines an acquisition target to be acquired by the acquisition unit based on the environmental information acquired by the acquisition unit;
The acquisition unit further acquires the acquisition target determined by the determination unit as the point cloud data.
A transmitting device according to any one of (1) to (12).
(14)
Each of the first point cloud data and the second point cloud data is uncompressed point cloud data or compressed point cloud data.
A transmitting device according to any one of (1) to (13).
(15)
At least one of the first point cloud data and the second point cloud data includes at least one of 3D (Dimensions) point cloud data and a plurality of divided point cloud data divided based on at least one of space and resolution.
A transmitting device according to any one of (1) to (14).
(16)
A decoding device, comprising:
an acquisition unit that acquires environmental information relating to a surrounding environment of the decoding device;
a determination unit that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit;
a receiving unit that receives the transmission target determined by the determining unit from a transmitting device;
A decoding device comprising:
(17)
The determination unit determines the transmission target in parallel with the display of the transmission target by the decoding device.
A decoding device according to (16).
(18)
A transmitting device and a decoding device are included,
The transmitting device
an acquisition unit that acquires environmental information relating to a surrounding environment of the transmitting device;
a determination unit that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit;
a transmission unit that transmits the transmission target determined by the determination unit to a decoding device;
having
The decoding device comprises:
a receiving unit that receives the transmission target determined by the determining unit from the transmitting device;
having
Transmission system.
(19)
A transmission method performed by a transmission device, comprising:
An acquisition step of acquiring environmental information relating to a surrounding environment of the transmitting device;
a determination step of determining a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition step;
a transmission step of transmitting the transmission target determined by the determination step to a decoding device;
A transmission method including:
(20)
A computer installed in the transmitting device
An acquisition process of acquiring environmental information related to a surrounding environment of the transmitting device;
a determination process of determining a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition process;
a transmission process of transmitting the transmission target determined by the determination process to a decoding device;
Execute the sending program.

1 Transmission system 10 Server (transmission device)
11, 22 Control unit 12, 21 Communication unit 13, 24 Storage unit 20 Client (decryption device)
23 Display unit 111 Division unit 112 Encoding unit 113 Acquisition unit 114 Determination unit 221 Decoding unit 222 Display control unit

Claims (20)

A transmitting device,
an acquisition unit that acquires environmental information relating to a surrounding environment of the transmitting device;
a determination unit that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit;
a transmission unit that transmits the transmission target determined by the determination unit to a decoding device;
A transmitting device comprising: the acquisition unit acquires the environment information including at least one of communication environment information related to communication between the transmission device and the decoding device, transmission device environment information related to the transmission device, decoding device environment information related to the decoding device, and point cloud data environment information related to the point cloud data;
The determination unit determines the transmission target based on information included in the environmental information acquired by the acquisition unit.
The transmitting device according to claim 1 . the acquisition unit acquires the environment information including the communication environment information related to performance of a communication path between the transmission device and the decoding device;
The determination unit determines the transmission target based on the communication environment information included in the environment information acquired by the acquisition unit.
The transmitting device according to claim 2. the acquiring unit acquires the environment information including the communication environment information related to at least one of a communication speed and a communication band of the communication path.
The transmitting device according to claim 3. the acquiring unit acquires the environmental information including the decoding device environmental information relating to at least one of a performance, a load, a decoding period, a reception status, and a display status of the decoding device;
The determination unit determines the transmission target based on the decoding device environment information included in the environment information acquired by the acquisition unit.
The transmitting device according to claim 2. the acquiring unit acquires the environmental information including the decoding device environmental information related to a decoding speed of the decoding device.
The transmitting device according to claim 5. the acquiring unit acquires the environmental information including the decoding device environmental information relating to a receiving state of the point cloud data by the decoding device.
The transmitting device according to claim 5. the acquiring unit acquires the environmental information including the transmitting device environmental information related to at least one of a performance, a load, and a transmission period of the transmitting device;
The determination unit determines the transmission target based on the transmission device environment information included in the environment information acquired by the acquisition unit.
The transmitting device according to claim 2. The acquisition unit acquires the environmental information including the transmission device environmental information related to the service capability of the transmission device.
The transmitting device according to claim 8. the acquisition unit acquires the environmental information including point cloud data environmental information related to an attribute of the point cloud data requested by the decoding device;
The determination unit determines the transmission target based on the point cloud data environment information included in the environment information acquired by the acquisition unit.
The transmitting device according to claim 2. The acquisition unit acquires the point cloud data environment information related to at least one of accuracy, size, resolution, number, space, and ROI (Region of Interest) of the point cloud data.
The transmitting device according to claim 10. the acquisition unit acquires the environmental information including the point cloud data environmental information related to the point cloud data requested as the transmission target from the decoding device;
The determination unit determines the transmission target based on the point cloud data environment information included in the environment information acquired by the acquisition unit.
The transmitting device according to claim 2. The determination unit determines an acquisition target to be acquired by the acquisition unit based on the environmental information acquired by the acquisition unit;
The acquisition unit further acquires the acquisition target determined by the determination unit as the point cloud data.
The transmitting device according to claim 1 . Each of the first point cloud data and the second point cloud data is uncompressed point cloud data or compressed point cloud data.
The transmitting device according to claim 1 . At least one of the first point cloud data and the second point cloud data includes at least one of 3D (Dimensions) point cloud data and a plurality of divided point cloud data divided based on at least one of space and resolution.
The transmitting device according to claim 1 . A decoding device, comprising:
an acquisition unit that acquires environmental information relating to a surrounding environment of the decoding device;
a determination unit that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit;
a receiving unit that receives the transmission target determined by the determining unit from a transmitting device;
A decoding device comprising: The determination unit determines the transmission target in parallel with the display of the transmission target by the decoding device.
17. A decoding device according to claim 16. A transmitting device and a decoding device are included,
The transmitting device
an acquisition unit that acquires environmental information relating to a surrounding environment of the transmitting device;
a determination unit that determines a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition unit;
a transmission unit that transmits the transmission target determined by the determination unit to a decoding device;
having
The decoding device comprises:
a receiving unit that receives the transmission target determined by the determining unit from the transmitting device;
having
Transmission system. A transmission method performed by a transmission device, comprising:
An acquisition step of acquiring environmental information relating to a surrounding environment of the transmitting device;
a determination step of determining a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition step;
a transmission step of transmitting the transmission target determined by the determination step to a decoding device;
A transmission method including: A computer installed in the transmitting device
An acquisition process of acquiring environmental information related to a surrounding environment of the transmitting device;
a determination process of determining a transmission target from point cloud data including at least one of first point cloud data and second point cloud data based on the environmental information acquired by the acquisition process;
a transmission process of transmitting the transmission target determined by the determination process to a decoding device;
Execute the sending program.

Images