WO2025205058A1 - Information processing device, information processing method, and program - Google Patents

Abstract

This information processing device has a RAW processing unit and a ranging calculation unit. The RAW processing unit acquires a normalized luminance image in which the luminance of respective colors is normalized from the RAW signal. The ranging calculation unit acquires depth information of a subject from the normalized luminance image by using a monocular depth estimation model trained with regard to correlations between depth and luminance.

Description

Information processing device, information processing method, and program

The present invention relates to an information processing device, an information processing method, and a program.

Endoscopic images are often taken with a monocular camera attached to the tip of the endoscope. However, it is difficult to obtain depth information from monocular camera images. It is difficult to measure size, which is calculated from depth information and two-dimensional information (such as RGB images), and this hinders diagnosis and surgery.

Japanese Patent Application Laid-Open No. 2022-172654

Currently, methods include inserting a tape measure through the forceps opening to directly measure the size of the object, or projecting images using lasers or mesh structures, but these tasks require a high level of skill. While artificial intelligence (AI) is also used to estimate depth information from endoscopic images, subtle changes in color can easily lead to errors, making it difficult to achieve high inference accuracy.

This disclosure therefore proposes an information processing device, information processing method, and program that are capable of acquiring depth information with high accuracy.

The present disclosure provides an information processing device having a RAW processing unit that acquires a normalized luminance image in which the luminance of each color is normalized from a RAW signal, and a ranging calculation unit that acquires depth information of a subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance. The present disclosure also provides an information processing method in which the information processing of the information processing device is executed by a computer, and a program that causes a computer to realize the information processing of the information processing device.

FIG. 1 is a diagram illustrating an overview of a distance measurement method according to the present disclosure. FIG. 1 is a diagram illustrating an overview of a distance measurement method according to the present disclosure. FIG. 1 is a diagram illustrating an overview of a distance measurement method according to the present disclosure. 10A and 10B are diagrams illustrating inference results of depth information obtained by applying the depth estimation method of the present disclosure to a monocular image. 10A and 10B are diagrams illustrating inference results of depth information obtained by applying the depth estimation method of the present disclosure to a monocular image. FIG. 10 is a diagram showing an example of capturing an image of a rectangular plate with a monocular camera of an endoscope. FIG. 10 is a diagram showing an example of photographing an incision in implant surgery. FIG. 1 is a diagram illustrating an example of an information processing system that implements a depth estimation technique according to the present disclosure. FIG. 2 illustrates an example of the configuration of an image distance acquisition unit. FIG. 10 is a diagram illustrating switching of a learning model according to lighting conditions. 10A and 10B are diagrams illustrating variations of the information processing system. FIG. 2 illustrates an example of a hardware configuration of an information processing device.

Embodiments of the present disclosure will be described in detail below with reference to the drawings. In each of the following embodiments, identical components will be designated by the same reference numerals, and duplicate descriptions will be omitted.

The explanation will be given in the following order.
[1. Overview of the Distance Measurement Method of the Present Disclosure]
2. Configuration of information processing device
[3. Switching learning models according to lighting conditions]
[4. System Variations]
[5. Hardware Configuration Example]
6. Effects

[1. Overview of the Distance Measurement Method of the Present Disclosure]
1 to 3 are diagrams for explaining an overview of the distance measurement method of the present disclosure.

Demand for depth estimation technology is increasing for devices and applications such as surgical support robots and endoscopy robots. Well-known methods for obtaining depth information include those using structured light and triangulation technology, and those using lidar sensors.

While these methods can obtain highly accurate depth information, they require special optical systems and dedicated sensors separate from the main system, resulting in large-scale equipment. This not only increases costs, but also creates the problem of eliminating system configurations such as existing endoscope-based endoscopy robots. Depth estimation using stereo image matching, which does not require special lighting, can also be used, but this is subject to the significant restriction of being limited to stereo endoscopes.

This disclosure has been made in consideration of the above-mentioned problems. This disclosure proposes a depth estimation method that can acquire highly accurate depth information without using a special device configuration. The example in Figure 1 shows lighting conditions using a point light source LS, which is common in the field of monocular endoscopes. There is a correlation between image brightness and depth. For example, in a lighting environment using a point light source LS, brightness attenuates in proportion to the square of the distance based on the inverse square of the illuminance. The correlation between brightness and depth can change depending on the lighting conditions. In this disclosure, depth information is estimated from a RAW signal based on a correlation between brightness and depth that has been learned in advance.

In the example in Figure 2, an image of object OB is taken in an illumination environment using a point light source LS. Object OB has colors such as red, green, and blue. The luminance of each color differs depending on the wavelength characteristics (light source spectrum, color filter transmission characteristics, spectral reflectance characteristics of object OB, etc.), but if the luminance is corrected (normalized) to offset the wavelength characteristics, depth information can be obtained from the corrected luminance.

Normalization refers to the process of adjusting the scaling coefficient for each color to equalize the luminance range of each color obtained under the same lighting conditions. Normalization is performed on the image signal before development processing (the RAW signal, or a signal that has been processed by applying corrections such as denoising and improving distance detection capabilities to the RAW signal). Normalization is performed on the image signal before development processing to avoid significant distortion of the original color and luminance information during development processing.

In the example of Figure 3, the luminance of each color object OB has been converted to white-equivalent luminance through normalization. White-equivalent luminance refers to the luminance that would be expected to be obtained if the color of the object were white. In this disclosure, the process of converting the luminance of each color to white-equivalent luminance is referred to as white-equivalent luminance conversion. The left side of Figure 3 shows the luminance distribution contained in the RAW signal (luminance image before normalization). The right side of Figure 3 shows the luminance distribution after white-equivalent luminance conversion has been performed on the RAW signal (normalized luminance image).

Figures 4 and 5 show the inferred depth information obtained by applying the depth estimation method of the present disclosure to monocular images. The example in Figure 4 shows an endoscopic image of a lesion. The example in Figure 5 shows monocular images from multiple color panels. In both examples, accurate depth information is obtained that reflects the luminance information contained in the RAW signal.

Figure 6 shows an example of capturing an image of a rectangular plate with an endoscope's monocular camera. The plate is tilted at an angle relative to the optical axis of the monocular camera. The shape of the plate captured by the monocular camera is trapezoidal. The position of the plate in the depth direction is estimated based on the normalized brightness image. The planar shape of the rectangle is obtained by reconstructing the shape taking the depth into account.

Figure 7 shows an example of an image of an incision made during implant surgery. During implant surgery, part of the temporal bone is removed and an electrode is inserted into the cochlea. The incision, which is oblique in the depth direction, is covered with a thin, flat plate (plane plate) made of cartilage or similar material. The endoscope observes the area where the plane plate will be placed. Depth information is obtained using a normalized brightness image to measure the three-dimensional shape of the object being observed and the contour shape (two-dimensional shape) of the area where the plane plate will be placed.

By taking depth into account, the exact shape of the object being observed can be restored. For example, it becomes possible to obtain the outer shape of the object being observed projected onto a two-dimensional plane from a specified direction. Conventional methods cannot obtain accurate depth information from monocular images. Therefore, obtaining an accurate shape that takes depth into account requires the experience of a doctor and the effort of actually creating an object, inserting it into the affected area, and then reshaping it. The method disclosed herein makes it possible to derive an accurate shape without requiring the experience or effort of a doctor.

2. Configuration of information processing device
8 is a diagram illustrating an example of an information processing system that implements the depth estimation method of the present disclosure. This system acquires depth information from, for example, a monocular endoscopic image. The information processing system includes an information processing device 1, an endoscopic camera unit 2, and an external input device 3.

The endoscopic camera unit 2 has a monocular camera and a light source device. Light output from the light source device is projected from the tip of a light-guiding fiber toward the affected area. The tip of the fiber, which serves as the light source, serves as the light source for illumination. The endoscopic camera unit 2 uses the monocular camera to photograph the subject SB illuminated by the illumination light. The light source device can project visible light as illumination light, or special light that can improve the visibility of the affected area. The endoscopic camera unit 2 may be a flexible or rigid endoscope.

The information processing device 1 acquires a RAW signal RD of the subject SB from the endoscopic camera unit 2. The information processing device 1 analyzes the RAW signal RD to acquire information such as the depth, size, shape, and distance between points of the subject SB. For example, the information processing device 1 has an image acquisition unit 11, an image distance acquisition unit 12, an image development unit 13, and a measurement processing unit 14.

The image acquisition unit 11 acquires the RAW signal RD from the endoscopic camera unit 2 and outputs it to the image distance acquisition unit 12 and the image development unit 13. The RAW signal RD is an image signal that records the information received from the monocular camera (image sensor) in almost its original state. The RAW signal RD has not undergone any significant adjustments to color or brightness, as is done in the development process.

The image distance acquisition unit 12 analyzes the RAW signal RD to acquire depth information DI of the subject SB. Figure 9 is a diagram showing an example of the configuration of the image distance acquisition unit 12. For example, the image distance acquisition unit 12 has a RAW processing unit 15 and a distance measurement calculation unit 16.

The RAW processing unit 15 obtains a normalized luminance image in which the luminance of each color has been normalized from the RAW signal RD. Normalization refers to the process of adjusting the scaling coefficient for each color to equalize the luminance range of each color obtained under the same lighting conditions. Normalization does not involve significant adjustments to color or brightness, as is done in the development process. Therefore, the luminance information after normalization includes highly accurate depth information DI based on the correlation between depth and luminance.

The RAW processing unit 15 can perform various correction processes on the RAW signal RD, as long as they do not involve significant changes to color or brightness that would interfere with the calculation of the depth information DI. In the example of Figure 9, the correction processes performed include noise suppression, tone curve correction, and frequency correction.

Noise suppression processing removes noise that causes errors in distance measurement calculations. Tone curve correction matches the image sensor output to the characteristics of distance measurement calculations, obtaining a signal with good distance detection properties. Frequency correction emphasizes the contours of the subject SB by amplifying (boosting, multiplying) attenuated high-frequency signals. By highlighting the contours, it is possible to reduce errors in measuring the size of affected areas, etc.

The distance measurement calculation unit 16 obtains depth information DI of the subject SB from the normalized luminance image using a learning model (monocular depth estimation model 18: see Figure 10) that has learned about the correlation between depth and luminance. As mentioned above, the RAW signal RD does not undergo significant adjustments to color or brightness, as is done in the development process. Therefore, accurate depth information DI can be obtained by normalizing the luminance of each color contained in the RAW signal RD and applying the normalized luminance distribution (normalized luminance image) to the monocular depth estimation model 18.

The monocular depth estimation model 18 is trained using a group of images obtained under lighting conditions consistent with the actual shooting environment as training data. The training may be supervised learning or self-supervised learning. When the input to the monocular depth estimation model 18 is a raw signal RD that has undergone correction processing such as tone curve correction or frequency correction, a group of images that have undergone the same correction processing can be used as training data.

In the example of Figure 9, the depth information DI is obtained through a two-stage process: normalization and depth estimation. However, these processes may also be performed collectively by a single AI. For example, it is difficult to explicitly determine the formula for the white equivalent luminance conversion performed in normalization. Therefore, it is preferable that normalization be performed using a learning model (normalization model). By having the monocular depth estimation model MD take on the role of the normalization model, the depth information DI is obtained directly from the RAW signal RD. In this case, the distance measurement calculation unit 16 also serves as the RAW processing unit 15.

It is preferable that the lighting conditions used in the endoscopic camera unit 2 match the lighting conditions of the training data for the monocular depth estimation model 18. For example, the image acquisition unit 11 acquires, as a RAW signal RD, an image signal acquired under specific lighting conditions that are the same as the lighting conditions used for the training data for the monocular depth estimation model 18. The monocular depth estimation model 18 is a training model trained using a group of images that have a correlation between depth and brightness under specific lighting conditions.

The lighting conditions may be those commonly used in actual imaging environments. For example, in an endoscope, the light source is the tip of a thin fiber that guides light from a light source device. In this case, the specific lighting conditions indicate a lighting environment using a single point light source LS. The monocular depth estimation model 18 is a learning model that learns the correlation between depth and brightness based on the inverse squared measurement of illuminance.

The image development unit 13 develops the RAW signal RD to generate an RGB image CI. The development process may include exposure adjustment, contrast adjustment, white balance adjustment, saturation adjustment, cropping, perspective adjustment, etc. The measurement processing unit 14 acquires the position of the observation target depicted in the RGB image CI. The measurement processing unit 14 estimates various information about the observation target based on the position information and depth information DI of the observation target, and outputs the estimation results ES to an external device.

For example, the measurement processing unit 14 can obtain the size of the observation target based on the depth information DI of the observation target. The size can be calculated based on the image sensor size, the size of the corresponding area projected onto the pixels, the focal length of the lens, and so on. For example, the measurement processing unit 14 obtains the size of the observation target based on the magnification ratio used during imaging. The measurement processing unit 14 measures the size of the target in the endoscopic image and calculates the size by multiplying it by a magnification ratio corresponding to the depth information to the target.

The measurement processing unit 14 can reconstruct the three-dimensional shape of the observation target based on the depth information DI of the observation target. The measurement processing unit 14 can acquire the outer shape of the observation target projected onto a two-dimensional plane from a predetermined direction based on the depth information DI of the observation target. The measurement processing unit 14 can acquire two points included in the observation target as target points, and acquire the distance between the target points based on the depth information DI of each target point.

The external input device 3 inputs position information of the points and ranges to be measured to the measurement processing unit 14. The input may be performed manually by the user, or information extracted from the RGB image CI by image analysis may be input automatically.

[3. Switching learning models according to lighting conditions]
FIG. 10 is a diagram illustrating switching of the learning model according to the lighting conditions.

Users can switch between multiple endoscopic camera units 2 depending on the application. Each endoscopic camera unit 2 has different lighting, optics, and imaging conditions. Therefore, a different monocular depth estimation model 18 is required depending on the individual conditions. The image distance acquisition unit 12 recognizes the type of endoscopic camera unit 2 and selects the appropriate monocular depth estimation model 18, allowing for accurate distance measurements.

For example, the image distance acquisition unit 12 has a memory 17. The memory 17 stores multiple monocular depth estimation models 18 that have been trained for each lighting environment. The distance measurement calculation unit 16 switches between and acquires the monocular depth estimation model 18 to be used from the memory 17 according to the lighting conditions under which the image is captured.

In the example of Figure 10, "Endoscope 1," "Endoscope 2," and "Endoscope 3" are prepared as multiple available endoscopic camera units 2A, 2B, and 2C. Memory 17 stores "Learning Model 1," "Learning Model 2," and "Learning Model 3" as monocular depth estimation models 18A, 18B, and 18C corresponding to "Endoscope 1," "Endoscope 2," and "Endoscope 3."

[4. System Variations]
FIG. 11 shows variations of the information processing system.

In the example of Figure 8, the image acquisition unit 11, image distance acquisition unit 12, image development unit 13, and measurement processing unit 14 are implemented in a single processor. However, the image acquisition unit 11, image distance acquisition unit 12, image development unit 13, and measurement processing unit 14 may be distributed across multiple processors.

For example, in the example of Figure 11, the image acquisition unit 11 and image development unit 13 are implemented in the image processing device 4, and the image distance acquisition unit 12 and measurement processing unit 14 are implemented by an external PC or server 5. By assigning the processing of each unit to an appropriate device depending on the processing load, the capabilities of the device can be effectively utilized.

[5. Hardware Configuration Example]
FIG. 12 is a diagram illustrating an example of the hardware configuration of the information processing device 1.

Information processing by information processing device 1 is realized, for example, by computer 1000. Computer 1000 has a CPU (Central Processing Unit) 1100, RAM (Random Access Memory) 1200, ROM (Read Only Memory) 1300, HDD (Hard Disk Drive) 1400, communication interface 1500, and input/output interface 1600. Each part of computer 1000 is connected by bus 1050.

CPU 1100 operates based on programs (program data 1450) stored in ROM 1300 or HDD 1400, and controls each component. For example, CPU 1100 loads programs stored in ROM 1300 or HDD 1400 into RAM 1200 and executes processing corresponding to the various programs.

ROM 1300 stores boot programs such as BIOS (Basic Input Output System) that are executed by CPU 1100 when computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

HDD 1400 is a computer-readable, non-transitory recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to an embodiment, which is an example of program data 1450.

The communication interface 1500 is an interface that allows the computer 1000 to connect to an external network 1550 (such as the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or mouse via the input/output interface 1600. The CPU 1100 also transmits data to output devices such as a display device, speaker, or printer via the input/output interface 1600. The input/output interface 1600 may also function as a media interface that reads programs recorded on a specified recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

For example, when computer 1000 functions as information processing device 1 according to an embodiment, CPU 1100 of computer 1000 executes an information processing program loaded onto RAM 1200 to realize the functions of the aforementioned components. HDD 1400 also stores the information processing program, various models, and various data according to the present disclosure. While CPU 1100 reads and executes program data 1450 from HDD 1400, as another example, it may also obtain these programs from another device via external network 1550. The processing of the present disclosure may also be performed by a processor other than CPU 1100, such as a GPU (Graphics Processing Unit).

6. Effects
The information processing device 1 has a RAW processing unit 15 and a distance measurement calculation unit 16. The RAW processing unit 15 acquires a normalized luminance image in which the luminance of each color is normalized from the RAW signal RD. The distance measurement calculation unit 16 acquires depth information DI of the subject SB from the normalized luminance image using a monocular depth estimation model 18 that has learned about the correlation between depth and luminance. In the information processing method disclosed herein, the processing of the information processing device 1 is executed by a computer 1000. A program disclosed herein causes the computer 1000 to realize the processing of the information processing device 1.

With this configuration, by using normalized brightness, variations between colors based on wavelength characteristics are reduced, allowing depth information DI to be calculated with high accuracy.

The information processing device 1 has an image acquisition unit 11. The image acquisition unit 11 acquires, as a RAW signal RD, an image signal acquired under specific lighting conditions that are the same as the lighting conditions used for the training data of the monocular depth estimation model 18.

With this configuration, learning results are accurately reflected in measurement results.

The specific lighting conditions indicate a lighting environment using a single point light source LS. The monocular depth estimation model 18 is a learning model that learns the correlation between depth and brightness based on the inverse squared measurement of illuminance.

With this configuration, depth information DI can be calculated with high accuracy based on the inverse square of the illuminance.

Monocular depth estimation model 18 is a learning model trained using a group of images that have a correlation between depth and brightness under specific lighting conditions.

This configuration allows for highly accurate measurements.

The information processing device 1 has a memory 17. The memory 17 stores multiple monocular depth estimation models 18 that have been trained for each lighting environment. The ranging calculation unit 16 switches between and acquires the monocular depth estimation model 18 to be used from the memory 17 according to the lighting conditions under which the image is captured.

This configuration enables accurate measurements in a variety of lighting environments.

The RAW processing unit 15 processes the RAW signal RD to emphasize the contours of the subject SB.

This configuration reduces errors in size measurements, etc.

The information processing device 1 has an image development unit 13 and a measurement processing unit 14. The image development unit 13 develops the RAW signal RD to generate an RGB image CI. The measurement processing unit 14 acquires the position of the observation target that appears in the RGB image CI.

With this configuration, the position of the observation target can be determined with high accuracy based on the RGB image CI.

The measurement processing unit 14 obtains the size of the object of observation based on the depth information DI of the object of observation.

This configuration allows the size of the object being observed to be determined with high accuracy.

The measurement processing unit 14 reconstructs the two-dimensional and three-dimensional shapes of the object taking into account the depth of the object based on the depth information DI of the object.

This configuration allows the two-dimensional and three-dimensional shapes of the object being observed to be determined with high accuracy, taking depth into account.

The measurement processing unit 14 acquires two points included in the observation target as target points. The measurement processing unit 14 acquires the distance between the target points based on the depth information DI of each target point.

This configuration allows the three-dimensional distance between target points to be determined with high accuracy.

Please note that the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[Note]
The present technology can also be configured as follows.
(1)
a RAW processing unit that acquires a normalized luminance image in which the luminance of each color is normalized from the RAW signal;
a distance measurement calculation unit that acquires depth information of a subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance;
An information processing device having the above.
(2)
an image acquisition unit that acquires, as the RAW signal, an image signal acquired under specific lighting conditions that are the same as lighting conditions used for training data of the monocular depth estimation model;
The information processing device according to (1) above.
(3)
the specific lighting condition indicates a lighting environment using a single point light source,
The monocular depth estimation model is a learning model that learns a correlation between the depth and the luminance based on an inverse squared illuminance measurement.
The information processing device according to (2) above.
(4)
The monocular depth estimation model is a learning model that is learned using a group of images having a correlation between depth and luminance under the specific lighting condition.
The information processing device according to (2) or (3) above.
(5)
a memory that stores a plurality of monocular depth estimation models that have been trained for each lighting environment;
the distance measurement calculation unit switches and acquires the monocular depth estimation model to be used from the memory in accordance with the lighting conditions under which the image is captured;
The information processing device according to any one of (2) to (4) above.
(6)
the RAW processing unit performs processing on the RAW signal to enhance the contours of the subject;
The information processing device according to any one of (1) to (5) above.
(7)
an image developing unit that develops the RAW signal to generate an RGB image;
a measurement processing unit for acquiring the position of an observation target appearing in the RGB image;
The information processing device according to any one of (1) to (6) above,
(8)
the measurement processing unit acquires a size of the observation target based on the depth information of the observation target.
The information processing device according to (7) above.
(9)
the measurement processing unit acquires the size of the observation target based on a magnification ratio at the time of photographing;
The information processing device according to (8) above.
(10)
the measurement processing unit reconstructs a three-dimensional shape of the observation target based on the depth information of the observation target.
The information processing device according to (7) above.
(11)
the measurement processing unit acquires an outer shape of the observation target projected onto a two-dimensional plane from a predetermined direction based on the depth information of the observation target.
The information processing device according to (7) above.
(12)
the measurement processing unit acquires two points included in the observation target as target points, and acquires the distance between the target points based on the depth information of each target point;
The information processing device according to (7) above.
(13)
A normalized luminance image in which the luminance of each color is normalized is obtained from the RAW signal;
acquiring depth information of the subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance;
10. A computer-implemented information processing method comprising:
(14)
A normalized luminance image in which the luminance of each color is normalized is obtained from the RAW signal;
acquiring depth information of the subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance;
A program that makes a computer do something.

REFERENCE SIGNS LIST 1 Information processing device 11 Image acquisition unit 13 Image development unit 14 Measurement processing unit 15 RAW processing unit 16 Distance measurement calculation unit 17 Memory 18 Monocular depth estimation model CI RGB image DI Depth information RD RAW signal

Claims (14)

a RAW processing unit that acquires a normalized luminance image in which the luminance of each color is normalized from the RAW signal;
a distance measurement calculation unit that acquires depth information of a subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance;
An information processing device having the above. an image acquisition unit that acquires, as the RAW signal, an image signal acquired under specific lighting conditions that are the same as lighting conditions used for training data of the monocular depth estimation model;
The information processing device according to claim 1 . the specific lighting condition indicates a lighting environment using a single point light source,
The monocular depth estimation model is a learning model that learns a correlation between the depth and the luminance based on an inverse squared illuminance measurement.
The information processing device according to claim 2 . The monocular depth estimation model is a learning model that is learned using a group of images having a correlation between depth and luminance under the specific lighting condition.
The information processing device according to claim 2 . a memory that stores a plurality of monocular depth estimation models that have been trained for each lighting environment;
the distance measurement calculation unit switches and acquires the monocular depth estimation model to be used from the memory in accordance with the lighting conditions under which the image is captured;
The information processing device according to claim 2 . the RAW processing unit performs processing on the RAW signal to enhance the contours of the subject;
The information processing device according to claim 1 . an image developing unit that develops the RAW signal to generate an RGB image;
a measurement processing unit for acquiring the position of an observation target appearing in the RGB image;
The information processing device according to claim 1 , further comprising: the measurement processing unit acquires a size of the observation target based on the depth information of the observation target.
The information processing device according to claim 7 . the measurement processing unit acquires the size of the observation target based on a magnification ratio at the time of photographing;
The information processing device according to claim 8 . the measurement processing unit reconstructs a three-dimensional shape of the observation target based on the depth information of the observation target.
The information processing device according to claim 7 . the measurement processing unit acquires an outer shape of the observation target projected onto a two-dimensional plane from a predetermined direction based on the depth information of the observation target.
The information processing device according to claim 7 . the measurement processing unit acquires two points included in the observation target as target points, and acquires the distance between the target points based on the depth information of each target point;
The information processing device according to claim 7 . A normalized luminance image in which the luminance of each color is normalized is obtained from the RAW signal;
acquiring depth information of the subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance;
10. A computer-implemented information processing method comprising: A normalized luminance image in which the luminance of each color is normalized is obtained from the RAW signal;
acquiring depth information of the subject from the normalized luminance image using a monocular depth estimation model that has learned about the correlation between depth and luminance;
A program that makes a computer do something.