WO2025173550A1 - Information processing device, information processing method, and program - Google Patents

Abstract

An information processing device according to the present invention comprises: an acquisition unit (for example, a communication unit (51)) that acquires a first image including a specific tooth, obtained by imaging the inside of an oral cavity; a detection unit (52) that detects a first tooth region of the specific tooth from the first image; and a selection unit (53) that, when the ratio of the first tooth region in the first image is equal to or greater than a prescribed value, selects the first image as a reference image of the specific tooth.

Description

Information processing device, information processing method, and program

This disclosure relates to an information processing device, an information processing method, and a program.

Patent document 1 discloses a device that sets tooth regions in an image of teeth lined up side by side.

Japanese Patent Application Laid-Open No. 2017-18152

Incidentally, it is desirable to perform processing such as setting plaque areas on images suitable for checking the condition of the oral cavity. However, Patent Document 1 does not disclose technology for acquiring images suitable for checking the condition of the oral cavity.

The present disclosure therefore provides an information processing device, information processing method, and program that can acquire images that are more suitable for checking the condition of the oral cavity.

An information processing device according to one aspect of the present disclosure includes an acquisition unit that acquires a first image including a specific tooth obtained by photographing the inside of the oral cavity, a detection unit that detects a first tooth region of the specific tooth from the first image, and a selection unit that selects the first image as a reference image of the specific tooth if the proportion of the first tooth region in the first image is equal to or greater than a predetermined value.

An information processing method according to one aspect of the present disclosure acquires an image containing a specific tooth obtained by photographing the inside of the oral cavity, detects the tooth region of the specific tooth from the image, and, if the proportion of the tooth region in the image is equal to or greater than a predetermined value, selects the image as a reference image for the specific tooth.

A program according to one aspect of the present disclosure is a program for causing a computer to execute the above-described information processing method.

According to one aspect of the present disclosure, it is possible to realize an information processing device or the like that can acquire images that are more suitable for checking the condition of the oral cavity.

FIG. 1 is a perspective view of an intraoral camera in an information processing system according to an embodiment. FIG. 2 is a schematic configuration diagram of an information processing system according to an embodiment. FIG. 3 is a block diagram illustrating a functional configuration of the mobile terminal according to the embodiment. FIG. 4 is a flowchart showing a first operation of the information processing system according to the embodiment. FIG. 5 is a diagram for explaining step S13 shown in FIG. FIG. 6 is a flowchart showing a second operation of the information processing system according to the embodiment. FIG. 7A is a first diagram for explaining step S24 shown in FIG. FIG. 7B is a second diagram for explaining step S24 shown in FIG. FIG. 8 is a flowchart showing the operation of evaluating the oral care state according to the embodiment. FIG. 9 is a diagram for explaining a shooting guide in the information processing system according to the first modification of the embodiment. FIG. 10 is a flowchart showing the operation of the information processing system according to the first modification of the embodiment. FIG. 11 shows a screen for selecting a tooth to be photographed according to the second modification of the embodiment.

The information processing device according to the first aspect of the present disclosure includes an acquisition unit that acquires a first image including a specific tooth obtained by photographing the inside of the oral cavity, a detection unit that detects a first tooth region of the specific tooth from the first image, and a selection unit that selects the first image as a reference image of the specific tooth if the proportion of the first tooth region in the first image is equal to or greater than a predetermined value.

This allows for the acquisition of a reference image with a larger tooth area than an image in which the proportion of the first tooth area is less than a predetermined value, making it easier to check the condition of a specific tooth. This makes it possible to acquire an image that is more suitable for checking the condition of the oral cavity.

Furthermore, for example, an information processing device according to a second aspect may be the information processing device according to the first aspect, wherein the acquisition unit further acquires a second image including the specific tooth obtained by photographing the oral cavity, the detection unit detects a second tooth region of the specific tooth from the second image, and the selection unit selects a second image having a second tooth region similar to the first tooth region of the first image as a comparison image to be compared with the reference image.

This makes it possible to obtain a comparison image in which the tooth region is similar to that of the reference image. Such a comparison image makes it easy to compare the state of the tooth region with that of the reference image. Therefore, it is possible to obtain an image that is more suitable for comparing and checking the state of the oral cavity.

Furthermore, for example, the information processing device according to the third aspect may be the information processing device according to the second aspect, wherein the acquisition unit acquires one or more of the second images, and the selection unit selects, from among the one or more second images, a second image in which the degree of overlap between the second tooth region in the second image and the first tooth region in the reference image is equal to or greater than a predetermined degree, as the comparison image.

This allows reference images and comparison images with large overlapping tooth regions to be selected, making it possible to calculate a more accurate plaque score, for example, when calculating a plaque score that indicates the condition of the oral cavity using the area of the tooth region.

Furthermore, for example, the information processing device according to the fourth aspect may be the information processing device according to the second or third aspect, and may further include a control unit that, when the imaging device that captures images of the oral cavity captures the second image, causes an imaging guide based on the reference image to be superimposed on the image acquired from the imaging device and displayed on the display device.

This helps the user capture a second image that is similar to the reference image.

Furthermore, for example, the information processing device according to the fifth aspect may be the information processing device according to the fourth aspect, and the imaging guide may include a silhouette, outline, or semi-transparent display of the specific tooth included in the reference image.

This can assist the user in capturing a second image similar to the reference image by displaying a silhouette, outline, or semi-transparent representation of a particular tooth.

Furthermore, for example, an information processing device according to a sixth aspect may be an information processing device according to any one of the second to fifth aspects, wherein the detection unit further detects a first plaque region of the specific tooth based on the reference image and a second plaque region of the specific tooth based on the comparison image, and further includes a score calculation unit that calculates a first plaque score based on the first tooth region and the first plaque region in the reference image and calculates a second plaque score based on the second tooth region and the second plaque region in the comparison image, and a comparison unit that compares oral care conditions based on the first plaque score and the second plaque score.

As a result, the plaque score is calculated using an image with a large tooth area, which can result in a more accurate plaque score. Therefore, it is possible to obtain an image that is more suitable for comparing plaque scores.

Furthermore, for example, an information processing device according to a seventh aspect may be the information processing device according to the sixth aspect, wherein the score calculation unit calculates the first plaque score based on the ratio of the number of pixels in the first plaque region to the number of pixels in the natural tooth region in the reference image, and calculates the second plaque score based on the ratio of the number of pixels in the second plaque region to the number of pixels in the natural tooth region in the comparison image.

This makes it possible to calculate a more accurate plaque score when calculating the plaque score based on the ratio of the number of pixels in the plaque area to the number of pixels in the natural tooth area.

Furthermore, for example, an information processing device according to an eighth aspect is an information processing device according to any one of the first to seventh aspects, and the selection unit may select the first image as the reference image if the ratio of the number of pixels in the first tooth region to the total number of pixels in the first image is equal to or greater than the predetermined value.

This allows the ratio of the number of pixels in the first tooth region to the total number of pixels to be used to select an image that is more suitable for checking the condition of the oral cavity.

Furthermore, for example, an information processing device according to a ninth aspect may be an information processing device according to any one of the second to seventh aspects, in which the first image is an image of the inside of the oral cavity before the user performs oral care treatment, and the second image is an image of the inside of the oral cavity after the user has performed oral care treatment.

This allows users to obtain reference images and comparison images that allow them to more appropriately compare before and after oral care treatment.

An information processing method according to a tenth aspect of the present disclosure acquires an image including a specific tooth obtained by photographing the inside of the oral cavity, detects the tooth region of the specific tooth from the image, and, if the proportion of the tooth region in the image is equal to or greater than a predetermined value, selects the image as a reference image for the specific tooth.

This achieves the same effects as the information processing device described above.

A program according to an eleventh aspect of the present disclosure is a program for causing a computer to execute the above-described information processing method.

This achieves the same effects as the information processing device described above.

Note that these general or specific aspects may be realized as a system, method, integrated circuit, computer program, or non-transitory recording medium such as a computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, or recording medium. The program may be pre-stored on the recording medium, or may be supplied to the recording medium via a wide area communication network, including the Internet.

Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in each figure, substantially identical components are assigned the same reference numerals, and duplicate explanations are omitted or simplified.

Furthermore, in this specification, terms indicating relationships between elements, such as "parallel," as well as numerical values and numerical ranges, are not expressions that express only the strict meaning, but also expressions that include a substantially equivalent range, for example, a difference of about a few percent (or about 10%).

Furthermore, in this specification, ordinal numbers such as "first" and "second" do not refer to the number or order of components, unless otherwise specified, but are used to avoid confusion and distinguish between components of the same type.

(Embodiment)
An information processing system according to this embodiment will be described below with reference to FIGS.

[1. Configuration of information processing system]
First, the configuration of an information processing system including an intraoral camera according to the present embodiment will be described with reference to Figures 1 to 3. Figure 1 is a perspective view of an intraoral camera 10 in the information processing system according to the present embodiment.

As shown in Figure 1, the intraoral camera 10 has a toothbrush-shaped housing that can be handled with one hand. The housing includes a head portion 10a that is placed in the user's oral cavity when taking a photograph, a handle portion 10b that is held by the user, and a neck portion 10c that connects the head portion 10a and the handle portion 10b.

The imaging unit 21 captures images of the surfaces of the dentition and dental plaque in the oral cavity irradiated with light including the wavelength range of blue light. The surfaces of the dentition include at least one of the cheek (outside) side surface of the dentition, the tongue (inside) side surface of the dentition, and the occlusal surface of the dentition. The dentition may also include, for example, one or more teeth. The imaging unit 21 is an example of an imaging device.

The photographing unit 21 is incorporated into the head unit 10a and neck unit 10c. The photographing unit 21 has an image sensor (not shown) and a lens (not shown) arranged on its optical axis LA.

The imaging element is a photographing device such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) element, and an image of the teeth is formed by a lens. The imaging element outputs a signal (image data) corresponding to the formed image to the outside. The image captured by the imaging element is also referred to as an RGB image. An RGB image is an image obtained by illuminating the dentition with blue light, and may be, for example, an image of the side of the dentition, or an image of the occlusal surface of the dentition. The image includes, for example, one or more images (e.g., time-series images) of the dentition along its alignment. The side of the dentition may be the lingual side or the buccal side. The RGB image includes, for example, one or more teeth.

The imaging unit 21 may further include an optical filter that blocks light of a color emitted from the illumination unit (illumination device) and transmits fluorescence emitted by plaque in response to that light. In this embodiment, the imaging unit 21 may include, as an optical filter, a blue light cut-off filter that cuts out blue wavelength light components contained in light incident on the imaging element. When light including the blue light wavelength range is irradiated onto teeth to detect plaque, if the light including the blue light wavelength range is strengthened to increase the excitation fluorescence of plaque, the blue pixel values will become dominant over the red and green pixel values, and the entire RGB image will appear blue. To address this, the blue light cut-off filter cuts out a portion of the light including the blue light wavelength range from the light before it enters the imaging element. Note that the imaging unit 21 does not necessarily have to include a blue light cut-off filter.

The intraoral camera 10 is also equipped with multiple first to fourth LEDs 23A-23D as an illumination unit that irradiates light onto the teeth to be photographed during photography. The first to fourth LEDs 23A-23D irradiate dental plaque with light of a color that causes the plaque to fluoresce (for example, single-color light). The first to fourth LEDs 23A-23D are, for example, blue LEDs that irradiate blue light including a wavelength whose peak is 405 nm (an example of a predetermined wavelength). Note that the first to fourth LEDs 23A-23D are not limited to blue LEDs, and may be any light source that irradiates light including the blue light wavelength range.

Figure 2 is a schematic diagram of an information processing system according to this embodiment. The information processing system according to this embodiment is generally configured so that the imaging unit 21 captures fluorescence emitted by plaque in response to light from the illumination unit 23, and from the one or more captured RGB images (one or more images), an image more suitable for checking the condition of the oral cavity is obtained. Furthermore, the information processing system according to this embodiment is configured to select two images that allow the condition of the oral cavity to be compared.

As shown in FIG. 2, the information processing system includes an intraoral camera 10 and a mobile terminal 50.

The intraoral camera 10 comprises a hardware unit 20, a signal processing unit 30, and a communication unit 40.

The hardware unit 20 is a physical element of the intraoral camera 10 and includes a photographing unit 21, a sensor unit 22, an illumination unit 23, and an operation unit 24.

The photographing unit 21 generates image data by photographing the teeth in the user's oral cavity. It can also be said that the photographing unit 21 generates image data by photographing the surface of the dentition in the oral cavity and the plaque, which are irradiated with light of a predetermined wavelength that excites the fluorescent substance contained in the plaque. The photographing unit 21 receives a control signal from the camera control unit 31, performs operations such as photographing in accordance with the received control signal, and outputs video or still image data obtained by photographing to the image processing unit 32. The photographing unit 21 has the above-mentioned image sensor, optical filter, and lens. The image data is generated based on light that has passed through the optical filter, for example. Furthermore, the image data is an image showing multiple teeth, but it is sufficient that the image shows at least one tooth.

The sensor unit 22 detects external light entering the imaging area of the RGB image. For example, the sensor unit 22 detects whether external light is entering the oral cavity. The sensor unit 22 is arranged, for example, near the imaging unit 21. The sensor unit 22 may be arranged, for example, in the head unit 10a of the intraoral camera 10, similar to the imaging unit 21. In other words, the sensor unit 22 is located inside the user's oral cavity when the imaging unit 21 takes an image. Note that instead of using the sensor unit 22, external light may be detected by the image processing unit 32 (described below) using the image captured by the imaging unit 21. In other words, the signal processing unit 30 (for example, the image processing unit 32) may have the functions of the sensor unit 22.

The lighting unit 23 irradiates light onto the area of the multiple areas in the oral cavity that will be photographed by the imaging unit 21. As is also known from quantitative light-induced fluorescence (QLF (Quantitative Light-Induced Fluorescence)), porphyrin, a substance produced by bacteria in dental plaque, is known to fluoresce a reddish-pink color (excited fluorescence) when irradiated with blue light. In this embodiment, the lighting unit 23 irradiates blue light onto the area that will be photographed by the imaging unit 21.

The lighting unit 23 has the above-mentioned first to fourth LEDs 23A to 23D. The first to fourth LEDs 23A to 23D, for example, emit light from different directions toward the shooting area. This makes it possible to prevent shadows from appearing in the shooting area.

Each of the first to fourth LEDs 23A to 23D is configured so that at least the dimming can be controlled. Each of the first to fourth LEDs 23A to 23D may also be configured so that the dimming and color adjustment can be controlled. The first to fourth LEDs 23A to 23D are arranged to surround the imaging unit 21.

The illumination unit 23 controls the illumination intensity (light emission intensity) according to the shooting area. The illumination intensity of each of the first to fourth LEDs 23A to 23D may be controlled uniformly, or may be controlled to differ from one another. The number of LEDs in the illumination unit 23 is not particularly limited, and may be one, or five or more. Furthermore, the illumination unit 23 is not limited to having an LED as a light source, and may also have other light sources.

The operation unit 24 accepts operations from the user. The operation unit 24 is configured, for example, with push buttons, but may also be configured to accept operations via voice, etc.

The hardware unit 20 may also include a battery (e.g., a secondary battery) that supplies power to each component of the intraoral camera 10, a coil for wireless charging by an external charger connected to a commercial power source, and an actuator required for at least one of composition adjustment and focus adjustment.

The signal processing unit 30 has functional components implemented by a CPU (Central Processing Unit) or MPU (Micro Processor Unit) that execute various processes described below, and a memory unit 35 such as a ROM (Read Only Memory) or RAM (Random Access Memory) that stores programs for causing each functional component to execute various processes. Specifically, the signal processing unit 30 has a camera control unit 31, an image processing unit 32, a control unit 33, a lighting control unit 34, and the memory unit 35.

The camera control unit 31 is mounted, for example, on the handle portion 10b of the intraoral camera 10 and controls the imaging unit 21. The camera control unit 31 controls at least one of the aperture and shutter speed of the imaging unit 21 in response to a control signal from the image processing unit 32, for example.

The image processing unit 32 is mounted, for example, on the handle unit 10b of the intraoral camera 10, acquires the RGB image (image data) captured by the imaging unit 21, performs image processing on the acquired RGB image, and outputs the processed RGB image to the camera control unit 31 and the control unit 33. The image processing unit 32 may also output the processed RGB image to the memory unit 35, and store the processed RGB image in the memory unit 35.

The image processing unit 32 is composed of, for example, a circuit, and performs image processing such as noise removal and edge enhancement on RGB images. Note that noise removal and edge enhancement processing may also be performed by the mobile terminal 50.

The RGB image output from the image processing unit 32 (the RGB image after image processing) may be transmitted to the mobile terminal 50 via the communication unit 40, and an image based on the transmitted RGB image may be displayed on the display unit 56 of the mobile terminal 50. This allows the user to be presented with an image based on the RGB image.

The control unit 33 is a control device that controls the signal processing unit 30. The control unit 33 controls each component of the signal processing unit 30 based on, for example, the detection results of external light, etc. by the sensor unit 22.

The lighting control unit 34 is mounted, for example, on the handle portion 10b of the intraoral camera 10 and controls the turning on and off of the first to fourth LEDs 23A to 23D. The lighting control unit 34 is composed of, for example, a circuit. For example, when a user performs an operation on the display unit 56 of the mobile terminal 50 to start the intraoral camera 10, a corresponding signal is sent from the mobile terminal 50 to the signal processing unit 30 via the communication unit 40. The lighting control unit 34 of the signal processing unit 30 turns on the first to fourth LEDs 23A to 23D based on the received signal.

In addition to the above programs, the memory unit 35 also stores RGB images (image data) captured by the image capture unit 21. The memory unit 35 is realized by, for example, semiconductor memory such as ROM or RAM, but is not limited to this.

The communication unit 40 is a wireless communication module for wireless communication with the mobile terminal 50. The communication unit 40 is mounted, for example, on the handle portion 10b of the intraoral camera 10, and communicates wirelessly with the mobile terminal 50 based on control signals from the signal processing unit 30. The communication unit 40 performs wireless communication with the mobile terminal 50 in accordance with existing communication standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark). RGB images are sent from the intraoral camera 10 to the mobile terminal 50 via the communication unit 40, and operation signals are sent from the mobile terminal 50 to the intraoral camera 10.

The mobile terminal 50 performs processing to obtain images more suitable for checking the condition of the oral cavity, for example, by irradiating the teeth with light including the wavelength range of blue light, and using RGB images of the surfaces of the dentition and dental plaque that are reacting to fluorescence. The mobile terminal 50 functions as a user interface for the information processing system. The mobile terminal 50 is also an example of an information processing device.

Figure 3 is a block diagram showing the functional configuration of the mobile terminal 50 according to this embodiment.

As shown in FIG. 3, the mobile terminal 50 includes a communication unit 51, a detection unit 52, a selection unit 53, a score calculation unit 54, a control unit 55, and a display unit 56. The mobile terminal 50 also includes a processor and memory. The memory may be a ROM or RAM, and may store programs executed by the processor. The communication unit 51, the detection unit 52, the selection unit 53, the score calculation unit 54, the control unit 55, and the display unit 56 are implemented by a processor that executes programs stored in the memory. The mobile terminal 50 may be implemented, for example, by a smartphone or tablet device capable of wireless communication.

The communication unit 51 is a wireless communication module for wireless communication with the intraoral camera 10. The communication unit 51 acquires RGB images from the intraoral camera 10. Specifically, the communication unit 51 acquires images including one or more teeth generated by the imaging unit 21. The RGB images are images obtained by the intraoral camera 10 by irradiating the teeth with light including the wavelength range of blue light and capturing images of the teeth that are undergoing a fluorescent reaction. The communication unit 51 may also include a wired communication module for wired communication with the intraoral camera 10. The communication unit 51 is an example of an acquisition unit. Hereinafter, RGB images will also be referred to as images.

The detection unit 52 detects the tooth region and plaque region of a specific tooth from the image acquired by the communication unit 51. Because teeth and plaque emit different fluorescence, the detection unit 52 detects the tooth and plaque based on the color of the fluorescent (excited fluorescence) portion in the image. Specifically, the detection unit 52 detects the first tooth region, second tooth region, first plaque region, and second plaque region, which will be described later. The first tooth region and second tooth region are natural tooth regions.

As explained above, when irradiated with blue light, dental plaque (dental plaque region) fluoresces a reddish-pink color (excitation fluorescence). It is also known that when teeth (dental regions) are irradiated with excitation light, excitation fluorescence is emitted from the dentin, which passes through the enamel and fluoresces green. Furthermore, fillings left behind after caries treatment (e.g., metal inlays) do not emit excitation fluorescence under blue LED light, and are captured darkly (at low brightness) on the camera. In other words, dental regions and plaque regions in the image are distinguishable, and within dental regions, natural dental regions and fillings are distinguishable. Hereinafter, dental regions (natural dental regions) refer to the areas of teeth excluding fillings. Note that any known technology may be used to detect dental regions and plaque regions.

The selection unit 53 selects an image that is more suitable for checking the condition of the oral cavity based on the image acquired by the communication unit 51. If the proportion of the tooth region in the image is equal to or greater than a first predetermined value, the selection unit 53 selects the first image as a reference image of a specific tooth. For example, the selection unit 53 may select an image as a reference image of a specific tooth if the proportion of the number of pixels in the tooth region to the total number of pixels in the image is equal to or greater than a first predetermined value. The first predetermined value is set in advance and stored in a memory unit (not shown) of the mobile terminal 50. The first predetermined value is an example of a predetermined value.

The selection unit 53 may also perform processing to select a comparison image to be compared with the reference image.

The score calculation unit 54 calculates a plaque score that indicates the degree of plaque based on the tooth region and plaque region in the image. The plaque score is a numerical value that indicates the condition of the oral cavity. In this embodiment, the score calculation unit 54 calculates the plaque score based on the ratio of the plaque area (e.g., the number of pixels in the plaque) or the amount of fluorescence from the plaque to the tooth area (e.g., the number of pixels in the tooth region). The score calculation unit 54 calculates the plaque score, for example, based on the following formula 1.

Plaque score = plaque area / tooth area...Equation 1

Note that the tooth area here refers to the area of the natural tooth region in the image.

Note that the method for calculating the plaque score is not limited to this, and it may be calculated using other methods that use the area of the tooth region.

The control unit 55 performs various processes on the plaque score calculated by the score calculation unit 54. The control unit 55 may, for example, compare oral care conditions based on the plaque scores of two images. The control unit 55 may, for example, compare the plaque scores of images taken before and after an oral care treatment such as brushing teeth. The control unit 55 may, for example, evaluate the effectiveness of an oral care treatment based on the plaque score of the oral care treatment. In this way, the control unit 55 may function as a comparison unit that compares oral care conditions.

The display unit 56 is a display device included in the mobile terminal 50, and displays the plaque score calculated by the score calculation unit 54, the oral care status comparison results by the control unit 55, and the like. The display unit 56 is realized, for example, by a liquid crystal display panel, but is not limited to this. The display unit 56 is an example of a display device.

2. Operation of Information Processing System
Next, the operation of the information processing system configured as described above will be described with reference to FIGS. 4 to 8. An example of checking for missed brushing areas by comparing images before and after tooth brushing will be described below. FIG. 4 is a flowchart showing a first operation (information processing method) of the information processing system according to this embodiment. FIG. 4 describes the operation of selecting an appropriate image as an image before tooth brushing. Note that in this embodiment, the information processing system only needs to execute at least the operation shown in FIG. 4 (for example, the operation of selecting a before image) among the operations shown in FIGS. 4, 6, and 8.

As shown in FIG. 4, the communication unit 51 acquires a before image from the intraoral camera 10, which is an image of the inside of the user's oral cavity before brushing (S11). The before image is an example of a first image, and in this case, is an image that includes a specific tooth that is the target tooth to be checked for any remaining brushing.

Next, the control unit 55 determines whether there are multiple before images (S12). The control unit 55 determines whether there are multiple before images acquired by the communication unit 51 in step S11.

Next, if the control unit 55 determines that there are multiple before images (Yes in S12), the selection unit 53 determines whether any of the multiple before images has a tooth area ratio equal to or greater than a first predetermined value (S13).

In step S13, the detection unit 52 first detects the tooth area of a specific tooth in each of the multiple before images, and calculates the tooth area ratio by dividing the area (e.g., number of pixels) of the tooth area in the before image by the area (e.g., number of pixels) of the entire before image.

Figure 5 is a diagram for explaining step S13 shown in Figure 4. Images A and B are schematic diagrams of before images of the same tooth (specific tooth) that have been binarized. The before image here is an image of the occlusal surface of a first molar. In the schematic diagram, the natural tooth area is shown without hatching, and areas other than the natural tooth area are shown with diagonal hatching. Furthermore, image B is an image in which the area occupied by the natural tooth area is smaller on the entire screen because the photographing position has been shifted to the right compared to image A.

As shown in images A and B in Figure 5, the detection unit 52, for example, binarizes the tooth area and other areas in the before image and calculates the proportion of natural tooth area in the image. In image A, the proportion of natural tooth area (tooth area proportion) is 62%, and in image B, the proportion of natural tooth area is 27%.

For example, when calculating the plaque score using Equation 1, the plaque score for Image B is more susceptible to the influence of plaque remaining at the boundary between the natural tooth and the prosthesis than for Image A. Because plaque tends to remain at the boundary, if Image B is an image taken after oral care treatment, the plaque score may increase despite the fact that the oral care treatment has been performed. In other words, if the proportion of natural tooth area is small, there is a risk that the plaque score will not be calculated accurately. Therefore, in the present disclosure, as shown in step S13, a process is performed to extract images in which the tooth area proportion is equal to or greater than a first predetermined value.

The first predetermined value is preferably a high value from the perspective of accurately calculating the plaque score, and may be 40% or more, more preferably 50% or more, and even more preferably 60% or more. Furthermore, the first predetermined value may be set to a different value for each type of tooth (e.g., "incisor," "canine," "molar"), or the same value may be set.

The detection unit 52 calculates the tooth area ratio for each of the multiple before images. The selection unit 53 then determines whether or not there is an image among the multiple before images in which the tooth area ratio is equal to or greater than a first predetermined value.

Referring again to FIG. 4, next, if the selection unit 53 determines that there is an image in which the tooth area ratio is equal to or greater than a first predetermined value (Yes in S13), it selects a before image in which the tooth area ratio satisfies a predetermined condition (S14). The predetermined condition may be that the tooth area ratio is close to the average, median, or mode of the tooth area ratios in one or more before images in which the tooth area ratio is equal to or greater than the first predetermined value, or that the tooth area ratio is the maximum value. For example, the selection unit 53 may select a before image that is closest to the average value of the tooth area ratios in the one or more before images in which the tooth area ratio is equal to or greater than the first predetermined value.

Next, the selection unit 53 determines the before image whose tooth area ratio satisfies a predetermined condition, i.e., the selected before image, as the reference image (S15). In other words, the selection unit 53 selects, from one or more before images, a before image that satisfies the predetermined condition as the reference image. The selection unit 53 stores the reference image in the storage unit.

Note that although the selection unit 53 determines one before image as the reference image, it may also determine, for example, two or more before images that satisfy predetermined conditions as the reference image. In this way, the selection unit 53 functions as a first selection unit for selecting a reference image from one or more before images.

Furthermore, if the control unit 55 determines that there are not multiple before images (No in S12), the selection unit 53 determines whether the tooth area ratio of the before image is equal to or greater than a first predetermined value (S16), and if it determines that the tooth area ratio is equal to or greater than the first predetermined value (Yes in S16), proceeds to step S15. For example, if the ratio of the number of pixels in the tooth region to the total number of pixels in the before image is equal to or greater than a first predetermined value, the selection unit 53 may select the before image as the reference image. The pixel number ratio is synonymous with the area ratio.

Furthermore, if the selection unit 53 determines that there are no images in which the tooth area ratio is equal to or greater than the first predetermined value (No in S13), or if the selection unit 53 determines that the tooth area ratio is not equal to or greater than the first predetermined value (No in S16), the control unit 55 issues a notification to re-photograph the oral cavity (S17, S18) because there are no before images that can be used as reference images. This makes it possible to prevent plaque scores from being calculated for teeth that are mostly covered by metal inlays, or for images in which natural teeth are not visible or are visible but are small.

Next, the operation of selecting an appropriate image as an image after brushing teeth will be described with reference to Figures 6 to 7B. Figure 6 is a flowchart showing the second operation (information processing method) of the information processing system according to this embodiment.

As shown in FIG. 6, the communication unit 51 acquires an After image, which is an image of the user's oral cavity after brushing, from the intraoral camera 10 (S21). The After image is an example of a second image, and in this case, it is an image that includes a specific tooth that is the target tooth for which brushing is to be checked.

Next, the control unit 55 determines whether there are multiple After images (S22). The control unit 55 determines whether there are multiple After images acquired by the communication unit 51 in step S21.

Next, if the control unit 55 determines that there are multiple After images (Yes in S22), the selection unit 53 determines whether any of the multiple After images has a tooth area ratio equal to or greater than a first predetermined value (S23). The first predetermined value used in step S23 is, for example, the same value as the first predetermined value used in step S13, but is not limited to this. The first predetermined value used in step S23 may be changed, for example, depending on the tooth area ratio of the Before image determined as the reference image in step S15.

Note that the determination method in step S23 is the same as the determination method in step S13, and so a description thereof will be omitted.

Next, if the selection unit 53 determines that there is an image with a tooth area ratio equal to or greater than a first predetermined value (Yes in S23), it further determines whether there is an image with a tooth area ratio equal to or greater than the first predetermined value that has an overlapping degree with the reference image equal to or greater than a second predetermined value (S24). From the one or more After images for which the determination in step S23 is Yes is made, the selection unit 53 extracts After images that have a similar angle of view to the reference image, i.e., that are similar to the reference image. For example, the selection unit 53 may select, as the comparison image, an After image that has a second tooth area similar to the first tooth area in the Before image. Furthermore, for example, the selection unit 53 may select, as the comparison image, an After image from one or more second images that shows the same tooth as the tooth included in the reference image, captured from a position closer (e.g., the closest) to the position at which the reference image was captured. Note that the overlapping degree is an example of the overlapping degree, and the second predetermined value is an example of the predetermined degree.

In addition, the determination of whether the tooth is the same as the tooth included in the reference image may be made by any known method, and can be made, for example, based on the posture (position and orientation) of the intraoral camera 10 when the reference image and comparison image were captured, the shape of the tooth (for example, pattern matching based on the shape of the tooth included in the reference image), etc.

The selection unit 53 extracts at least three feature points on the contours of the natural teeth contained in the reference image and the after image, and performs image correction using the feature points in the reference image and the after image to align the relative positions of each feature point so that their coordinates match. Image correction will be explained with reference to Figures 7A and 7B.

FIGS. 7A and 7B are diagrams illustrating the image correction performed in step S24 shown in FIG. 6. FIG. 7A is a diagram illustrating the process of adjusting the relative position when the after image is misaligned in the X-axis and Y-axis directions with respect to the reference image, and FIG. 7B is a diagram illustrating the process of adjusting the relative position when the after image is rotated with respect to the reference image. Note that (c) of FIG. 7A and (c) of FIG. 7B are enlarged images for convenience.

(a) in Figure 7A shows the reference image, and (b) in Figure 7A shows the after image for which step S23 was determined to be Yes, which shows deviations in the X-axis and Y-axis directions. Deviations in the X-axis and Y-axis directions can occur, for example, when the position at which the reference image was captured and the position at which the after image was captured are misaligned in the X-axis and Y-axis directions.

In this case, as shown in (c) of Figure 7A, the selection unit 53 uses feature points to shift at least one of the reference image and the after image parallel to the X-axis and Y-axis directions so that the contours of the tooth region (natural tooth region) in the reference image and the tooth region (natural tooth region) in the after image overlap. Shifting parallel is an example of image correction.

(a) in Figure 7B shows the reference image, and (b) in Figure 7B shows the after image for which step S23 was determined to be Yes, in which rotation has occurred. Rotation can occur, for example, when the orientation of the intraoral camera 10 when the after image was captured is such that the intraoral camera 10 has rotated around the optical axis of the intraoral camera 10 relative to the orientation of the intraoral camera 10 when the reference image was captured.

In this case, as shown in (c) of Figure 7B, the selection unit 53 uses the feature points to rotate at least one of the reference image and the after image so that the contours of the tooth region (natural tooth region) in the reference image and the tooth region (natural tooth region) in the after image overlap. Rotation is an example of image correction.

Then, the selection unit 53 calculates the area of the overlapping portion of the tooth regions when the contours of the tooth regions overlap, as shown in (c) of Figure 7A or (c) of Figure 7B.

As shown in Figures 7A and 7B, the selection unit 53 aligns the reference image and the after image relative to each other based on feature points (e.g., feature points on the contour line) in the outline of the tooth region in the reference image and feature points in the outline of the tooth region in the after image, and identifies overlapping areas between the aligned reference image and the after image. In Figures 7A (c) and 7B (c), the overlapping areas between the tooth region in the reference image and the tooth region in the after image are indicated by dot hatching.

Note that image correction is not limited to the above and may also include, for example, enlarging or reducing the image.

The degree of overlap may be the value obtained by dividing the area (or number of pixels) of the overlapping portion by the area (or number of pixels) of the tooth region in the reference image, or it may be the area of the overlapping portion itself. The area of the overlapping portion may also be the number of pixels in the overlapping portion.

Note that the tooth region in the images shown in Figures 7A(a) and 7B(a) is an example of a first tooth region, and the tooth region in the images shown in Figures 7A(b) and 7B(b) is an example of a second tooth region.

Referring again to FIG. 6, next, the selection unit 53 determines, as the comparison image, an After image whose degree of overlap is equal to or greater than a second predetermined value (S25). In other words, the selection unit 53 selects, from one or more After images, an After image whose degree of overlap is equal to or greater than a second predetermined value as the comparison image.

Such reference and comparison images are similar images with many commonalities in the tooth regions shown, and can be images that make it easy to compare the state of the teeth. It can also be said that the selection unit 53 selects, as the comparison image, an After image that has a high similarity to the reference image of a specific tooth from, for example, one or more After images. The similarity is, for example, a value that corresponds to the degree of overlap of the tooth regions. It is assumed that the higher the degree of overlap, the higher the similarity. The overlap is an example of a value based on the area of the overlapping portion between the tooth region of the reference image and the tooth region of the After image. The value based on the area is not limited to the degree of overlap, and may be, for example, the area of the tooth region.

The selection unit 53 associates the comparison image with the reference image and stores it in the storage unit. Note that the selection unit 53 determines one After image as the comparison image, but may also determine, for example, two or more After images whose degree of overlap is equal to or greater than a second predetermined value as the comparison images. In this way, the selection unit 53 functions as a second selection unit for selecting a comparison image from one or more After images.

Furthermore, if the control unit 55 determines that there are not multiple after images (No in S22), the selection unit 53 determines whether the tooth area ratio of the after image is equal to or greater than a first predetermined value (S26), and if it determines that the tooth area ratio is equal to or greater than the first predetermined value (Yes in S26), the selection unit 53 proceeds to step S24.

Furthermore, if the selection unit 53 determines that the degree of overlap is less than the second predetermined value (No in S24), if it determines that there are no images with a tooth area ratio equal to or greater than the first predetermined value (No in S23), or if it determines that the tooth area ratio is not equal to or greater than the first predetermined value (No in S26), the control unit 55 issues a notification to re-photograph the oral cavity (S27, S28) because there are no after images that can be used as comparison images.

In step S24, the selection unit 53 may determine whether the difference between the proportion of tooth regions in the After image and the proportion of tooth regions in the reference image is within a third predetermined value, and select the After image for which it is determined that the difference is within the third predetermined value as the comparison image. The third predetermined value is set in advance and stored in the storage unit.

Next, the operation of evaluating the condition of the oral cavity using the reference image and comparison image determined above will be described with reference to Figure 8. Figure 8 is a flowchart showing the operation (information processing method) of evaluating the oral care condition according to this embodiment.

As shown in FIG. 8, the detection unit 52 detects plaque regions in the reference image and the comparison image (S41). The detection unit 52 detects areas in the reference image and the comparison image where porphyrin, a substance produced by bacteria in plaque, fluoresces reddish pink (excited fluorescence) as plaque regions. The plaque region of a specific tooth based on the reference image is an example of a first plaque region, and the plaque region of a specific tooth based on the comparison image is an example of a second plaque region.

Next, the score calculation unit 54 calculates plaque scores for the reference image and the comparison image (S42). The score calculation unit 54 calculates a plaque score (first plaque score) indicating the condition of the teeth based on the first tooth region and the first plaque region for the reference image, and calculates a plaque score (second plaque score) based on the second tooth region and the second plaque region for the comparison image. For example, the score calculation unit 54 calculates the first plaque score based on the ratio of the number of pixels in the first plaque region to the number of pixels in the natural tooth region in the reference image, and calculates the second plaque score based on the ratio of the number of pixels in the second plaque region to the number of pixels in the natural tooth region in the comparison image. The first plaque score and the second plaque score are plaque scores for the same tooth.

Next, the control unit 55 evaluates the user's oral care status based on the plaque scores of the reference image and the comparison image (S43). If the reference image and the comparison image are images before and after oral care treatment (e.g., before and after brushing), the control unit 55 may evaluate the effectiveness of the oral care treatment based on the first plaque score and the second plaque score. For example, the control unit 55 may calculate a plaque score for each tooth, thereby identifying which teeth have been left unbrushed the most.

The control unit 55 can also be said to compare the oral care state before and after the oral care treatment based on the first plaque score and the second plaque score. In this way, the control unit 55 may function as a comparison unit that compares the oral care state.

Next, the control unit 55 causes the display unit 56 to display the evaluation results of the oral care state (S44). The control unit 55 may cause the display unit 56 to display the first plaque score and the second plaque score, or may cause the display unit 56 to display a numerical value or level indicating the effectiveness of the oral care treatment based on the first plaque score and the second plaque score.

(First Modification of the Embodiment)
The information processing system according to this modification will be described below with reference to Figures 9 and 10. The following description will focus on differences from the embodiment, and descriptions of content that is the same as or similar to the embodiment will be omitted or simplified. The configuration of the information processing system according to this modification may be the same as the information processing system according to the embodiment, and description thereof will be omitted. The following description will also use the reference numerals of the information processing system according to the embodiment.

FIG. 9 is a diagram illustrating the imaging guide in the information processing system according to this modified example. This modified example differs from the information processing system according to the embodiment in that a imaging guide image is displayed on the display unit 56 when capturing an after image.

(a) in Figure 9 shows a reference image, and (b) in Figure 9 shows a photography guide image for photography guidance based on the reference image. The photography guide image is an image that includes, as a photography guide, either a silhouette, outline, or semi-transparent display of a specific tooth included in the reference image. In (b) in Figure 9, the tooth included in the reference image and its surrounding outline are shown with dashed lines.

(c) in Figure 9 shows an image in which a shooting guide image is superimposed on the captured image. In (c) in Figure 9, the captured image displayed by the light received by the shooting unit 21 is shown by a solid line, and the shooting guide image superimposed on the captured image is shown by a dashed line. The captured image here is an image that has not yet been captured, but will be displayed on the display unit 56 when a shooting operation is performed. Note that the captured image, i.e., the After image, does not include a shooting guide.

The user uses the intraoral camera 10 to capture an image of the oral cavity after oral care treatment (after image). At that time, the user adjusts the shooting position, etc., while checking the captured image displayed on the display unit 56 using light received by the imaging unit 21. In this modified example, a shooting guide image is superimposed on the captured image to adjust the shooting position, etc.

This allows the user to refer to the imaging guide and, for example, adjust the posture (position and orientation) of the intraoral camera 10 while checking the position of the imaging guide in the imaging guide image and the teeth in the captured image. For example, by capturing an image when the imaging guide in the imaging guide image and the teeth in the captured image match or are closer than a predetermined distance, an After image captured at a position close to the reference image can be obtained.

The display of the imaging guide image on the display unit 56 is executed, for example, by the control unit 55. For example, when the user takes an after image using the intraoral camera 10, the control unit 55 superimposes the imaging guide on the image (taken image) acquired from the intraoral camera 10 and displayed on the display unit 56.

Next, the operation of the information processing system according to this modified example will be described with reference to Figure 10. Figure 10 is a flowchart showing the operation of the information processing system according to this modified example (information processing system).

As shown in Figure 10, the information processing system executes step S31 in addition to the flowchart shown in Figure 6.

The control unit 55 superimposes a shooting guide on the display of the display unit 56 before the intraoral camera 10 captures an After image (S31). For example, the control unit 55 may display the shooting guide on the display unit 56 when it receives an operation from the user indicating that an After image is to be captured. The display of the shooting guide can be set to on or off, and the control unit 55 may display the shooting guide on the display unit 56 only when the setting for displaying the shooting guide is on, for example. The control unit 55 may also display the shooting guide on the display unit 56 when the notification of step S27 or S28 is given and re-shooting is to be performed.

(Modification 2 of the embodiment)
The information processing system according to this modification will be described below with reference to FIG. 11 . The following description will focus on differences from the embodiment, and descriptions of content that is the same as or similar to the embodiment will be omitted or simplified. The configuration of the information processing system according to this modification may be the same as the information processing system according to the embodiment, and description thereof will be omitted. The following description will also use the reference numerals of the information processing system according to the embodiment.

Figure 11 shows a screen for selecting the tooth to be photographed in this modified example. Figure 11 illustrates an example in which RGB images (e.g., reference image, after image) photographed by the user are stored in association with information indicating which tooth the image represents.

As shown in FIG. 11, the display unit 56 of the mobile terminal 50 displays a screen that allows the user to select the RGB image to be captured. The display unit 56 displays, for example, the rows of teeth of the upper and lower jaws. The display of the rows of teeth may be an image or an illustration.

The user selects the next tooth to photograph. Figure 11 shows an example in which the first molar on the left side of the lower jaw has been selected.

After receiving the user's selection, the control unit 55 acquires an RGB image from the intraoral camera 10, and stores the acquired RGB image in association with information indicating the first molar on the left side of the mandible. For example, the control unit 55 stores the reference image in association with information indicating the first molar on the left side of the mandible.

Note that tooth selection may be performed before or after capturing the RGB image. It is sufficient that the RGB image and the tooth type or position are stored in association with each other.

In addition, imaging direction information indicating whether the RGB image was taken from the cheek side, tongue side, or occlusal side of the tooth may also be stored in association with the RGB image. The imaging direction information may be input by the user, for example.

Furthermore, when an After image is captured, if the user selects a reference image to serve as a reference, the display unit 56 may display information indicating which tooth the reference image is an image of. For example, in a display of the rows of teeth in the oral cavity as shown in FIG. 11, the display unit 56 may indicate which tooth the reference image is an image of, or may display text information.

The degree of overlap between the reference image and the after image captured in this manner may then be determined.

Furthermore, the control unit 55 may store the tooth type and the amount of plaque in association with each other in the memory unit.

Furthermore, when the control unit 55 reads the reference image and the after image from the storage unit and displays them on the display unit 56, the tooth type or tooth type may be superimposed on the image.

(Other embodiments)
Although the information processing system according to the embodiments of the present disclosure has been described above, the present disclosure is not limited to the embodiments.

For example, in the above embodiments, an example has been described in which an intraoral camera 10 is used whose main purpose is to photograph teeth, but the intraoral camera 10 may also be an oral care device equipped with a camera. For example, the intraoral camera 10 may also be an oral irrigator equipped with a camera.

Furthermore, in the above embodiments, a mobile terminal 50 is used as an example of an information processing device, but the information processing device may also be a stationary information terminal or a server device.

Furthermore, in the above embodiments, the degree of overlap is given as an example of the degree of overlap, but the degree of overlap may also be indicated as a scale such as "high," "medium," or "low."

Furthermore, in the above embodiments, an example has been described in which the information processing device (in the embodiments, the mobile terminal 50) is equipped with a display unit, but this is not limited to this, and a display device capable of communicating with the information processing device may also be provided as a device separate from the information processing device.

Furthermore, each processing unit included in the information processing system according to the above-described embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

Furthermore, integrated circuits are not limited to LSIs, but may be realized using dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI manufacturing, or reconfigurable processors, which allow the connections and settings of circuit cells within the LSI to be reconfigured, may also be used.

Furthermore, in each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program appropriate for that component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

Furthermore, the division of functional blocks in the block diagram is one example; multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to other functional blocks. Furthermore, the functions of multiple functional blocks with similar functions may be processed in parallel or time-shared by a single piece of hardware or software.

Furthermore, the information processing device (e.g., mobile terminal 50) according to the above-described embodiments may be realized as a single device, or may be realized by multiple devices. When an information processing device is realized by multiple devices, the components of the information processing device may be distributed in any manner among the multiple devices. For example, at least some of the functions of the information processing device may be realized by the intraoral camera 10 (e.g., signal processing unit 30). When an information processing device is realized by multiple devices, the communication method between the multiple devices is not particularly limited, and may be wireless communication or wired communication. Furthermore, wireless communication and wired communication may be combined between the devices.

The present disclosure may also be realized as an information processing method executed by an information processing system. The present disclosure may also be realized as an intraoral camera, mobile terminal, or cloud server included in the information processing system.

Furthermore, the order in which each step in the sequence diagram is executed is merely an example to specifically explain the present disclosure, and orders other than those described above may also be used. Furthermore, some of the steps may be executed simultaneously (in parallel) with other steps.

Furthermore, one aspect of the present disclosure may be a computer program that causes a computer to execute each of the characteristic steps included in the information processing method shown in any of Figures 4, 6, and 8.

Furthermore, for example, the program may be a program to be executed by a computer. Furthermore, one aspect of the present disclosure may be a computer-readable non-transitory recording medium on which such a program is recorded. For example, such a program may be recorded on a recording medium and distributed or circulated. For example, by installing the distributed program in a device having another processor and having that processor execute the program, it becomes possible to cause that device to perform each of the above processes.

The information processing system, etc. relating to one or more aspects has been described above based on the embodiments, etc., but the present disclosure is not limited to these embodiments, etc. As long as it does not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art could conceive of to this embodiment, or forms constructed by combining components from different embodiments, may also be included within the scope of one or more aspects.

This disclosure can be applied to an information processing system for checking the condition of the oral cavity.

10 Intraoral camera 10a Head part 10b Handle part 20 Hardware part 21 Photography part (photography device)
22 Sensor unit 23 Illumination unit 23A First LED
23B Second LED
23C Third LED
23D Fourth LED
24 Operation unit 30 Signal processing unit 31 Camera control unit 32 Image processing unit 33 Control unit 34 Lighting control unit 35 Memory unit 40 Communication unit 50 Portable terminal (information processing device)
51 Communication unit (acquisition unit)
52 Detection unit 53 Selection unit (first selection unit, second selection unit)
54 Score calculation unit 55 Control unit (comparison unit)
56 Display section (display device)

Claims (11)

an acquisition unit that acquires a first image including a specific tooth by photographing the inside of the oral cavity;
a detection unit that detects a first tooth region of the specific tooth from the first image;
a selection unit that selects the first image as a reference image of the specific tooth when a proportion of the first tooth region in the first image is equal to or greater than a predetermined value. The acquisition unit further acquires a second image including the specific tooth obtained by photographing the oral cavity,
the detection unit detects a second tooth region of the specific tooth from the second image;
The information processing device according to claim 1 , wherein the selection unit selects a second image having a second tooth region similar to the first tooth region of the first image as a comparison image to be compared with the reference image. the acquisition unit acquires one or more second images;
The information processing device according to claim 2, wherein the selection unit selects, from among one or more of the second images, a second image in which the degree of overlap between the second tooth region in the second image and the first tooth region in the reference image is equal to or greater than a predetermined degree as the comparison image. The information processing device according to claim 2 or 3, further comprising a control unit that, when an imaging device that photographs the oral cavity photographs the second image, superimposes an imaging guide based on the reference image on an image acquired from the imaging device and displayed on a display device. The information processing device according to claim 4 , wherein the imaging guide includes a silhouette, an outline, or a semi-transparent display of the specific tooth included in the reference image. the detection unit further detects a first plaque region of the specific tooth based on the reference image and a second plaque region of the specific tooth based on the comparison image;
moreover,
a score calculation unit that calculates a first plaque score based on the first tooth region and the first plaque region in the reference image, and calculates a second plaque score based on the second tooth region and the second plaque region in the comparison image;
The information processing device according to claim 2 or 3, further comprising a comparison unit that compares an oral care state based on the first plaque score and the second plaque score. The score calculation unit
calculating the first plaque score based on a ratio of the number of pixels in the first plaque region to the number of pixels in the natural tooth region in the reference image;
The information processing device according to claim 6 , wherein the second plaque score is calculated based on a ratio of the number of pixels in the second plaque region to the number of pixels in the natural tooth region in the comparison image. The information processing device according to any one of claims 1 to 3, wherein the selection unit selects the first image as the reference image when a ratio of the number of pixels of the first tooth region to the total number of pixels of the first image is equal to or greater than the predetermined value. the first image is an image of the inside of the oral cavity taken before the user performs oral care treatment,
The information processing device according to claim 2 or 3, wherein the second image is an image of the inside of the oral cavity after the user has performed oral care treatment. Obtaining images of specific teeth obtained by photographing the oral cavity,
detecting a dental region of the specific tooth from the image;
If the proportion of the tooth region in the image is equal to or greater than a predetermined value, the image is selected as a reference image for the specific tooth. A program for causing a computer to execute the information processing method described in claim 10.