WO2025197661A1 - Periodontal disease detection system, periodontal disease detection method, and program - Google Patents

Abstract

This periodontal disease detection system is used for detecting periodontal disease of a user on the basis of an image obtained by capturing the interior of the oral cavity, said periodontal disease detection system comprising: a first acquisition unit that acquires a first RGB image obtained by irradiating the oral cavity of the user with light and capturing an imaging region including a specific tooth and a periodontal region of the specific tooth, the periodontal region including the gum; a second acquisition unit that acquires reference color data indicating a reference for the color of a natural tooth included in the imaging region, the reference color data corresponding to the user; an image processing unit (53) that generates a second RGB image obtained by adjusting, on the basis of the reference color data, the gains of at least two color components among the red, green, and blue color components constituting the natural tooth region within the first RGB image; and a periodontal disease detection unit (55) that detects periodontal disease of the user on the basis of image data based on the second RGB image.

Description

Periodontal disease detection system, periodontal disease detection method and program

This disclosure relates to a periodontal disease detection system, a periodontal disease detection method, and a program.

In recent years, with the widespread use of mobile devices equipped with cameras, systems have been proposed that allow users to take a photo of the area around their oral cavity with the camera on their mobile device before undergoing a dental examination, and then easily make an initial assessment of the condition of the oral cavity from the resulting image data. For example, Patent Document 1 discloses a system that, when a photo of the oral cavity region is taken with the camera on a mobile device, extracts image data of an oral cavity target region to be estimated from the image data of the oral cavity region, and estimates the condition of the oral cavity target region based on the image data of the extracted oral cavity target region.

Japanese Patent Application Laid-Open No. 2019-155027

When detecting periodontal diseases such as periodontal disease using images, it is desirable to improve the accuracy of detecting periodontal disease.

The present disclosure therefore provides a periodontal disease detection system, periodontal disease detection method, and program that can improve the accuracy of periodontal disease detection using images.

A periodontal disease detection system according to one aspect of the present disclosure is a system for detecting periodontal disease in a user based on an image captured inside the oral cavity, and includes: a first acquisition unit that irradiates light into the user's oral cavity to acquire a first RGB image of an image capture area including a specific tooth and the periodontal region of the specific tooth including the gums; a second acquisition unit that acquires reference color data that indicates the color standard of natural teeth included in the image capture area and is user-specific; an image processing unit that generates a second RGB image in which the gains of at least two of the red, green, and blue color components that make up the natural tooth region in the first RGB image are adjusted based on the reference color data; and a periodontal disease detection unit that detects periodontal disease in the user based on image data derived from the second RGB image.

A periodontal disease detection method according to one aspect of the present disclosure is a periodontal disease detection method executed by a periodontal disease detection system that detects periodontal disease in a user based on an image captured inside the oral cavity, the method including irradiating light into the user's oral cavity to obtain a first RGB image capturing an image of an imaged region including a specific tooth and the periodontal region of the specific tooth including the gums, obtaining reference color data that indicates the color standard of natural teeth included in the imaged region and that is specific to the user, generating a second RGB image in which the gains of at least two of the red, green, and blue color components that make up the natural tooth region in the first RGB image are adjusted based on the reference color data, and detecting periodontal disease in the user based on image data derived from the second RGB image.

A periodontal disease detection method according to one aspect of the present disclosure is a periodontal disease detection method executed by one or more processors, wherein the one or more processors irradiate light into the oral cavity of a user, output a first RGB image capturing an image of a capture area including a specific tooth and the periodontal region of the specific tooth including the gums, obtain a detection result of the user's periodontal disease based on the first RGB image, and perform predetermined processing on the obtained detection result, wherein the obtained detection result includes the detection result of the user's periodontal disease detected based on image data based on a second RGB image adjusted based on the reference color data corresponding to the user, where the gains of at least two of the red, green, and blue color components constituting the natural tooth region in the first RGB image are reference color data indicating the standard color of the natural tooth included in the capture area.

A program according to one aspect of the present disclosure is a program for causing a computer to execute the periodontal disease detection method described above.

According to one aspect of the present disclosure, it is possible to realize a periodontal disease detection system that can improve the accuracy of detecting periodontal disease using images.

FIG. 1 is a perspective view of an intraoral camera in a periodontal disease detection system according to a first embodiment. FIG. 2 is a schematic diagram of the periodontal disease detection system according to the first embodiment. FIG. 3 is a block diagram showing a functional configuration of the mobile terminal according to the first embodiment. FIG. 4 is a flowchart showing the operation of the periodontal disease detection system according to the first embodiment. FIG. 5 is a first diagram for explaining the periodontal region in the oral cavity according to the first embodiment. FIG. 6 is a second diagram for explaining the periodontal region in the oral cavity according to the first embodiment. FIG. 7A is a diagram for explaining image data extracted by an image data extraction unit according to the first embodiment. FIG. 7B is a diagram showing a specific example of a method for forming a rectangular region according to the first embodiment. FIG. 8A is a diagram illustrating an example of a first group rule base according to the first modification of the first embodiment. FIG. 8B is a diagram illustrating an example of a second group rule base according to the first modification of the first embodiment. FIG. 8C is a diagram illustrating an example of a third group rule base according to the first modification of the first embodiment. FIG. 9 is a diagram illustrating learning data according to the second modification of the first embodiment. FIG. 10 is a block diagram showing a functional configuration of a mobile terminal according to the second embodiment. FIG. 11 is a diagram for explaining a plurality of regions in the oral cavity according to the second embodiment. FIG. 12 is a diagram showing an example of an image obtained by capturing images of a plurality of regions according to the second embodiment. FIG. 13 is a diagram showing an example of an image used for learning the learning model according to the second embodiment. FIG. 14 is a flowchart showing the operation of the periodontal disease detection system according to the second embodiment. FIG. 15 is a flowchart showing the operation of the periodontal disease detection system according to the modified example of the second embodiment. FIG. 16 is a diagram showing the angle of the tip of the interdental papilla gingiva according to a modification of the second embodiment. FIG. 17A is a first diagram illustrating a method for calculating the angle of the tip of the interdental papilla gingiva according to a modified example of the second embodiment. FIG. 17B is a second diagram illustrating a method for calculating the angle of the tip of the interdental papilla gingiva according to a modified example of the second embodiment. FIG. 17C is a third diagram illustrating a method for calculating the angle of the tip of the interdental papilla gingiva according to a modified example of the second embodiment. FIG. 18A is a first diagram showing the radius of the tip of the interdental papilla gingiva according to a modified example of the second embodiment. FIG. 18B is a second diagram showing the radius of the tip of the interdental papilla gingiva according to a modified example of embodiment 2.

(Background to this disclosure)
When highlighting plaque in an image of the inside of a user's (e.g., a subject's) mouth, a process (achromatization process) may be performed to make the teeth a white region by adjusting the white balance gain so that the average values of R, G, and B are equal based on the tooth region. This process allows for effective highlighting of plaque.

When detecting periodontal disease using images, it is desirable to use images that more faithfully reproduce the color of the user's gums. However, color adjustment using the above-mentioned white balance involves uniform correction so that the average values of R, G, and B values are equal, making it difficult to achieve a natural color tone for the gums, which can vary from user to user. In other words, performing white balance makes it difficult to obtain an image suitable for detecting periodontal disease.

Furthermore, when assessing periodontal disease using image color data, it is necessary to correct color differences under light of different wavelengths or outdoors in order to improve the accuracy of the assessment. For example, in Patent Document 1, color calibration color patches are captured as images along with the inside of the oral cavity, and the color data of the periodontal image data is corrected using the color data of the color calibration color patches in the captured image. However, it is difficult to use color calibration color patches when photographing the back tooth area, where the cheek muscles and tongue must be avoided, and there is room for improvement.

As described above, it has traditionally been difficult to generate images that are more suitable for accurately detecting periodontal disease, which can be a factor in reducing the accuracy of periodontal disease detection.

The inventors of the present application therefore conducted extensive research into periodontal disease detection systems that can improve the accuracy of periodontal disease detection using images, and in particular, periodontal disease detection systems that can generate images that are more suitable for accurately detecting periodontal disease, and have devised the periodontal disease detection system described below.

A periodontal disease detection system according to a first aspect of the present disclosure is a system for detecting periodontal disease in a user based on an image captured inside the oral cavity, and includes: a first acquisition unit that irradiates light into the user's oral cavity to acquire a first RGB image of an image capture area including a specific tooth and the periodontal region of the specific tooth including the gums; a second acquisition unit that acquires reference color data that indicates the color standard of natural teeth included in the image capture area and is user-specific; an image processing unit that generates a second RGB image in which the gains of at least two of the red, green, and blue color components that make up the natural tooth region in the first RGB image are adjusted based on the reference color data; and a periodontal disease detection unit that detects periodontal disease in the user based on image data derived from the second RGB image.

This allows the color of the teeth and gums to be corrected based on reference color data tailored to the user. The color of the teeth and gums in the corrected image can be closer to the color of the user's teeth and gums, making it possible to more accurately detect periodontal disease from the corrected image. In other words, it is possible to generate an image that is more suitable for accurately detecting periodontal disease. This can improve the accuracy of periodontal disease detection using images.

Furthermore, for example, the periodontal disease detection system according to the second aspect may be the periodontal disease detection system according to the first aspect, and may further include an image data extraction unit that extracts, from the second RGB image, periodontal region image data including the free gingiva, which is the gingiva that surrounds the cervical region including the interdental papilla and marginal gingiva in a band shape, and a portion of the attached gingiva that is continuous with the free gingiva and extends from the bottom of the gingival sulcus to the gingival-alveolar junction, as the image data based on the second RGB image.

As a result, the image data based on the second RGB image is an image in which part of the tooth region in the second RGB image has been deleted. The tooth region is an area that is not necessary for detecting periodontal disease, and can be a factor in reducing detection accuracy. Therefore, periodontal disease can be detected more accurately by detecting periodontal disease using an image in which part of the tooth region in the second RGB image has been deleted.

Furthermore, for example, the periodontal disease detection system according to the third aspect may be the periodontal disease detection system according to the second aspect, and the periodontal region image data may further include the left and right interdental papillae and gingiva.

This includes both the left and right interdental papillae and gingiva, effectively preventing missed detection of periodontal disease.

Furthermore, for example, the periodontal disease detection system according to the fourth aspect may be a periodontal disease detection system according to any one of the first to third aspects, and the reference color data may be set based on the color of the user's natural teeth.

This can improve the accuracy of detecting periodontal disease for the user.

Furthermore, for example, a periodontal disease detection system according to a fifth aspect is a periodontal disease detection system according to any one of the first to fourth aspects, wherein the image processing unit identifies a specific pixel area including pixels whose values based on the pixel values of the second RGB image are within a predetermined range, and generates a third RGB image by adjusting the gain of at least two of the red, green, and blue color components that make up the area of the natural tooth in the first RGB image excluding the specific pixel area based on the reference color data, and the image data based on the second RGB image may be image data based on the third RGB image.

As a result, the third RGB image is an image corrected based on the color components of the tooth portion excluding the specific pixel area, and is therefore an image with more accurate gain adjustment. Detecting periodontal disease using image data based on such a third RGB image allows for more accurate detection of periodontal disease.

Furthermore, for example, a periodontal disease detection system according to a sixth aspect may be the periodontal disease detection system according to the fifth aspect, wherein the image processing unit generates an HSV image by converting the color space of the second RGB image into an HSV space, and identifies as the specific pixel area a pixel area in which one or more pixels of the HSV image that satisfy at least one of a first predetermined range for saturation, a second predetermined range for hue, and a third predetermined range for brightness are located.

This allows for more accurate identification of specific pixel areas by using an HSV image.

Furthermore, for example, a periodontal disease detection system according to a seventh aspect may be the periodontal disease detection system according to the fifth aspect, wherein the image processing unit identifies as the specific pixel area a pixel area in which one or more pixels are located, where the red pixel values of the multiple images constituting the natural tooth area in the second RGB image satisfy at least one of the following conditions: the multiple red pixel values are within a first range, the multiple green pixel values are within a second range, and the multiple blue pixel values are within a third range.

This allows specific pixel areas to be directly identified from the second RGB image, thereby reducing the processing load of the periodontal disease detection system.

Furthermore, for example, the periodontal disease detection system according to the eighth aspect may be a periodontal disease detection system according to any one of the third to seventh aspects, and the specific pixel region may include a plaque region.

This reduces the effect of plaque areas on the amount of gain adjustment, allowing for more accurate gain adjustment.

Furthermore, for example, the periodontal disease detection system according to the ninth aspect may be a periodontal disease detection system according to any one of the second to eighth aspects, and the image data extraction unit may extract the periodontal region from the tip of the interdental papilla gingiva of the specific tooth to the free gingival sulcus from the second RGB image as the periodontal region image data.

As a result, the periodontal region image data includes the periodontal region from the tip of the interdental papilla gingiva of the tooth to the free gingival sulcus, which is prone to changes when periodontal disease develops, allowing for more accurate detection of periodontal disease.

Furthermore, for example, a periodontal disease detection system according to a tenth aspect may be the periodontal disease detection system according to the ninth aspect, wherein the image data extraction unit extracts, from the second RGB image, a rectangular area that circumscribes the left and right sides of the contour of the specific tooth and includes from the tip of the specific tooth to the free gingival sulcus as the periodontal region image data.

This makes it possible to easily and reliably extract periodontal region image data that allows for more accurate detection of periodontal disease.

Furthermore, for example, a periodontal disease detection system according to an eleventh aspect may be a periodontal disease detection system according to any one of the first to tenth aspects, wherein the periodontal disease detection unit detects the periodontal disease of the user by inputting image data based on the second RGB image into a learning model that is trained to receive image data including a gingival region as input and output information related to periodontal disease in the image data.

This makes it possible to detect periodontal disease using a learning model.

Furthermore, for example, the periodontal disease detection system according to the twelfth aspect may be the periodontal disease detection system according to the eleventh aspect, and the learning model may output an estimated result of at least one of periodontal pocket depth, BOP (Bleeding On Probing), and GI (Gingival Index) value as information related to the periodontal disease.

This makes it possible to obtain estimated results for at least one of the following information related to periodontal disease: periodontal pocket depth, BOP (Bleeding On Probing), and GI (Gingival Index) value. Such information is useful for understanding periodontal disease.

Furthermore, for example, a periodontal disease detection system according to a thirteenth aspect is a periodontal disease detection system according to any one of the first to twelfth aspects, and the periodontal disease detection unit may input the R, G, and B values of the gum color acquired from image data based on the second RGB image and detect the user's periodontal disease in that area based on a periodontal disease detection tool created from the range of R, G, and B values in the RGB color space of normal gum color and the range of R, G, and B values of gum color for multiple stages of periodontal disease.

This allows periodontal disease to be effectively detected using the R, G, and B values.

Furthermore, for example, a periodontal disease detection system according to a fourteenth aspect may be a periodontal disease detection system according to any one of the first to twelfth aspects, wherein the periodontal disease detection unit may input at least one of the hue information, saturation information, and brightness information of the color of normal gums in the HSV color space and at least one range of the hue information, saturation information, and brightness information of the color of gums for each of multiple stages of periodontal disease, and detect the periodontal disease of the user in that area.

This makes it possible to effectively detect periodontal disease using at least one of hue information, saturation information, and brightness information.

Furthermore, for example, a periodontal disease detection system according to a fifteenth aspect is a periodontal disease detection system according to any one of the first to twelfth aspects, wherein the periodontal disease detection unit may input at least one of the hue information, saturation information, and luminance information of the color of normal gums in the HSL color space and at least one range of the hue information, saturation information, and luminance information of the color of gums for each of multiple stages of periodontal disease, and detect the periodontal disease of the user in the relevant area.

This makes it possible to effectively detect periodontal disease using at least one of hue information, saturation information, and brightness information.

Furthermore, for example, the periodontal disease detection system according to the sixteenth aspect may be a periodontal disease detection system according to any one of the first to fifteenth aspects, and may further include a tooth type identification unit that identifies the type of the specific tooth from the first RGB image or the second RGB image.

This makes it possible to obtain the type of tooth for which periodontal disease has been detected. For example, by outputting information indicating the type of tooth along with the detection results, it is possible to output information useful for confirming periodontal disease, etc.

Furthermore, for example, a periodontal disease detection system according to a seventeenth aspect is a periodontal disease detection system according to any one of the first to sixteenth aspects, and further includes an output unit that outputs information indicating the periodontal disease detected by the periodontal disease detection unit to a first information terminal of a dentist other than the user, and the output unit may further output examination necessity information regarding the need for the user to undergo examination, which information was acquired via the first acquisition unit and entered into the first information terminal, to a second information terminal of the user.

This allows dentists to understand the periodontal disease status of remote users (i.e., users they are not actually examining), thereby assisting in the treatment of periodontal disease.

Furthermore, for example, the periodontal disease detection system according to the eighteenth aspect may be a periodontal disease detection system according to any one of the first to seventeenth aspects, and may further include an output unit that outputs information indicating the periodontal disease detected by the periodontal disease detection unit to the user's second information terminal.

This allows users to be notified of their periodontal disease status.

Furthermore, for example, the periodontal disease detection system according to the 19th aspect may be a periodontal disease detection system according to any one of the first to 18th aspects, in which the second acquisition unit acquires the reference color data before acquiring the first RGB image, and may further include a storage unit that stores the reference color data.

This allows the reference color data to be stored in the memory unit. By obtaining the user's reference color data in advance and storing it in the memory unit, the convenience of the periodontal disease detection system is improved.

Furthermore, for example, a periodontal disease detection system according to a twentieth aspect may be a periodontal disease detection system according to any one of the first to nineteenth aspects, in which the reference color data is color data of the natural tooth obtained when a first light is irradiated onto the natural tooth, and the first RGB image is an image obtained when a second light different from the first light is irradiated onto the imaging area.

This allows the first RGB image obtained when the first light is irradiated to be corrected into a second RGB image that is closer to the image obtained when the second light is irradiated.

Furthermore, for example, a periodontal disease detection system according to a 21st aspect is a periodontal disease detection system according to any one of the first to 20th aspects, wherein the at least two color components include at least two of a first red pixel average value of multiple red pixel values possessed by multiple pixels constituting the natural tooth region in the first RGB image, a first green pixel average value of multiple green pixel values possessed by the multiple pixels, and a first blue pixel average value of multiple blue pixel values possessed by the multiple pixels, and the image processing unit may generate the second RGB image by adjusting the gain so that the difference between the at least two color components and the reference color data falls within a predetermined range.

This allows the gain to be adjusted to generate a second RGB image corrected using reference color data corresponding to the user.

A periodontal disease detection method according to a 22nd aspect of the present disclosure is a periodontal disease detection method executed by a periodontal disease detection system that detects periodontal disease in a user based on an image taken inside the oral cavity, and includes irradiating light into the user's oral cavity to obtain a first RGB image of an imaged area including a specific tooth and the periodontal area of the specific tooth including the gums, obtaining reference color data that indicates the color standard of natural teeth included in the imaged area and that corresponds to the user, generating a second RGB image in which the gains of at least two of the red, green, and blue color components that constitute the area of the natural tooth in the first RGB image are adjusted based on the reference color data, and detecting periodontal disease in the user based on image data based on the second RGB image. Furthermore, a periodontal disease detection method according to a 23rd aspect of the present disclosure is a periodontal disease detection method executed by one or more processors, wherein the one or more processors irradiate light into the oral cavity of a user, output a first RGB image capturing an imaging area including a specific tooth and the periodontal region of the specific tooth including the gums, obtain a detection result of the user's periodontal disease based on the first RGB image, and perform predetermined processing on the obtained detection result, wherein the obtained detection result includes the detection result of the user's periodontal disease detected based on image data based on a second RGB image adjusted based on the reference color data corresponding to the user, where the gains of at least two of the red, green, and blue color components constituting the natural tooth region in the first RGB image are reference color data indicating the reference color of the natural tooth included in the imaging area.

This achieves the same effect as the periodontal disease detection system described above.

Furthermore, a program according to a 24th aspect of the present disclosure is a program for causing a computer to execute the periodontal disease detection method according to the 22nd or 23rd aspect.

This achieves the same effect as the periodontal disease detection system described above.

Note that these general or specific aspects may be realized as a system, method, integrated circuit, computer program, or non-transitory recording medium such as a computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, or recording medium. The program may be pre-stored on the recording medium, or may be supplied to the recording medium via a wide area communication network, including the Internet.

Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in each figure, substantially identical components are assigned the same reference numerals, and duplicate explanations are omitted or simplified.

Furthermore, in this specification, terms indicating relationships between elements, such as "parallel," as well as numerical values and numerical ranges, are not expressions that express only the strict meaning, but also expressions that include a substantially equivalent range, for example, a difference of about a few percent (or about 10%).

Furthermore, in this specification, ordinal numbers such as "first" and "second" do not refer to the number or order of components, unless otherwise specified, but are used to avoid confusion and distinguish between components of the same type.

(Embodiment 1)
Hereinafter, the periodontal disease detection system according to this embodiment will be described with reference to FIGS. 1 to 7B.

[1-1. Configuration of periodontal disease detection system]
First, the configuration of a periodontal disease detection system according to this embodiment will be described with reference to Figures 1 to 3. Figure 1 is a perspective view of an intraoral camera 10 in the periodontal disease detection system according to this embodiment.

As shown in Figure 1, the intraoral camera 10 has a toothbrush-shaped housing that can be handled with one hand. The housing includes a head portion 10a that is placed in the user's oral cavity when taking a photograph, a handle portion 10b that is held by the user, and a neck portion 10c that connects the head portion 10a and the handle portion 10b.

The imaging unit 21 images the surface of the dentition and the periodontal region in the oral cavity, which are illuminated with light including the wavelength range of blue light. The surface of the dentition includes at least one of the buccal (outer) side surface of the dentition and the lingual (inner) side surface of the dentition. The dentition may also include, for example, one or more teeth. Blue light is an example of the second light.

The photographing unit 21 is incorporated into the head unit 10a and neck unit 10c. The photographing unit 21 has an image sensor (not shown) and a lens (not shown) arranged on its optical axis LA.

The imaging element is a photographing device such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) element, and an image of the teeth is formed by a lens. The imaging element outputs a signal (image data) corresponding to the formed image to the outside. The image captured by the imaging element is an example of a first RGB image or a first image.

Furthermore, the captured image is obtained by irradiating the dentition with blue light, and therefore has a blue tint. The captured image is, for example, an image of the side of the dentition and the periodontal region. The side of the dentition may be the side on the lingual side or the side on the buccal side. The side of the dentition may be the side on the maxillary side or the side on the mandibular side. The captured image is, for example, an image of a capture region including one or more teeth (an example of a specific tooth) and the periodontal region of the one or more teeth. The periodontal region is a region including the gingiva of the tooth. The gingiva is the periodontal tissue surrounding the root of a tooth and is also called the gum. The specific tooth may be any tooth, for example, a tooth in an area where a user or a medical professional such as a dentist wants to detect periodontal disease, or any tooth.

The imaging unit 21 may further include an optical filter that blocks light of a color emitted from the illumination unit (illumination device) and transmits fluorescence emitted by plaque in response to that light. In this embodiment, the imaging unit 21 may include, as an optical filter, a blue light cut-off filter that cuts out the blue wavelength light component contained in the light incident on the imaging element. When irradiating teeth with light including the blue wavelength range to detect plaque, if the light including the blue wavelength range is strengthened to increase the excitation fluorescence of plaque, the blue pixel values will become dominant compared to the red and green pixel values, and the entire captured image will appear blue. To address this, the blue light cut-off filter cuts out a portion of the light including the blue wavelength range from the light before it enters the imaging element. Note that the imaging unit 21 does not necessarily have to include a blue light cut-off filter.

The intraoral camera 10 is also equipped with multiple first to fourth LEDs 23A-23D as an illumination unit that irradiates light onto the teeth to be photographed during photography. The first to fourth LEDs 23A-23D irradiate dental plaque with light of a color that causes the plaque to fluoresce (for example, single-color light). The first to fourth LEDs 23A-23D are, for example, blue LEDs that irradiate blue light including a wavelength whose peak is 405 nm (an example of a predetermined wavelength). Note that the first to fourth LEDs 23A-23D are not limited to blue LEDs, and may be any light source that irradiates light including the blue light wavelength range.

Furthermore, the first to fourth LEDs 23A to 23D may be configured to emit light for capturing reference color data that serves as a reference for the color of the user's teeth. The light for capturing the reference color data includes light of a color different from blue light, such as white light. White light is an example of the first light.

Figure 2 is a schematic diagram of a periodontal disease detection system according to this embodiment. The periodontal disease detection system according to this embodiment is, generally speaking, an information processing system capable of detecting periodontal disease with greater accuracy. Specifically, the information processing system is configured such that the imaging unit 21 captures fluorescence emitted by plaque in response to light from the illumination unit 23, and from one or more captured images, an image more suitable for detecting periodontal disease is obtained, and periodontal disease is detected based on the acquired image.

As shown in Figure 2, the periodontal disease detection system includes an intraoral camera 10 and a mobile terminal 50.

The intraoral camera 10 comprises a hardware unit 20, a signal processing unit 30, and a communication unit 40.

The hardware unit 20 is a physical element of the intraoral camera 10 and includes a photographing unit 21, a sensor unit 22, an illumination unit 23, and an operation unit 24.

The imaging unit 21 generates image data by photographing the side of the dentition and the periodontal region in the oral cavity, which are illuminated with light of a specific wavelength that excites fluorescent substances contained in plaque. The imaging unit 21 receives control signals from the camera control unit 31, performs operations such as photographing in accordance with the received control signals, and outputs video or still image data obtained by photographing to the image processing unit 32. The imaging unit 21 has the above-mentioned image sensor, optical filter, and lens. The image data is generated, for example, based on light that has passed through the optical filter. The image data is an image that shows multiple teeth, but it is sufficient that the image shows at least one tooth. Note that plaque may adhere to the side of the dentition.

The photographing unit 21 may also perform photographing to obtain reference color data. The photographing unit 21 may generate image data for obtaining reference color data by photographing, for example, the side of the dentition and the periodontal region in the oral cavity illuminated with reference light such as white light. The reference light may be light from the illumination unit 23, or light from a light source external to the intraoral camera 10 (external light).

The sensor unit 22 detects external light entering the capture area of the captured image. For example, the sensor unit 22 detects whether external light is entering the oral cavity. The sensor unit 22 is, for example, arranged near the capture unit 21. The sensor unit 22 may, for example, be arranged in the head unit 10a of the intraoral camera 10, similar to the capture unit 21. In other words, the sensor unit 22 is located inside the user's oral cavity when the capture unit 21 captures images. Furthermore, the sensor unit 22 may detect whether white light within a predetermined chromaticity range or a predetermined color temperature is being irradiated into the oral cavity when capturing images to obtain reference color data.

The lighting unit 23 irradiates light onto the area of the multiple areas in the oral cavity that will be photographed by the imaging unit 21. As is also known from quantitative visible light induced fluorescence (QLF), bacteria in dental plaque are known to fluoresce a reddish-pink color (excited fluorescence) when irradiated with blue light; in this embodiment, the lighting unit 23 irradiates blue light onto the area that will be photographed by the imaging unit 21.

The lighting unit 23 has the above-mentioned first to fourth LEDs 23A to 23D. The first to fourth LEDs 23A to 23D, for example, emit light from different directions toward the shooting area. This makes it possible to prevent shadows from appearing in the shooting area.

Each of the first to fourth LEDs 23A to 23D is configured so that at least the dimming can be controlled. Each of the first to fourth LEDs 23A to 23D may also be configured so that the dimming and color adjustment can be controlled. The first to fourth LEDs 23A to 23D are arranged to surround the imaging unit 21.

The illumination unit 23 controls the illumination intensity (light emission intensity) according to the shooting area. The illumination intensity of each of the first to fourth LEDs 23A to 23D may be controlled uniformly, or may be controlled to differ from one another. The number of LEDs in the illumination unit 23 is not particularly limited, and may be one, or five or more. Furthermore, the illumination unit 23 is not limited to having an LED as a light source, and may also have other light sources.

The illumination unit 23 may also be configured to emit reference light for obtaining reference color data of the user's teeth. The reference light is emitted at a different timing than the blue light.

The operation unit 24 accepts operations from the user. The operation unit 24 is configured, for example, with push buttons, but may also be configured to accept operations via voice, etc. The operation unit 24 accepts operations from the user, for example, as to whether to take an image for detecting periodontal disease or to take an image for obtaining reference color data.

The hardware unit 20 may also include a battery (e.g., a secondary battery) that supplies power to each component of the intraoral camera 10, a coil for wireless charging by an external charger connected to a commercial power source, and an actuator required for at least one of composition adjustment and focus adjustment.

The signal processing unit 30 has functional components implemented by a CPU (Central Processing Unit) or MPU (Micro Processor Unit) that execute various processes described below, and a memory unit 35 such as a ROM (Read Only Memory) or RAM (Random Access Memory) that stores programs for causing each functional component to execute various processes. Specifically, the signal processing unit 30 has a camera control unit 31, an image processing unit 32, a control unit 33, a lighting control unit 34, and a memory unit 35.

The camera control unit 31 is mounted, for example, on the handle portion 10b of the intraoral camera 10 and controls the imaging unit 21. The camera control unit 31 controls at least one of the aperture and shutter speed of the imaging unit 21 in response to a control signal from the image processing unit 32, for example.

The image processing unit 32 is mounted, for example, on the handle unit 10b of the intraoral camera 10, acquires the captured image captured by the imaging unit 21, performs image processing on the acquired captured image, and outputs the processed captured image to the camera control unit 31 and the control unit 33. The image processing unit 32 may also output the processed captured image to the memory unit 35, and store the processed captured image in the memory unit 35.

The image processing unit 32 is composed of, for example, a circuit, and performs image processing such as noise removal and edge enhancement on the captured image. Note that noise removal and edge enhancement processing may also be performed by the mobile terminal 50.

The captured image output from the image processing unit 32 (the captured image after image processing) may be transmitted to the mobile terminal 50 via the communication unit 40, and an image based on the transmitted captured image may be displayed on the display unit 56 of the mobile terminal 50. This allows an image based on the captured image to be presented to the user.

The control unit 33 is a control device that controls the signal processing unit 30. The control unit 33 controls each component of the signal processing unit 30 based on, for example, the detection results of external light, etc. by the sensor unit 22.

The lighting control unit 34 is mounted, for example, on the handle portion 10b of the intraoral camera 10 and controls the turning on and off of the first to fourth LEDs 23A to 23D. The lighting control unit 34 is composed of, for example, a circuit. For example, when a user operates the display unit 56 of the mobile terminal 50 to start the intraoral camera 10, a corresponding signal is sent from the mobile terminal 50 to the signal processing unit 30 via the communication unit 40. The lighting control unit 34 of the signal processing unit 30 turns on the first to fourth LEDs 23A to 23D based on the received signal. For example, the lighting control unit 34 may cause the lighting unit 23 to emit blue light when the operation unit 24 receives an operation indicating that imaging for periodontal disease detection will be performed, or cause the lighting unit 23 to emit white light when the operation unit 24 receives an operation indicating that imaging for reference color data acquisition will be performed.

In addition to the above programs, the memory unit 35 also stores images captured by the image capture unit 21. The memory unit 35 is realized by, for example, semiconductor memory such as ROM or RAM, but is not limited to this.

The communication unit 40 is a wireless communication module for wireless communication with the mobile terminal 50. The communication unit 40 is mounted, for example, on the handle portion 10b of the intraoral camera 10, and communicates wirelessly with the mobile terminal 50 based on control signals from the signal processing unit 30. The communication unit 40 performs wireless communication with the mobile terminal 50 in accordance with existing communication standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark). Captured images are sent from the intraoral camera 10 to the mobile terminal 50, and operation signals are sent from the mobile terminal 50 to the intraoral camera 10, via the communication unit 40.

The mobile terminal 50 performs processing to obtain images more suitable for detecting periodontal disease and to detect periodontal disease, for example, by irradiating the teeth with light including the wavelength range of blue light and using captured images of the surface of the dentition and the periodontal region that are fluorescently reacting. The mobile terminal 50 functions as a user interface for the periodontal disease detection system. The mobile terminal 50 is an example of a second information terminal.

Figure 3 is a block diagram showing the functional configuration of the mobile terminal 50 according to this embodiment.

As shown in FIG. 3, the mobile terminal 50 includes an acquisition unit 51, a tooth type identification unit 52, an image processing unit 53, an image data extraction unit 54, a periodontal disease detection unit 55, a display unit 56, an output unit 57, and a storage unit 58. The mobile terminal 50 also includes a processor and memory. The memory may be a ROM or RAM, and can store programs executed by the processor. The acquisition unit 51, the tooth type identification unit 52, the image processing unit 53, the image data extraction unit 54, the periodontal disease detection unit 55, the display unit 56, and the output unit 57 are realized by a processor that executes programs stored in the memory. The mobile terminal 50 may be realized, for example, by a smartphone or tablet terminal capable of wireless communication.

The acquisition unit 51 is a wireless communication module for wireless communication with the intraoral camera 10. The acquisition unit 51 acquires captured images from the intraoral camera 10. Specifically, the acquisition unit 51 acquires captured images including one or more teeth and periodontal regions generated by the imaging unit 21. The captured images are images obtained by the intraoral camera 10 photographing teeth that are undergoing a fluorescent reaction by irradiating the teeth with light including the wavelength range of blue light. In this way, the acquisition unit 51 functions as a first acquisition unit that acquires captured images.

The acquisition unit 51 also acquires reference color data that indicates the reference color of the natural teeth included in the imaging area and is user-specific. For example, the acquisition unit 51 may acquire, as reference color data, an image (color image) obtained by illuminating the oral cavity with white light from the intraoral camera 10, or may acquire reference color information for the natural teeth. The color information may be, for example, the chromaticity of the natural teeth in the image obtained by illuminating the oral cavity with white light. The color information for the natural teeth may be color information for any one location on the natural teeth, color information for a specific natural tooth, or a statistical value of color information for multiple locations on the natural teeth (e.g., multiple natural teeth). For example, taking chromaticity as an example, the statistical value of the color information is the average value of the chromaticity, but it may also be the maximum value, minimum value, mode, median, etc. In this way, the acquisition unit 51 functions as a second acquisition unit that acquires reference color data.

Furthermore, the acquisition unit 51 may acquire information (e.g., medical examination necessity information, described below) from a first information terminal carried by a person other than the user (e.g., a medical professional such as a dentist). The first information terminal may be, for example, a portable terminal such as a mobile terminal, or a stationary terminal such as a PC. Furthermore, the first information terminal is an information terminal different from the portable terminal 50.

The acquisition unit 51 may also include a wired communication module that performs wired communication with the intraoral camera 10.

The tooth type identification unit 52 identifies the type of tooth (e.g., a specific tooth) contained in the captured image (i.e., the first image) from the captured image. Identifying the type of tooth may mean identifying whether the tooth is an incisor, canine, or molar, or whether the tooth is a central incisor, lateral incisor, canine, first premolar, second premolar, first molar, second molar, or third molar (wisdom tooth). The tooth type identification unit 52 may also identify the region of the oral cavity (upper jaw, lower jaw, left or right) in which the tooth is located. Note that there are no particular limitations on the method by which the tooth type identification unit 52 identifies the type of tooth, and may, for example, use a machine learning model, a pattern matching method, or any other known method. The machine learning model is a learning model that is trained to output the type of tooth appearing in an image containing teeth when that image is input.

A machine learning model (learning model) is an example of a periodontal disease detection tool. In other words, a periodontal disease detection tool may include multiple learning models, each trained to receive image data containing an area as input and output information related to periodontal disease in that area. A periodontal disease detection tool may be represented by a machine learning model such as a neural network for estimating periodontal disease detection results from input information (here, an image).

In addition, the tooth type identification unit 52 may use a corrected image (described below) instead of a captured image to identify the type of tooth contained in the corrected image.

The image processing unit 53 generates a corrected image in which the gain of at least two of the red, green, and blue color components that make up the natural tooth area in the captured image is adjusted based on the reference color data. In this embodiment, since the captured image is a bluish image, the image processing unit 53 performs processing to correct the color of the natural teeth in the captured image to the color of the actual user's natural teeth based on the reference color data of the user's natural teeth. In this way, the reference color data is set based on, for example, the color of the user's natural teeth.

The image processing unit 53 does not simply perform a process to make the teeth achromatic (so-called white balance processing), but rather uses reference color data stored in the memory unit 58 to determine the amount of gain adjustment required to correct the color of the natural teeth in the captured image to the reference color data, and uniformly corrects the entire captured image (i.e., the entire image including the teeth and periodontal region) using the determined gain adjustment amount. "Uniformly" here means that the teeth and periodontal region are corrected using the same gain adjustment amount. The corrected image is an example of the second RGB image or second image.

In the corrected image generated in this way, the color of the natural teeth matches the color of the actual user's natural teeth. Furthermore, because the teeth and periodontal region are corrected with the same gain, the color of the periodontal region is closer to the color of the actual user's periodontal region than when white balance processing is performed.

The image processing unit 53 may generate a corrected image from the captured image by adjusting the gain of at least two color components, for example, a first red pixel average value of multiple red pixel values of multiple pixels (first pixels) constituting the natural tooth region in the captured image, a first green pixel average value of multiple green pixel values of multiple pixels (first pixels), and a first blue pixel average value of multiple blue pixel values of multiple pixels (first pixels), so that the difference from the reference color data falls within a predetermined range. Note that the image processing unit 53 is not limited to using the first red pixel average value, the first blue pixel average value, and the first green pixel average value, and may generate a corrected image from the captured image using a statistical value of the first red pixels, a statistical value of the first blue pixels, and a statistical value of the first green pixels. Statistical values include, but are not limited to, maximum values, minimum values, modes, medians, etc.

It is known that when natural teeth are irradiated with excitation light, excitation fluorescence is emitted from the dentin, which passes through the enamel (see Figure 6 described below) and fluoresces green. It is also known that fillings in caries treatment marks (e.g., metal inlays) do not emit excitation fluorescence under blue LED light, and are captured as dark (low brightness) on the camera. For these reasons, the image processing unit 53 can detect the natural teeth excluding the caries treatment marks from the captured image. The caries treatment mark area is an example of a specific pixel area.

The image processing unit 53 may also detect plaque regions on the teeth from the corrected image. Because teeth and plaque emit different fluorescence, the image processing unit 53 can detect teeth and plaque from the color of the fluorescent (excited fluorescence) parts in the corrected image. Plaque (plaque region) fluoresces a reddish pink color (excited fluorescence) when illuminated with blue light. Note that any other known method for detecting tooth regions and plaque regions may be used. Plaque regions are an example of specific pixel regions.

The image data extraction unit 54 extracts periodontal region image data from the corrected image, including the gingival region near the boundary between the tooth and the gingiva. It can also be said that the image data extraction unit 54 extracts periodontal region image data from the corrected image, including the free gingiva, which is the gingiva that surrounds the cervical region, including the interdental papilla and marginal gingiva, in a band shape, and a portion of the attached gingiva that is continuous with the free gingiva and extends from the bottom of the gingival sulcus to the gingival-alveolar junction. For example, the periodontal region image data may include the interdental papilla and gingiva on both sides of the tooth.

Details will be described later using Figures 7A and 7B, but the periodontal region image data is an image extracted from the corrected image that includes the periodontal region where periodontal disease develops, but that minimizes the inclusion of tooth regions that are unnecessary for detecting periodontal disease. The image data extraction unit 54 may extract the periodontal region image data by removing part of the tooth region from the corrected image. The periodontal region image data is an example of image data based on the corrected image.

The periodontal disease detection unit 55 detects the user's periodontal disease based on image data derived from the corrected image. The periodontal disease here may be the current progress of periodontal disease, or a sign of periodontal disease. A sign includes the presence of a precursory phenomenon that is not currently recognized as periodontal disease, but which makes it highly likely that periodontal disease will occur. Examples of precursory phenomena include, but are not limited to, the occurrence of gingivitis.

In this embodiment, the periodontal disease detection unit 55 detects the user's periodontal disease by inputting image data based on the corrected image (in this embodiment, periodontal region image data) into a learning model that has been trained to input image data including the gingival region and output information related to periodontal disease in the image data. The information related to periodontal disease includes the presence or absence of periodontal disease, the progress of periodontal disease, signs of periodontal disease, etc. The information related to periodontal disease may also include an estimated result of at least one of the periodontal pocket depth, BOP, and GI value.

The learning model is trained in advance through supervised learning using a learning dataset in which image data including the gingival region (in this embodiment, periodontal region image data) is used as input data and information regarding periodontal disease is used as correct answer data. The learning model may be, for example, a model that can determine the occurrence of each of a plurality of diseases, including cases in which multiple diseases have occurred, obtained in advance by training using training data including a plurality of sample data having features for identifying the presence or absence of diseases related to the periodontal region of a plurality of teeth.

The image data used for learning is a plurality of images showing different stages of periodontal disease progression, such as image data of a normal periodontal region, image data of a periodontal region suspected of having periodontal disease, and image data of periodontal regions with mild, moderate, or severe periodontitis. Furthermore, images for learning may be prepared for each type of tooth, region in which the tooth is located, and shooting direction (such as whether the image is taken from the lingual side or the buccal side). The image data may be an image including the periodontal region extracted by the image data extraction unit 54 (i.e., an edited image), or it may be the corrected image itself.

The processing unit that generates the learning model may be provided by the periodontal disease detection system, or by a device outside the periodontal disease detection system.

The display unit 56 is a display device provided in the mobile terminal 50, and displays information indicating the periodontal disease detected by the periodontal disease detection unit 55. The display unit 56 is realized, for example, by a liquid crystal display panel, but is not limited to this.

The output unit 57 is a wireless communication module for wireless communication with a first information terminal of a person other than the user (for example, a medical professional such as a dentist). The output unit 57 outputs information indicating periodontal disease detected by the periodontal disease detection unit 55 to the first information terminal of the person other than the user. The output unit 57 also outputs to the user's second information terminal the medical examination necessity information acquired by the acquisition unit 51, which information is regarding the need for the user to receive medical examination and entered by the person into the first information terminal. The second information terminal may be, for example, an information terminal different from the mobile terminal 50. The dentist may be, for example, the user's doctor.

The output unit 57 may associate information indicating periodontal disease detected by the periodontal disease detection unit 55 with the type of tooth identified by the tooth type identification unit 52 and output the associated information to at least one of the first information terminal and the second information terminal. In other words, the output unit 57 may display information indicating periodontal disease associated with the type of tooth on at least one of the first information terminal and the second information terminal. This makes it possible to notify at least one of the user and medical professionals of the type of periodontal disease (or whether the periodontal region of each tooth has periodontal disease) (or whether it has no periodontal disease).

The output unit 57 may also include a wired communication module that performs wired communication with the first information terminal and the second information terminal.

The memory unit 58 is a storage device that stores various information for detecting periodontal disease in a user. The memory unit 58 may store, for example, a learning model used by the periodontal disease detection unit 55, or may store reference color data. The memory unit 58 may be realized, for example, by a semiconductor memory or the like, but is not limited to this.

It is sufficient that the mobile terminal 50 has a configuration for generating an image that is more suitable for detecting periodontal disease. For example, the mobile terminal 50 does not have to have a configuration for detecting periodontal disease, such as the periodontal disease detection unit 55. In this case, the dentist makes a diagnosis while checking an image (e.g., a corrected image) that is more suitable for detecting periodontal disease generated by the mobile terminal 50. In other words, the mobile terminal 50 may function as a support device that generates images that allow the dentist to make a more accurate diagnosis.

[1-2. Operation of Periodontal Disease Detection System]
Next, the operation of the periodontal disease detection system configured as described above will be described with reference to Figures 4 to 7B. Figure 4 is a flowchart showing the operation of the periodontal disease detection system (periodontal disease detection method) according to this embodiment.

As shown in FIG. 4, first, the acquisition unit 51 acquires reference color data of the user's natural teeth and stores it in the storage unit 58 (S11). The acquisition unit 51 may acquire the reference color data before acquiring the captured image. Note that the reference color data only needs to be acquired before executing step S14, and is not limited to being acquired before acquiring the captured image.

Furthermore, since the reference color data differs for each user, the acquisition unit 51 may store the reference color data in the storage unit 58 in association with information indicating the user. Furthermore, the processing of step S11 may be performed, for example, once for each user. For example, if reference color data is already stored for the user, step S11 may be omitted.

Next, the acquisition unit 51 acquires a captured image of the imaging area inside the user's oral cavity (S12). The captured image may be an image taken by the user themselves, or an image of the inside of the user's oral cavity taken by a medical professional. The acquisition unit 51 may store the captured image in the storage unit 58.

Next, the tooth type identification unit 52 identifies the tooth area in the captured image (S13). For example, the tooth type identification unit 52 may use the tooth area that is the output of a trained machine learning model obtained by inputting the captured image into the machine learning model to identify in which region of the oral cavity the tooth in the captured image is located.

Next, the image processing unit 53 generates a corrected image in which the color of the teeth in the captured image has been corrected based on the reference color data (S14). The image processing unit 53 determines the amount of gain adjustment so that the color data of the teeth in the captured image matches the reference color data or falls within a predetermined range, and uniformly corrects the entire captured image (i.e., the entire image including the teeth and periodontal region) using the determined amount of gain adjustment.

Here, the image processing unit 53 may perform the following processing to generate a corrected image. For example, the image processing unit 53 may identify a specific pixel area including pixels whose values based on the pixel values of the first corrected image obtained by correcting the color of the tooth in the captured image are within a predetermined range, and may generate a third corrected image as the above corrected image by adjusting the gain of at least two of the red, green, and blue color components that make up the area of the natural tooth in the captured image excluding the specific pixel area based on the reference color data. Image data based on the third corrected image is an example of image data based on a corrected image. The specific pixel area is an area of the natural tooth where the natural tooth is not exposed, for example, an area where plaque is attached.

Furthermore, for example, the image processing unit 53 may generate an HSV image by converting the color space of a first corrected image obtained by correcting the color of the teeth in the captured image into an HSV space, and identify as the specific pixel area a pixel area in which one or more pixels of the HSV image that satisfy at least one of the following conditions are located: saturation within a first predetermined range, hue within a second predetermined range, and brightness within a third predetermined range.

Furthermore, for example, the image processing unit 53 may identify as the specific pixel area a pixel area in which one or more pixels are located, where the multiple red pixel values of the multiple images constituting the natural tooth area in the first corrected image satisfy at least one of the following conditions: the multiple green pixel values are within a first range, the multiple blue pixel values are within a second range, and the multiple blue pixel values are within a third range.

In this way, the image processing unit 53 may identify the specific pixel area from the HSV image converted from the first corrected image, or may identify the specific pixel area from the first corrected image itself.

Next, the image data extraction unit 54 extracts the gingival region based on the corrected image (S15). Note that the image data extraction unit 54 may, for example, generate an HSV image by converting the color space of the first image (RGB image) into HSV space, and detect the region whose hue (H) falls within a predetermined range as the gingival region.

Here, the extracted image of the gingival region (periodontal region image) will be further described with reference to Figures 5 to 7B. First, the periodontal region will be described with reference to Figures 5 and 6. Figure 5 is a first diagram for explaining the periodontal region within the oral cavity in this embodiment. Figure 5 is an image captured of the oral cavity including the dental region and the periodontal region. Figure 6 is a second diagram for explaining the periodontal region within the oral cavity in this embodiment. Figure 6 shows a cross-sectional view of the periodontal region. Note that although Figure 6 is a cross-sectional view, hatching has been omitted for convenience.

As shown in Figures 5(a) and (b), the gingiva comprises the free gingiva, attached gingiva, and alveolar mucosa. The free gingiva is the part of the gingiva that is closest to the head of the tooth and moves when the cheek or other parts of the body move. The attached gingiva is the part of the gingiva that is attached to the bone and does not move when the cheek or other parts of the body move. The alveolar mucosa is the part of the attached gingiva that is located on the opposite side of the free gingiva from the free gingiva and moves when the cheek or other parts of the body move. The interdental papilla gingiva is the thin gingiva found between the teeth; it is weak and prone to inflammation and swelling, and is a place where dirt and plaque tend to accumulate. Periodontal disease is one of the causes of loss of the interdental papilla gingiva.

As shown in Figure 6, the free gingiva is the part of the gingiva that forms the gingival sulcus. The gingival sulcus is a groove between the tooth and gingiva, and is a place where plaque that leads to periodontal disease tends to accumulate. The gingival margin is the top of the free gingiva and is located opposite the attached gingiva. Furthermore, the free gingival sulcus is located at the border between the free gingiva and attached gingiva, and the gingival-alveolar junction is located at the border between the attached gingiva and alveolar mucosa. The attached gingiva is the part of the gingiva from the free gingival sulcus to the gingival-alveolar junction.

FIG. 7A is a diagram illustrating image data extracted by the image data extraction unit 54 according to this embodiment. (a) of FIG. 7A is a diagram illustrating the area extracted from the corrected image by the image data extraction unit 54, and (b) of FIG. 7A is a diagram illustrating the extracted image data. The image shown in (a) of FIG. 7A is an example of a corrected image, and (b) of FIG. 7A is an image obtained by extracting the rectangular area surrounded by the dashed line in (a) of FIG. 7A (periodontal region image data extracted by the image data extraction unit 54).

As shown in (a) of Figure 7A, the image data extraction unit 54 draws dashed-dotted lines (dotted-dotted lines extending vertically on the paper) that circumscribe the left and right sides of the tooth contour and are perpendicular to the tooth alignment direction. The left and right sides are the parts of the tooth that protrude most toward the adjacent tooth. The image data extraction unit 54 also draws dashed-dotted lines (the upper dashed-dotted lines extending horizontally on the paper) parallel to the tooth alignment direction so as to include the apex (tip) of the interdental papilla gingiva, and draws dashed-dotted lines (the lower dashed-dotted lines extending horizontally on the paper) parallel to the tooth alignment direction so as to include the peripheral tooth edge.

Then, as shown in (b) of Figure 7A, the image data extraction unit 54 generates periodontal region image data by extracting the rectangular region enclosed by the dashed line shown in (a) of Figure 7A from the corrected image. In this way, the image data extraction unit 54 extracts from the corrected image, as periodontal region image data, a rectangular region of the periodontal region of the tooth in question that includes from the tip of the interdental papilla gingiva (e.g., the periodontal margin) to the free gingival sulcus. In this case, the periodontal region image data is an image that includes only a portion of the tooth. The tooth region is an area that is unnecessary for detecting periodontal disease (e.g., an area that may reduce detection accuracy), and as shown in (b) of Figure 7A, it is desirable that the periodontal region image data include as little tooth region as possible.

When the periodontal region image data includes the area from the periodontal margin to the free gingival sulcus, it is possible to estimate the depth of the periodontal pocket based on that image data, and when the interdental papilla gingiva becomes diseased, the interdental papilla gingiva recedes, appearing to create gaps between the teeth, making it possible to determine the progression of the periodontal disease.

Note that the image data extraction unit 54 is not limited to extracting the region shown in (a) of Figure 7A, and may, for example, extract from the corrected image as periodontal region image data a rectangular region that is long in the vertical direction and is surrounded by a dashed line extending in the vertical direction (for example, a region that includes the periodontal region as well as most of the tooth). Note that, from the perspective of detecting periodontal disease with greater accuracy, it is preferable to extract the region surrounded by the dashed line shown in (a) of Figure 7A. Note also that the extracted image data is not limited to being a rectangular image. Note that the image data extraction unit 54 may also extract from the captured image as periodontal region image data a region in the captured image that corresponds to the dashed line shown in (a) of Figure 7A.

Here, a method for forming a rectangular area will be described with reference to Figure 7B. Figure 7B is a diagram showing a specific example of a method for forming a rectangular area according to this embodiment. The frame that shows a rectangular area will also be referred to as a rectangular frame. Figure 7B shows an example of a rectangular frame for a maxillary central incisor.

As shown in (a) of Figure 7B, the image data extraction unit 54 forms a rectangular frame F1 that surrounds a specific tooth in accordance with the dentition. The image data extraction unit 54 identifies the tooth based on, for example, the color of the tooth and gums, and forms a rectangular frame F1 that surrounds one tooth based on the shape of the tooth. The four sides of the rectangular frame F1 are formed so as to be in contact with the outer shape of one tooth.

As shown in (b) of Figure 7B, the image data extraction unit 54 then forms a rectangular frame F2 by moving the bottom edge of the rectangular frame F1 upward within a range that includes the left and right interdental papillae and gingiva (for example, the tips of the interdental papillae and gingiva). At this time, it is advisable to move the frame upward as much as possible within a range that includes the left and right interdental papillae and gingiva in order to remove as much tooth information as possible.

As shown in (c) of Figure 7B, the image data extraction unit 54 then forms a rectangular frame F3 by moving the upper end of the rectangular frame F2 upward from the gingival margin by a predetermined distance D. The predetermined distance D is, for example, 2 mm, but is not limited to this, and may be 1 mm or 3 mm or more. This forms the dashed-dotted frame shown in (b) of Figure 7A.

In feature engineering, it is more important to find meaningful "features" that are closely related to the target variable (disease progression) than to find complex mathematical or statistical transformations to improve prediction accuracy. However, finding such features requires knowledge, experience, and intuition related to dentistry, knowledge of data (the meaning of data items and relationships between tables), and knowledge of statistics and machine learning (statistical stability and predictive power). Feature engineering is one of the most important and most difficult steps in the process of developing a machine learning model.

Through extensive research, the inventors discovered that, as described above, by reducing the area in which teeth are visible (reducing the influence of teeth on the detection of periodontal disease), they can generate training data that makes it easier to obtain gingival features.

Referring again to FIG. 4, the periodontal disease detection unit 55 performs detection related to periodontal disease based on the image of the gingival region (S16). The periodontal disease detection unit 55 inputs the periodontal region image data extracted by the image data extraction unit 54 (for example, the image shown in (b) of FIG. 7A) into the learning model, and obtains an estimated result of periodontal disease, which is the output of the learning model. The periodontal disease detection unit 55 obtains, for example, the estimated result as the detection result of periodontal disease.

Next, the output unit 57 outputs the detection results of the periodontal disease detection unit 55 (S17). The output unit 57 transmits the detection results via communication to the information terminal of at least one of the user and medical professional. This allows the periodontal disease detection results to be notified to at least one of the user and medical professional.

As described above, the periodontal disease detection system according to this embodiment corrects color casts on specific teeth and gums caused by the light irradiating the oral cavity when capturing an image based on pre-stored reference color data for specific teeth that is specific to the user, without using color calibration color patches or the like. This allows the colors of the teeth and gums in the image to be closer to the colors of the teeth and gums of the user. By detecting periodontal disease using such an image (corrected image), it is possible to prevent a decrease in the accuracy of periodontal disease detection due to the image.

(First Modification of First Embodiment)
The periodontal disease detection system according to this modification will be described below with reference to Figures 8A to 8C. The following description will focus on the differences from embodiment 1, and descriptions of content that is the same as or similar to embodiment 1 will be omitted or simplified. In this modification, an example will be described in which the periodontal disease detection unit detects periodontal disease using a rule base.

For example, in this modified example, when assessing periodontal disease risk, etc., a method is adopted in which risk is assessed using a judgment rule for content for which a person can construct a judgment rule (for example, a judgment rule based on the shape of the tip of the interdental papilla gingiva). The judgment rule is an example of a periodontal disease detection tool. The judgment rule is generated in order to output information related to periodontal disease from image data.

The configuration of the periodontal disease detection system of this modified example may be the same as that of the periodontal disease detection system 1 of embodiment 1, and will be described below using the reference numerals of the periodontal disease detection system 1. In step S16 shown in Figure 4, the periodontal disease detection unit 55 of this modified example automatically detects the user's periodontal disease using preset determination rules.

In all R, G, and B values of the RGB images of the gums, there is a large variation in the color distribution of the free and attached gums across users, for both normal and diseased gums, and there is overlap in the color distribution of the normal gums and gums suspected of having periodontal disease. Therefore, if a judgment is made without taking into account the R, G, and B values of the user's normal gums, light gum color may be gum suspected of having periodontal disease, and dark gum color may be normal gum.

Therefore, the judgment rules for this modified example set multiple judgment rules based on the ranges of R, G, and B values of the normal gingival group. A specific setting method is described below. Figures 8A to 8C are diagrams showing an example of each group rule base (judgment rule) for this modified example. Figure 8A shows the judgment rule when the normal gingival R value is A1 to A2 for free gingiva and B1 to B2 for attached gingiva. Figure 8B shows the judgment rule when the normal gingiva R value is A3 to A4 for free gingiva and B3 to B4 for attached gingiva. Figure 8C shows the judgment rule when the normal gingiva R value is A5 to A6 for free gingiva and B5 to B6 for attached gingiva.

As shown in Figures 8A to 8C, for example, the nth group rule base stores the periodontal disease determination criteria when the normal range for free gingiva is A2n-1≦R value≦A2n, and when the normal range for attached gingiva is B2n-1≦R value≦B2n. Here, the rule base groups are classified only by R value, but the determination rules may also be classified based on, for example, G value, B value, or a combination of two of R value, G value, and B value, or a combination of three of R value, G value, and B value.

For example, the rule base (each judgment rule) for each group may set a rule for determining the progression level of periodontal disease based on the amount of change in the R value, G value, and B value from normal gingiva obtained from the user. The rule base (each judgment rule) for each group may divide gingiva into free gingiva and attached gingiva, and set the progression level of periodontal disease based on the amount of change in each of the R value, G value, and B value from normal gingiva values. In other words, the criteria for determining periodontal disease may be set for each user. The progression level is an example of the degree of progression.

The progression level may be, for example, the stage of progression of periodontal disease. For example, progression level 1 may be gingivitis, progression level 2 may be mild periodontal disease, progression level 3 may be moderate periodontal disease, and progression level 4 may be severe periodontal disease.

In general, as the progression level of periodontal disease progresses from normal gingiva to attached gingiva, all brightness indices (R, G, B) decrease, and the amount of change tends to be greater for free gingiva compared to attached gingiva. Based on this knowledge, the periodontal disease detection unit 55 may output the progression level of periodontal disease by prioritizing an evaluation based on free gingiva. Alternatively, the periodontal disease detection unit 55 may output the progression level of periodontal disease for both free gingiva and attached gingiva.

The judgment rules shown in Figures 8A to 8C are set in advance and stored in the memory unit 58. There is no particular limit to the number of judgment rules stored in the memory unit 58, as long as it is two or more. The judgment rules may also be rules for detecting the presence or absence of periodontal disease.

Note that each judgment rule may be set for each color space, or may be set individually for each region in the oral cavity, or, if color information of the user's healthy gums (e.g., absolute color values and color variation) has been acquired in advance, may be set according to that color information. In other words, judgment rules may be set for each user. For example, color information (absolute color values, e.g., R value, G value, B value) at multiple points on the user's gums may be acquired, the variation in R value may be calculated, and the normal range of the R value may be determined based on the statistical value of the R value (e.g., average value) and the variation in the R value. The G value and B value may also be determined in the same way as the R value.

In step S16 shown in Figure 4, the periodontal disease detection unit 55 detects periodontal disease using the above-mentioned judgment rules. The periodontal disease detection unit 55 is configured to compare the input image data with the judgment rules and detect periodontal disease depending on the comparison results. Processing the input image data using the judgment rules is also referred to as inputting the image data into the judgment rules.

For example, the periodontal disease detection unit 55 detects the user's periodontal disease based on the R, G, and B values of the gum color obtained from image data based on the second RGB image and the judgment rule. The periodontal disease detection unit 55 extracts a judgment rule from multiple judgment rules that corresponds to the region to which the target tooth belongs or the user, and uses the extracted judgment rule to judge periodontal disease.

The periodontal disease detection unit 55, for example, extracts color information of the gums from image data based on the second RGB image, and determines whether the gums are normal or not based on the difference (amount of change) between the color value indicated by the extracted color information and a preset normal range, and if not normal, determines the progression level of the periodontal disease for both the free gums and the attached gums. The color values may be R values, G values, and B values, or may be hue information, saturation information, lightness information, or hue information, saturation information, and brightness information.

For example, the periodontal disease detection unit 55 may use a determination rule created from the range of R, G, and B values in the RGB color space for normal gum color and the range of R, G, and B values for gum color for multiple stages of periodontal disease to input the R, G, and B values of the gum color obtained from image data based on the second RGB image and detect the user's periodontal disease in that area.

Furthermore, for example, the periodontal disease detection unit 55 may input at least one of the hue information, saturation information, and brightness information of the color of normal gums in the HSV color space, and at least one of the ranges of the hue information, saturation information, and brightness information of the color of gums for each of multiple stages of periodontal disease, and detect the user's periodontal disease in the relevant area.

Furthermore, for example, the periodontal disease detection unit 55 may input at least one of the hue information, saturation information, and luminance information of the color of normal gums in the HSL color space, and at least one of the ranges of the hue information, saturation information, and luminance information of the color of gums for each of multiple stages of periodontal disease, and detect the user's periodontal disease in that area.

Note that the difference in gum color is the difference in color between the darkest and lightest parts of the gums, and the determination rule may be a rule that associates this difference with normal and advanced levels. Since each user's gum color (e.g., normal gum color) is different, by using the difference in color between the darkest and lightest parts of the gums, periodontal disease can be detected without relying on each user's own gum color.

Note that, while an example has been described in which color is used as an evaluation parameter when using a judgment rule, gum shape may be used instead of or in addition to color. An example in which gum shape is used will be described in a modified version of embodiment 2.

(Modification 2 of Embodiment 1)
The periodontal disease detection system according to this modification will be described below with reference to Fig. 9. The following description will focus on differences from embodiment 1, and descriptions of content that is the same as or similar to embodiment 1 will be omitted or simplified. In this modification, training data for training a machine learning model will be described.

Note that the configuration of the periodontal disease detection system of this modified example may be the same as that of the periodontal disease detection system 1 according to embodiment 1, and the following description will use the reference numerals of the periodontal disease detection system 1.

Figure 9 is a diagram explaining the learning data for this modified example. In Figure 9, numbers are assigned to each type of tooth in the upper and lower jaws. For example, "1" is a central incisor and "2" is a lateral incisor.

As shown in Figure 9, multiple points are set for each tooth to obtain correct answer data for the training data. In the example of Figure 9, three points (1 (left), 2 (center), 3 (right)) are set on the outside of the maxillary central incisors, and three points (4 to 6) are set on the buccal side of the maxillary central incisors. In the example of Figure 9, three points (11 to 13) are set on the outside of the mandibular central incisors, and three points (14 to 16) are set on the buccal side of the mandibular central incisors. The measurement points are set in the gingival area at the boundary between the tooth and the gingiva.

Measurements include at least one of periodontal pockets and BOP values, and the measurement items in this modified example include both periodontal pockets and BOP values. In other words, in this modified example, periodontal pockets and BOP values are measured at six locations for each tooth (for example, locations 1 to 6 for the maxillary central incisors). Also, in this modified example, periodontal pockets and BOP values are measured at three locations for each tooth (for example, locations 1 to 3 or 4 to 6 for the maxillary central incisors).

The periodontal pocket values may be set into nine classes, one for each of 1 to 9 mm, or two classes, 3 mm or less and 4 mm or more, or three classes, 2 mm or less, 3 to 5 mm, and 6 mm or more, or five classes, 2 mm or less, 3 mm, 4 mm, 5 mm, and 6 mm or more.

BOP scores are assigned a value of 0 or 1 depending on whether or not there is bleeding.

Here, the following additional data is added to the image as training data for the machine learning model.

It includes (1) information indicating whether the image was taken of the upper or lower jaw, (2) information indicating whether the image was taken of the right or left side, with the front teeth (central incisors) at the center, (3) information indicating the tooth number (1-8), (4) information indicating whether the image is of the lingual side or the buccal side, (5) information indicating the periodontal pocket value, and (6) information indicating the BOP value. (5) Information indicating the periodontal pocket value and (6) information indicating the BOP value include the measurement results for each measurement point. For example, the following additional information is added: "upper jaw, left, number 5, lingual side, pocket values 3, 2, 5, BOP values 0, 0, 1."

(5) Periodontal pocket value and (6) BOP value are used as correct labels during learning. Note that tooth numbers are set to exclude primary teeth, but this is not a limitation.

By providing a set containing a large number of periodontal pocket depth measurements (correct labels) in each periodontal region as training data and performing machine learning using a specified algorithm, the accuracy of the generated machine learning model can be improved. Furthermore, based on the estimated periodontal pocket depth and the BOP value, the machine learning model can determine at least one of the degree of progression of periodontal disease and the need for periodontal surgical treatment.

(Embodiment 2)
Hereinafter, the periodontal disease detection system according to this embodiment will be described with reference to FIGS.

[2-1. Configuration of periodontal disease detection system]
The periodontal disease detection system according to this embodiment is an information processing system for detecting periodontal disease in a user based on an image of a predetermined area in the oral cavity, and differs from the periodontal disease detection system according to embodiment 1 in that it includes a mobile terminal 50a instead of the mobile terminal 50. The following description will focus on differences from embodiment 1, such as the configuration of the mobile terminal 50a, and descriptions of identical or similar configurations will be omitted or simplified.

Figure 10 is a block diagram showing the functional configuration of the mobile terminal 50a according to this embodiment.

As shown in FIG. 10, the mobile terminal 50a does not include the tooth type identification unit 52 of the mobile terminal 50 according to embodiment 1, but does include an area detection unit 151 and a rule selection unit 152.

The image processing unit 53 generates a corrected image in which the colors have been corrected from the captured image. The image processing unit 53 may generate the corrected image from the captured image using reference color data as in embodiment 1, or may generate the corrected image from the captured image by performing white balance processing.

The area detection unit 151 detects which of the multiple areas of the oral cavity defined by dividing the tooth row the tooth in the corrected image generated by the image processing unit 53 belongs to. The multiple areas are areas obtained by dividing the oral cavity into groups of teeth with similar characteristics. Similar characteristics include similar tooth shapes. Note that if the shape of the teeth changes, the shape of the gums may also change similarly. Therefore, in this case, similar tooth shapes mean similar gum shapes.

FIG. 11 is a diagram illustrating multiple regions within the oral cavity according to this embodiment. (a) of FIG. 11 is a diagram in which regions with similar shapes of teeth and gums are separated by dashed lines. (b) of FIG. 11 is a diagram showing an example of multiple regions set in this embodiment.

In Figure 11 (a), six regions are shown, three on each of the upper and lower jaws. Specifically, the upper jaw includes three regions: the right upper jaw, which includes the right molars; the front upper jaw, which includes the central incisors, lateral incisors, and canines; and the left upper jaw, which includes the left molars. The lower jaw includes three regions: the right lower jaw, which includes the right molars; the front lower jaw, which includes the central incisors, lateral incisors, and canines; and the left lower jaw, which includes the left molars.

As shown in (b) of Figure 11, the multiple regions include a first region R1 to a fourth region R4. Specifically, the first region R1 indicates the tongue-side region of the molars of the upper and lower jaws, the second region R2 indicates the buccal-side region of the molars of the upper and lower jaws, the third region R3 indicates the tongue-side region of the anterior teeth of the upper and lower jaws, and the fourth region R4 indicates the buccal-side region of the anterior teeth of the upper and lower jaws.

Note that the number of regions may be two or more, and may be two or three, or five or more.

FIG. 12 shows an example of an image obtained by photographing multiple regions in this embodiment. (a) of FIG. 12 shows an image of a mandibular anterior tooth photographed from the lingual side. The image shown in (a) of FIG. 12 is an example of an image photographed in the third region R3 shown in (b) of FIG. 11. (b) of FIG. 12 shows an image of a mandibular anterior tooth photographed from the buccal side. The image shown in (b) of FIG. 12 is an example of an image photographed in the fourth region R4 shown in (b) of FIG. 11. (c) of FIG. 12 shows an image of a mandibular molar photographed from the lingual side. The image shown in (c) of FIG. 12 is an example of an image photographed in the first region R1 shown in (b) of FIG. 11. (d) of FIG. 12 shows an image of a mandibular molar photographed from the buccal side. The image shown in (d) of FIG. 12 is an example of an image photographed in the second region R2 shown in (b) of FIG. 11.

In this embodiment, the region detection unit 151 detects which of the first region R1 to fourth region R4 the corrected image is an image of. It can also be said that the region detection unit 151 detects from which of the first region R1 to fourth region R4 the teeth and periodontal region shown in the corrected image were photographed.

The method by which the region detection unit 151 detects regions is not particularly limited, and may be, for example, a method using a machine learning model, a method using pattern matching, or any other known method. The machine learning model is a learning model that is trained to output, when an image including teeth and periodontal regions is input, which of the first region R1 to fourth region R4 the teeth and periodontal region shown in the image belongs to.

As shown in Figures 12(a) to 12(d), the shapes of teeth and gums vary depending on at least one of the tooth type (molar, canine, anterior tooth) and imaging direction (cheek side, lingual side). It is also believed that the type of periodontal disease and the degree of gum deformation associated with its progression also vary depending on at least one of the tooth type and imaging direction. Therefore, if all teeth and gums are trained using a single learning model, there is a risk that the unique shapes of each tooth and gum may result in incorrect detection of the degree of progression of periodontal disease.

In this embodiment, the memory unit 58 pre-stores, for each of the multiple regions, multiple learning models that have been trained using images corresponding to that region. The multiple learning models correspond to each of the multiple regions, and each is a learning model that has been trained to input image data of the teeth and gums in that region and output information related to periodontal disease in that region. In other words, for each of the multiple regions, a learning model that corresponds one-to-one to that region is generated. Information related to periodontal disease includes the presence or absence of periodontal disease, the progress of periodontal disease, signs of periodontal disease, etc. The information related to periodontal disease may also include estimated results for at least one of periodontal pocket depth, BOP, and GI value.

Each of the multiple learning models is associated with information indicating which of the multiple regions the learning model corresponds to. In this embodiment, each of the multiple learning models is associated with information indicating one of the first region R1 to fourth region R4.

In this embodiment, there are learning models corresponding to the corrected image based on the captured image of the first region R1, the corrected image based on the captured image of the second region R2, the corrected image based on the captured image of the third region R3, and the corrected image based on the captured image of the fourth region R4. In other words, in this embodiment, four learning models corresponding one-to-one to each of the four regions are created in advance and stored in the memory unit 58.

The rule selection unit 152 selects a learning model that corresponds to the area detected by the area detection unit 151 from among multiple learning models stored in the memory unit 58. The rule selection unit 152 is an example of a tool selection unit.

The periodontal disease detection unit 55 detects the user's periodontal disease using the learning model selected by the rule selection unit 152. The periodontal disease detection unit 55 detects the user's periodontal disease by inputting image data based on the corrected image (e.g., periodontal region image data) into the selected learning model.

Here, images used for training multiple learning models will be described with reference to Figure 13. Figure 13 is a diagram showing an example of an image used for training a learning model according to this embodiment. In each image shown in Figure 13, the area enclosed by the dashed-dotted rectangular frame may be extracted by the image data extraction unit 54 or the like, and the extracted image data may be used as image data for training. Note that the dashed-dotted rectangular frame may be formed by the method shown in Figure 7A (a).

The image shown in Figure 13 (a) is an image of a lower jaw front tooth photographed from the lingual side, and is used for training a learning model to which an image of the third region R3 is input. The image shown in Figure 13 (b) is an image of a lower jaw front tooth photographed from the buccal side, and is used for training a learning model to which an image of the fourth region R4 is input. The image shown in Figure 13 (c) is an image of a lower jaw back tooth photographed from the lingual side, and is used for training a learning model to which an image of the first region R1 is input. The image shown in Figure 13 (d) is an image of a lower jaw back tooth photographed from the buccal side, and is used for training a learning model to which an image of the second region R2 is input.

In this way, learning is performed by generating a learning image (periodontal region image) for each group of teeth belonging to a region and inputting the periodontal region image corresponding to that region into the learning model corresponding to that region. Furthermore, the images shown in (a), (b), and (d) of FIG. 13 are not used in the learning model into which an image of the first region R1 is input; the images shown in (a) to (c) of FIG. 13 are not used in the learning model into which an image of the second region R2 is input; the images shown in (b) to (d) of FIG. 13 are not used in the learning model into which an image of the third region R3 is input; and the images shown in (a), (c), and (d) of FIG. 13 are not used in the learning model into which an image of the fourth region R4 is input.

Each learning model is trained using multiple images showing different stages of periodontal disease progression. The training method for each learning model is the same as in embodiment 1, so further explanation will be omitted.

The processing unit that generates the learning model may be provided by the periodontal disease detection system, or by a device outside the periodontal disease detection system.

[2-2. Operation of Periodontal Disease Detection System]
Next, the operation of the periodontal disease detection system configured as above will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the operation of the periodontal disease detection system (periodontal disease detection method) according to this embodiment.

As shown in FIG. 14, first, the acquisition unit 51 acquires a captured image of the imaging area inside the user's oral cavity (S101). Step S101 corresponds to step S12 shown in FIG. 4.

Next, the image processing unit 53 and image data extraction unit 54 generate an image of the periodontal region of a specific tooth from the captured image (S102). Step S102 corresponds to steps S14 and S15 shown in Figure 4.

Next, the region detection unit 151 detects an intraoral region from the image of the periodontal region or the corrected image based on the type of tooth shown in the image and the imaging direction (S103). The region detection unit 151 detects which of the first region R1 to fourth region R4 the image corresponds to by using a machine learning model, pattern matching, or the like.

Next, the rule selection unit 152 selects a learning model that performs periodontal disease detection from among the multiple learning models stored in the memory unit 58, based on the intraoral region detected by the region detection unit 151 (S104). The rule selection unit 152 selects, from the multiple learning models, the learning model that corresponds to the region detected by the region detection unit 151 as the learning model that performs periodontal disease detection in the image.

In this way, there are learning models corresponding to each of the multiple regions within the oral cavity, and the learning model corresponding to the region detected by the region detection unit 151 is selected by the rule selection unit 152.

Next, the periodontal disease detection unit 55 performs periodontal disease detection based on the image of the gingival region and the selected learning model (S105). The periodontal disease detection unit 55 inputs the image of the gingival region into the selected learning model, and obtains the detection results for periodontal disease as the output of the learning model. Step S105 corresponds to step S16 shown in Figure 4.

Next, the output unit 57 outputs the detection result of the periodontal disease detection unit 55 (S106). Step S106 corresponds to step S17 shown in Figure 4. Note that information indicating which learning model was used to detect periodontal disease may be included in the detection result and output.

In this way, in this embodiment, periodontal disease in the periodontal region of a tooth can be detected based on a learning model trained using periodontal region images with similar tooth shapes, thereby improving the accuracy of periodontal disease detection compared to, for example, detecting periodontal disease using a single learning model. Furthermore, evaluation is performed for each region using a learning model corresponding to that region, thereby improving the accuracy of periodontal disease detection.

Furthermore, when detecting periodontal disease using a single learning model, the types of classification (deformation, discoloration) performed by the learning model are the same as when detecting periodontal disease using learning models for each region as in this embodiment, and the number of classes (e.g., progression levels of periodontal disease) is not very large, so a large network is not required. Therefore, the learning model according to this embodiment can reduce the amount of processing.

Furthermore, because the determining features (i.e., the number of classes) are narrowed down, the learning efficiency of the learning model is high. In other words, according to this embodiment, the learning efficiency of the learning model is high, so the detection accuracy of the learning model can be effectively improved.

(Modification of the second embodiment)
The periodontal disease detection system according to this modification will be described below with reference to FIGS. 15 to 18B. The following description will focus on differences from the second embodiment, and descriptions of content that is the same as or similar to the second embodiment will be omitted or simplified. This modification differs from the periodontal disease detection system according to the embodiment in that the multiple periodontal disease detection tools selected by the rule selection unit (an example of a tool selection unit) include at least one or more determination rules. The multiple periodontal disease detection tools selected by the rule selection unit may include one or more machine learning models, and may include, for example, one or more determination rules and one or more machine learning models. The machine learning model is generated to output information related to periodontal disease from image data. Furthermore, the determination rule and the machine learning model are examples of tools.

Note that the configuration of the periodontal disease detection system of this modified example may be the same as that of the periodontal disease detection system 2 according to embodiment 2, and the following description will use the reference numerals of the periodontal disease detection system 2.

As shown in FIG. 15, when the area detection unit 151 detects an area within the oral cavity (S103), the rule selection unit 152 selects two periodontal disease detection tools for detecting periodontal disease from among the multiple periodontal disease detection tools stored in the memory unit 58, based on the area within the oral cavity detected by the area detection unit 151 (S204).

Here, each of the multiple periodontal disease detection tools includes at least one or more judgment rules for inputting an image of the gingiva between adjacent teeth for a tooth in image data including the relevant region and outputting information related to periodontal disease in the relevant region. The one or more judgment rules are created, for example, from gingival shape data at multiple stages of periodontal disease for each tooth belonging to the multiple regions.

The multiple periodontal disease detection tools may include multiple judgment rules, or may include one or more machine learning models and one or more judgment rules. The judgment rules may be rules using gingival color, as shown in the modified example of embodiment 1, or may be rules using gingival shape. The gingival color and gingival shape (for example, the shape of the gingival papilla) are evaluation parameters in the rule base.

For example, when a machine learning model and a judgment rule are selected, periodontal disease that is difficult to detect by one of the machine learning model and the judgment rule may be detected by the other of the machine learning model and the judgment rule. Since the detection of periodontal disease can complement each other, it is possible to prevent periodontal disease from being overlooked.

The rule selection unit 152 may select two periodontal disease detection tools using a table that associates regions within the oral cavity with the periodontal disease detection tools to be used, or may select two periodontal disease detection tools based on the history of periodontal disease detection tools used to detect periodontal disease in that region, or may select two periodontal disease detection models using some other method.

In addition, the rule selection unit 152 may select three or more periodontal disease detection tools in step S204. When three or more periodontal disease detection tools are selected, there is no particular limit to the number of machine learning models and judgment rules selected. The number of periodontal disease detection tools selected by the rule selection unit 152 may be set by the user, for example.

Next, the periodontal disease detection unit 55 performs periodontal disease detection based on the image and the two selected periodontal disease detection models (S205). When a machine learning model and a judgment rule are selected as the two periodontal disease detection models, the periodontal disease detection unit 55 performs periodontal disease detection using the machine learning model, and performs periodontal disease detection using the judgment rule. Furthermore, when two judgment rules are selected as the two periodontal disease detection models, the periodontal disease detection unit 55 performs periodontal disease detection using each of the two judgment rules.

Here, the process of detecting periodontal disease using the shape of the gingiva will be explained with reference to Figures 16 to 18B. The angle of the tip of the interdental papilla gingiva, which is one example of the shape of the gingiva, will be explained with reference to Figures 16 to 17C, and the radius of the tip of the interdental papilla gingiva, which is another example of the shape of the gingiva, will be explained with reference to Figures 18A and 18B.

Figure 16 is a diagram showing the angle θ of the tip of the interdental papilla gingiva in this modified example. When periodontal disease develops, the gums between the teeth become inflamed and swollen, causing the shape of the interdental papilla gingiva to change. Therefore, by comparing the angle θ based on the tip of the interdental papilla gingiva as the shape of the interdental papilla gingiva with the angle based on the tip of the interdental papilla gingiva in healthy gums, it is possible to determine whether gingivitis or periodontal disease is present from the angle of the tip.

Figure 16 shows the angle θ when normal for each region. The angle θ when normal can differ for each region. Therefore, one or more judgment rules are used in each region that associate the angle θ with normal and advanced levels. For convenience, Figure 16 only shows the angle θ of one of the interdental papilla gingiva on the left and right sides of the tooth.

When using a judgment rule that includes the angle θ, it is necessary to detect the angle θ, so an image that includes a portion of the tooth and at least one of the left and right interdental papilla and gingiva of the tooth is required. For example, from the perspective of preventing periodontal disease from being overlooked, it is advisable to prepare an image that includes a portion of the tooth and the left and right interdental papilla and gingiva of the tooth. The example in Figure 16 shows an image that includes the left and right interdental papilla and gingiva.

For example, the image data extraction unit 54 may extract from the second image an area including at least a portion of a tooth (e.g., a specific tooth) and at least one of the left and right interdental papilla gingiva of that tooth as image data based on the second image. This extraction process may be performed in step S102 or in step S205. In this way, when using a determination rule that includes a rule related to shape, it is preferable to prepare an image that is different from the image input to the machine learning model.

Figures 17A to 17C are diagrams explaining the method for calculating the angle of the tip of the interdental papilla gingiva in this modified example.

As shown in Figure 17A, the periodontal disease detection unit 55 uses image processing to identify the tip (e.g., apex) of the interdental papilla gingiva between teeth (e.g., a specific tooth) from the image. The periodontal disease detection unit 55 may, for example, determine the position of the gingiva located highest between the teeth as the tip of the interdental papilla gingiva, or may determine a position having a predetermined shape as the tip of the interdental papilla gingiva. Figure 17A shows an example in which the "●" portion at the end of the arrow has been identified as the tip of the interdental papilla gingiva.

As shown in Figures 17B and 17C, the periodontal disease detection unit 55 then draws a line segment downward from the tip of the interdental papilla gingiva, rotates the line segment clockwise and counterclockwise around the tip as an axis, and fixes the line segment at the position where it first comes into contact with the tooth. This forms an angle θ between the two line segments and the tip.

If the gingival shape includes the angle θ, a determination rule is used that associates the angle θ with normal and advanced levels, or normal and abnormal.

Figures 18A and 18B are diagrams showing the radius r of the tip of the interdental papilla gingiva in this modified example. When periodontal disease develops, the gums between the teeth swell due to inflammation, causing the tips of the triangles formed by the interdental papilla gingiva to become rounded. Therefore, by comparing the radius of the rounded tip of the interdental papilla gingiva of healthy gums with the rounded tip of the interdental papilla gingiva of periodontal disease, it is possible to determine whether gingivitis or periodontal disease is present from the radius of the rounded tip.

Figure 18A shows the radius r when periodontal disease is present, and Figure 18B shows the radius r when periodontal disease is not present. The radius r may be, for example, the radius of curvature of the tip of the interdental papilla gingiva. When periodontal disease is present, the radius r tends to be a large value.

If the gingival shape includes the radius r, a determination rule is used that associates the radius r with normal and advanced levels, or normal and abnormal.

Then, the periodontal disease detection unit 55 detects the user's periodontal disease based on the periodontal disease detection results of each of the two periodontal disease detection tools. For example, the periodontal disease detection unit 55 may compare the periodontal disease detection results of each of the two periodontal disease detection tools and determine the detection result of the one with the more severe periodontal disease symptoms as the detection result for the user's periodontal disease.

Furthermore, when a judgment rule and a machine learning model are selected, the periodontal disease detection unit 55 detects the user's periodontal disease by taking into consideration both the first detection result, which is the periodontal disease detection result according to the judgment rule, and the second detection result, which is the periodontal disease detection result according to the machine learning model. For example, if periodontal disease is not detected in the first detection result and periodontal disease is detected in the second detection result, the periodontal disease detection unit 55 may prioritize the first detection result and determine that periodontal disease has not been detected as the detection result for the user's periodontal disease.

Furthermore, a detection result that indicates worse periodontal disease symptoms can be rephrased as a detection result of someone with more advanced periodontal disease.

Referring again to FIG. 15, next, the output unit 57 outputs the detection result of the periodontal disease detection unit 55 (S106). If the periodontal disease detection unit 55 determines that the detection result showing worse periodontal disease symptoms is the detection result of the user's periodontal disease, the output unit 57 notifies the user of the worse detection result. Furthermore, when a machine learning model is used, the symptoms of periodontal disease may be detected as worse than they actually are due to the influence of factors such as the pattern of the gums. Therefore, if the periodontal disease detection unit 55 prioritizes the first detection result, the output unit 57 may also notify the user of the second detection result as additional information.

(Other embodiments)
The periodontal disease detection systems according to the first and second embodiments and the modifications (embodiments, etc.) of the present disclosure have been described above, but the present disclosure is not limited to these embodiments, etc.

For example, in the above embodiments, an example has been described in which an intraoral camera 10 is used primarily for photographing teeth, but the intraoral camera 10 may also be an oral care device equipped with a camera. For example, the intraoral camera 10 may also be an oral irrigator equipped with a camera.

Furthermore, in the above embodiments, mobile terminals 50 and 50a are given as examples of the second information terminal, but the second information terminal may also be a stationary information terminal.

Furthermore, in the above embodiments, examples have been described in which the mobile terminals 50, 50a are equipped with a display unit, but this is not limited to this. A display device capable of communicating with the mobile terminals 50, 50a may also be provided as a separate device from the mobile terminals 50, 50a.

Furthermore, the machine learning model according to the above embodiments has one or more calculation parameters that can be adjusted by machine learning. The machine learning model may be configured, for example, using a neural network, a regression model, a decision tree model, a support vector machine, or other functional formulas (calculation models). The machine learning method is selected appropriately depending on the machine learning model employed, and examples include, but are not limited to, backpropagation.

Furthermore, the reference color data in the above-mentioned first embodiment and the modified example of the first embodiment is not limited to measured data, as long as it is based on the color of the user's teeth, and may be, for example, the color indicated by a shade guide.

Furthermore, the periodontal disease detection unit 55 according to the modified example of the second embodiment may, for example, determine for each intraoral region whether to adopt the detection results of the machine learning model or the detection results of the judgment rule, or may determine for all intraoral regions whether to adopt the detection results of the machine learning model or the judgment rule. The intraoral regions may, for example, include two or more of the first region R1 to the fourth region R4 described above, but are not limited to this and may be set arbitrarily.

Furthermore, the rule selection unit 152 according to the variation of the second embodiment may select only two or more machine learning models, i.e., may not select a judgment rule, or may select only two or more judgment rules, i.e., may not select a machine learning model.

Furthermore, the mobile terminal 50 in the above-mentioned first embodiment and the modified example of the first embodiment only needs to include at least an output unit 57, an acquisition unit 51, and a processing unit. The processing unit performs predetermined processing, such as storing the detection results in the memory unit 58, displaying the detection results on the display unit 56, and transmitting the detection results to another device. The periodontal disease detection method (information processing method) executed by such a mobile device 50 is a periodontal disease detection method executed by one or more processors, and includes irradiating the user's oral cavity with light, outputting a first RGB image capturing an image of a photographed area including a specific tooth and the periodontal region of the specific tooth, including the gums, acquiring a periodontal disease detection result for the user based on the first RGB image, and performing predetermined processing on the acquired detection result. The detection result may include the periodontal disease detection result for the user detected based on image data based on a second RGB image in which gains of at least two of the red, green, and blue color components constituting the natural tooth area in the first RGB image are reference color data indicating the color reference of the natural tooth included in the photographed area, and the reference color data is adjusted based on the user-specific reference color data. In this case, the functions of the tooth type identification unit 52, image processing unit 53, image data extraction unit 54, and periodontal disease detection unit 55 are executed by a device external to the mobile device, such as a server device.

Furthermore, the mobile terminal 50a in the above-mentioned second embodiment and the modified example of the second embodiment only needs to include at least an output unit 57, an acquisition unit 51, and a processing unit. The processing unit performs predetermined processing, such as storing the detection results in the memory unit 58, displaying the detection results on the display unit 56, and transmitting the detection results to another device. A periodontal disease detection method (information processing method) executed by such a mobile terminal 50a is a periodontal disease detection method executed by one or more processors, and includes outputting a first image of a specific tooth in the user's oral cavity and the periodontal region of the specific tooth including the gums, obtaining a detection result of the user's periodontal disease based on the first image, and performing predetermined processing on the obtained detection result, wherein the detection result may include the detection result of the user's periodontal disease detected based on image data based on the second image, and a periodontal disease detection tool selected from a plurality of periodontal disease detection tools corresponding to each of a plurality of regions in the oral cavity, each of which is trained to input image data including the region and output information about periodontal disease in the region, depending on which of the plurality of regions defined by dividing the tooth row a second image including the periodontal region of the specific tooth generated from the first image is an image of. In this case, the functions of the image processing unit 53, area detection unit 151, rule selection unit 152, image data extraction unit 54, and periodontal disease detection unit 55 are executed by a device external to the mobile terminal, such as a server device.

Furthermore, each processing unit included in the periodontal disease detection systems according to the first and second embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

Furthermore, integrated circuits are not limited to LSIs, but may be realized using dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI manufacturing, or reconfigurable processors, which allow the connections and settings of circuit cells within the LSI to be reconfigured, may also be used.

Furthermore, in the above-mentioned first and second embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program appropriate for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

Furthermore, the division of functional blocks in the block diagram is one example; multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to other functional blocks. Furthermore, the functions of multiple functional blocks with similar functions may be processed in parallel or time-shared by a single piece of hardware or software.

Furthermore, the periodontal disease detection systems according to the first and second embodiments (e.g., mobile terminals 50, 50a) may be realized as a single device or may be realized by multiple devices. When the periodontal disease detection system is realized by multiple devices, the components of the periodontal disease detection system may be distributed in any manner among the multiple devices. For example, at least some of the functions of the periodontal disease detection system may be realized by the intraoral camera 10 (e.g., signal processing unit 30). Furthermore, when the periodontal disease detection system is realized by multiple devices, the communication method between the multiple devices is not particularly limited, and may be wireless communication or wired communication. Furthermore, wireless communication and wired communication may be combined between the devices.

The present disclosure may also be realized as a periodontal disease detection method executed by a periodontal disease detection system. The present disclosure may also be realized as an intraoral camera, mobile terminal, or cloud server included in the periodontal disease detection system.

Furthermore, the order in which each step is executed in the sequence diagram is merely an example to specifically explain the present disclosure, and orders other than those described above may also be used. Furthermore, some of the steps may be executed simultaneously (in parallel) with other steps.

Furthermore, one aspect of the present disclosure may be a computer program that causes a computer to execute each of the characteristic steps included in the periodontal disease detection method shown in any of Figures 4, 14, and 15.

Furthermore, for example, the program may be a program to be executed by a computer. Furthermore, one aspect of the present disclosure may be a computer-readable non-transitory recording medium on which such a program is recorded. For example, such a program may be recorded on a recording medium and distributed or circulated. For example, by installing the distributed program in a device having another processor and having that processor execute the program, it becomes possible to cause that device to perform each of the above processes.

The periodontal disease detection system and the like relating to one or more aspects have been described above based on embodiments 1 and 2, but the present disclosure is not limited to these embodiments 1 and 2. As long as they do not deviate from the spirit of the present disclosure, various modifications that would occur to those skilled in the art to embodiments 1 and 2 and their respective modifications, as well as forms constructed by combining components from different embodiments, may also be included within the scope of one or more aspects.

This disclosure is useful for periodontal disease detection systems that detect periodontal disease.

10 Intraoral camera 10a Head unit 10b Handle unit 20 Hardware unit 21 Photography unit 22 Sensor unit 23 Illumination unit 23A First LED
23B Second LED
23C Third LED
23D Fourth LED
24 Operation unit 30 Signal processing unit 31 Camera control unit 32, 53 Image processing unit 33 Control unit 34 Lighting control unit 35 Memory unit 40 Communication unit 50, 50a Portable terminal (second information terminal)
51 Acquisition unit (first acquisition unit, second acquisition unit)
52 Tooth type identification unit 54 Image data extraction unit 55 Periodontal disease detection unit 56 Display unit 57 Output unit 58 Storage unit 151 Area detection unit 152 Rule selection unit (tool selection unit)
F1, F2, F3 Rectangular frame D Predetermined distance r Radius R1 First area R2 Second area R3 Third area R4 Fourth area θ Angle

Claims (24)

A periodontal disease detection system that detects periodontal disease in a user based on an image taken inside the oral cavity,
a first acquisition unit that irradiates light into the oral cavity of the user and acquires a first RGB image of an imaging area including a specific tooth and a periodontal region of the specific tooth including the gums;
a second acquisition unit that acquires reference color data indicating a reference color of natural teeth included in the photographed area according to the user;
an image processing unit that generates a second RGB image by adjusting the gains of at least two color components among red, green, and blue color components that constitute the natural tooth region in the first RGB image based on the reference color data;
a periodontal disease detection unit that detects periodontal disease of the user based on image data based on the second RGB image. The periodontal disease detection system of claim 1, further comprising an image data extraction unit that extracts, from the second RGB image, periodontal region image data including free gingiva, which is the gingiva that surrounds the cervical region including the interdental papilla and marginal gingiva in a band-like shape, and a portion of the attached gingiva that is continuous with the free gingiva and extends from the bottom of the gingival sulcus to the gingival-alveolar junction, as the image data based on the second RGB image. The periodontal disease detection system according to claim 2 , wherein the periodontal region image data further includes left and right interdental papillae and gingiva. The periodontal disease detection system according to any one of claims 1 to 3, wherein the reference color data is set based on the color of the user's natural teeth. The image processing unit
Identifying a specific pixel area including pixels whose values based on the pixel values of the second RGB image are within a predetermined range;
generating a third RGB image by adjusting the gains of at least two of the red, green, and blue color components that constitute the natural tooth region in the first RGB image based on the reference color data;
The periodontal disease detection system according to any one of claims 1 to 3, wherein the image data based on the second RGB image is image data based on the third RGB image. The periodontal disease detection system of claim 5, wherein the image processing unit generates an HSV image by converting the color space of the second RGB image into an HSV space, and identifies as the specific pixel area a pixel area in which one or more pixels of the HSV image that satisfy at least one of a first predetermined range for saturation, a second predetermined range for hue, and a third predetermined range for lightness are located. The periodontal disease detection system of claim 6, wherein the image processing unit identifies as the specific pixel area a pixel area in which one or more pixels are located, the red pixel values of which fall within a first range, the green pixel values of which fall within a second range, and the blue pixel values of which fall within a third range, of the images constituting the area of the natural tooth in the second RGB image. The periodontal disease detection system according to claim 6 , wherein the specific pixel region includes a plaque region. The periodontal disease detection system according to claim 2 , wherein the image data extraction unit extracts the periodontal region from the tip of the interdental papilla gingiva of the specific tooth to the free gingival sulcus from the second RGB image as the periodontal region image data. The periodontal disease detection system of claim 9, wherein the image data extraction unit extracts a rectangular frame area that circumscribes the left and right sides of the contour of the specific tooth and includes from the tip of the specific tooth to the free gingival sulcus from the second RGB image as the periodontal region image data. The periodontal disease detection system described in claim 1 or 2, wherein the periodontal disease detection unit detects the periodontal disease of the user by inputting image data based on the second RGB image into a learning model that is trained to input image data including a gingival area and output information regarding periodontal disease in the image data. The periodontal disease detection system according to claim 11, wherein the learning model outputs an estimation result of at least one of periodontal pocket depth, Bleeding On Probing (BOP), and Gingival Index (GI) value as the information related to the periodontal disease. The periodontal disease detection system according to claim 1 or 2, wherein the periodontal disease detection unit receives as input the R, G, and B values of the gum color acquired from image data based on the second RGB image and detects the user's periodontal disease in the area based on a periodontal disease detection tool created from the range of R, G, and B values in the RGB color space of normal gum color and the range of R, G, and B values of gum color for each of multiple stages of periodontal disease. The periodontal disease detection system of claim 1 or 2, wherein the periodontal disease detection unit inputs at least one of the hue information, saturation information, and brightness information of the color of the gums obtained from image data based on the second RGB image and detects the periodontal disease of the user in the area based on a periodontal disease detection tool created from at least one of the hue information, saturation information, and brightness information of the color of normal gums in the HSV color space and at least one range of the hue information, saturation information, and brightness information of the color of the gums for each of multiple stages of periodontal disease. The periodontal disease detection system according to claim 1 or 2, wherein the periodontal disease detection unit inputs at least one of the hue information, saturation information, and luminance information of the color of the gums obtained from image data based on the second RGB image and detects the periodontal disease of the user in the relevant area based on a periodontal disease detection tool created from at least one of the hue information, saturation information, and luminance information of the color of normal gums in the HSL color space and at least one range of the hue information, saturation information, and luminance information of the color of the gums for each of multiple stages of periodontal disease. The periodontal disease detection system according to claim 1 or 2, further comprising a tooth type identification unit that identifies the type of the specific tooth from the first RGB image or the second RGB image. an output unit that outputs information indicating the periodontal disease detected by the periodontal disease detection unit to a first information terminal of a dentist other than the user;
The periodontal disease detection system described in claim 1 or 2, wherein the output unit further outputs to the user's second information terminal information regarding whether the user needs to receive a medical examination, which information was acquired via the first acquisition unit and entered into the first information terminal. The periodontal disease detection system according to claim 1 or 2, further comprising an output unit that outputs information indicating the periodontal disease detected by the periodontal disease detection unit to a second information terminal of the user. the second acquisition unit acquires the reference color data before acquiring the first RGB image;
The periodontal disease detection system according to claim 1 or 2, further comprising a storage unit that stores the reference color data. the reference color data is color data of the natural tooth obtained when a first light is irradiated onto the natural tooth;
The periodontal disease detection system according to claim 1 or 2, wherein the first RGB image is an image obtained when the imaging area is irradiated with a second light different from the first light. the at least two color components include at least two of a first red pixel average value of a plurality of red pixel values of a plurality of pixels constituting the region of the natural tooth in the first RGB image, a first green pixel average value of a plurality of green pixel values of the plurality of pixels, and a first blue pixel average value of a plurality of blue pixel values of the plurality of pixels;
The periodontal disease detection system according to claim 1 or 2, wherein the image processing unit generates the second RGB image by adjusting the gain so that a difference between the at least two color components and the reference color data falls within a predetermined range. A periodontal disease detection method executed by a periodontal disease detection system that detects periodontal disease in a user based on an image taken inside the oral cavity, comprising:
irradiating light into the oral cavity of the user to acquire a first RGB image capturing an imaging area including a specific tooth and a periodontal region of the specific tooth including the gums;
acquiring reference color data indicating a reference color of the natural teeth included in the photographed area, the reference color data being corresponding to the user;
generating a second RGB image by adjusting the gains of at least two of the red, green, and blue color components that constitute the natural tooth region in the first RGB image based on the reference color data;
and detecting periodontal disease of the user based on image data based on the second RGB image. 1. A method for detecting periodontal disease executed by one or more processors, comprising:
The one or more processors:
irradiating light into the oral cavity of the user, and outputting a first RGB image obtained by capturing an image of an imaging area including a specific tooth and a periodontal region of the specific tooth including the gums;
obtaining a detection result of periodontal disease of the user based on the first RGB image;
Execute a predetermined process on the acquired detection result;
A periodontal disease detection method, wherein the obtained detection results include a detection result of the periodontal disease of the user, detected based on image data based on a second RGB image adjusted based on reference color data corresponding to the user, where the gains of at least two of the red, green, and blue color components that constitute the natural tooth area in the first RGB image are reference color data that indicate the standard color of the natural tooth included in the captured area. A program for causing a computer to execute the periodontal disease detection method described in claim 22 or 23.