JP7058035B2 - Dental plaque display system - Google Patents

Description

The present invention relates to a technique for visually emphasizing and visualizing a portion where dental plaque is present in information processing of a captured image, and in particular, it can be efficiently realized by using a general-purpose device and more reliably. It relates to a plaque display system capable of highlighting the presence of plaque.

Conventionally, by applying a plaque-revealing agent to the tooth surface and coloring the adhered part of plaque in red, it is possible to emphasize and visualize the plaque, check the unpolished part, and grasp the habit of brushing teeth. It has been. However, such a method is troublesome and burdensome to work, and it is necessary to be careful not to get the plaque-revealing agent on clothes, so it is a method that is difficult to use casually as a method for daily use. rice field.

On the other hand, an imaging unit that captures the inside of the oral cavity, an extraction unit that extracts the toothpaste portion based on color information from the original image data captured by the imaging unit, and an emphasized image that emphasizes the extracted toothpaste region. A toothpaste detection system including a highlighted image generation unit that generates data and a composite unit that synthesizes the emphasized image data with the original image data to generate composite image data and displays it on the display unit has been proposed (Patent Documents). See 1). However, this requires a dedicated device, recognizes the plaque part, determines the amount of adhesion based on the area ratio, and highlights it in a color according to the amount of adhesion, which requires a lot of calculations. There are problems that processing capacity is required, costs increase, errors due to the environment such as lighting are likely to occur, and stable white lighting is required.

Japanese Unexamined Patent Publication No. 2011-189293

Therefore, what the present invention seeks to solve in view of the above-mentioned situation is that it is possible to efficiently provide an image in which plaque is emphasized and visualized by a simple process, it can be realized by using a general-purpose device, and plaque is surely obtained. The point is to provide a plaque display system capable of highlighting.

As a result of diligent studies in view of the current situation, the present inventor emphasizes the plaque region by changing the hue value and saturation value of the captured image in the oral cavity without extracting the plaque region. It was found that it is possible to obtain a plaque-enhanced image by a simple process without having to recognize the plaque part or calculate the area ratio. The present invention has been completed.

That is, the present invention includes the following inventions.
(1) An image pickup means for imaging the inside of the oral cavity, an image adjustment means for changing at least the hue value and the saturation value of the image pickup image data captured by the image pickup means to obtain an image in which plaque is emphasized, and the image. A plaque display system equipped with an image display means for displaying an image adjusted by the adjustment means.

(2) A color space conversion means for converting the captured image data from an RGB color space into a color space composed of components including hue and saturation is provided, and the image adjusting means is used from the converted components including hue and saturation. (1) The toothpaste display system according to (1), which changes the hue value and the saturation value in a color space.

(3) A plurality of sets of changes in the hue value and the saturation value by the image adjusting means are set in advance according to the magnitude of the hue value or the saturation value in a specific region of the captured image data. The plaque display system according to (1) or (2).

(4) A region detecting means for detecting a tooth region from the captured image data is provided, and the image adjusting means limits the captured image data to the detected tooth region, and at least the hue value and the above-mentioned hue value. The plaque display system according to any one of (1) to (3), wherein the saturation value is changed to obtain an adjusted image in which plaque is emphasized.

(5) The region according to (4), wherein the region detecting means detects a region consisting of a white hue surrounded by a region consisting of a red hue as a tooth region based on the hue value of the captured image data. Plaque display system.

(6) A problem of the user's tooth brushing operation based on the image storage means for storing the adjusted image data adjusted by the image adjusting means and the plurality of adjusted image data of the user stored in the image storage means. An analysis means for analyzing points and a guidance information generation means for generating tooth brushing guidance information for the user based on the analyzed problem of tooth brushing are provided, and the image display means matches the display of the adjustment image. The toothpaste display system according to any one of (1) to (5), which displays the generated toothpaste instruction information.

(7) The tooth brushing motion storage means for accumulating the tooth brushing motion data of the user is provided, and the analysis means is based on the tooth brushing motion data of the user and the plurality of adjusted image data stored in the tooth brushing motion storage means. , The toothpaste display system according to (6), which analyzes the above-mentioned problems.

(8) The plaque display system according to (6) or (7), wherein the plurality of adjusted image data are adjusted image data before and after brushing teeth.

(9) The plaque display system according to any one of (6) to (8), wherein the image display means displays an image having a different degree of emphasis of plaque according to the content of the instruction information.

According to the present invention as described above, the image adjusting means for changing at least the hue value and the saturation value of the captured image data captured by the imaging means to obtain an image in which plaque is emphasized, and the image adjusting means. Since it is equipped with an image display means for displaying the adjusted image, there is no need to recognize the plaque part or calculate the amount of adhesion, and it is possible to efficiently provide an image that emphasizes and visualizes the plaque with simple processing, and the cost is high. It is possible to suppress the image, and it is not easily affected by the environment such as the surrounding brightness, and it is possible to visually highlight the plaque more reliably while using a general-purpose device.

In particular, it is provided with a color space conversion means for converting captured image data from an RGB color space into a color space composed of components including hue and saturation, and the image adjusting means is a color composed of components including converted hue and saturation. By changing the hue value and the saturation value in space, it is possible to obtain an adjusted image in which the toothpaste is emphasized more reliably.

Further, the amount of change in the hue value and the saturation value by the image adjusting means is selected from a plurality of sets of image adjustment conditions set in advance according to the magnitude of the hue value or the saturation value in a specific area of the captured image data. However, in what is determined, even if the environment such as the ambient brightness is different and the brightness of the original captured image is significantly different, it is possible to obtain an adjusted image in which the toothpaste is similarly emphasized.

Further, a region detecting means for detecting a tooth region from the captured image data is provided, and the image adjusting means limits the captured image data to the detected tooth region and at least performs a hue value and a saturation value. If the adjusted image is changed and the plaque is emphasized, it is possible to avoid the image in which the teeth other than the teeth and the lips are emphasized. Can provide a more readable image that can give.

Further, when the region detecting means defines that the region consisting of the white hue surrounded by the region consisting of the red hue is detected as the tooth region based on the hue value of the captured image data, the adjusted image is displayed. Using the hue value required to obtain it, the region of only the teeth can be efficiently detected.

Further, based on the image storage means for storing the adjusted image data adjusted by the image adjusting means and the plurality of the adjusted image data of the user stored in the image storage means, the problem of the tooth brushing operation of the user can be solved. An analysis means for analysis and a guidance information generation means for generating tooth brushing guidance information for the user based on the analyzed problem of tooth brushing are provided, and the image display means is aligned with the display of the adjustment image. In displaying the generated tooth brushing instruction information, not only the image is provided to make the user aware of the presence and position of plaque and the voluntary action (improvement of tooth brushing) is expected, but also the instruction information. As a result, it is possible to give advice on how to improve toothpaste suitable for the user from the analysis results based on the past adjusted image data, and it is possible to encourage more positive improvement, and the instruction content is also subjective to the user. It can be provided with objective content without being affected.

Further, the tooth brushing motion storage means for accumulating the tooth brushing motion data of the user is provided, and the analysis means is based on the tooth brushing motion data of the user and the plurality of adjusted image data stored in the tooth brushing motion storage means. In the analysis of the problem, it is possible to analyze the problem more accurately after grasping the tooth brushing operation of each user, and the content of the generated tooth brushing instruction information is also more suitable for the user. It can be the content of instruction. For example, there are multiple residual plaques, such as whether the position or angle at which the toothbrush bristles are applied is incorrect, the pressing force is insufficient, the toothbrush's operating speed or moving distance is not appropriate, or the operating time is insufficient. Appropriate problems can be extracted from the factors, and the instruction content can appropriately reflect the user's usual habit of brushing teeth.

Further, if the plurality of adjusted image data are the adjusted image data before and after brushing the teeth, the same problems of the tooth brushing as in the case of using the dental plaque revealing agent can be analyzed more accurately, and the user can also analyze the problems. The effect of brushing teeth can be recognized more easily and directly.

Further, if the image display means displays an image in which the degree of emphasis of dental plaque is strengthened or the color tone is changed to give a pathological impression according to the content of the instruction information, the user, especially a child, can see the image. It is possible to give a stronger motivation for brushing teeth.

In addition, by incorporating the present invention into an existing tooth brushing instruction system, it becomes possible to provide more appropriate instruction based on the actual residual condition of dental plaque, not by evaluation by comparison with a standard.

Explanatory drawing which shows the structure of the plaque display system which concerns on 1st Embodiment of this invention. A block diagram showing the configuration of a mobile terminal that also constitutes a plaque display system. Similarly, a flow chart showing a processing procedure in a plaque display system. Explanatory drawing which shows how the captured image is changed to the adjustment image which emphasized plaque. Similarly, an explanatory diagram showing how the captured image is changed to the adjusted image. The block diagram which shows the structure of the mobile terminal of the plaque display system which concerns on 2nd Embodiment of this invention. Similarly, a flow chart showing a processing procedure in a plaque display system. (A) is an explanatory diagram showing how the captured image is changed to an adjusted image limited to the tooth region, and (b) is an explanatory diagram showing an example of displaying as a composite image. Similarly, (a) is an explanatory diagram showing how the captured image is changed to an adjusted image limited to the tooth region, and (b) is an explanatory diagram showing an example of displaying as a composite image. Explanatory drawing which shows the structure of the plaque display system which concerns on 3rd Embodiment of this invention. A block diagram showing the configuration of a mobile terminal that also constitutes a plaque display system. A block diagram showing the configuration of a server computer that also constitutes a plaque display system. Similarly, a flow chart showing a processing procedure in a plaque display system. The block diagram which shows the structure of the server computer which comprises the plaque display system which concerns on 4th Embodiment of this invention. Similarly, a flow chart showing a processing procedure in a plaque display system. Explanatory drawing which shows the structure of the plaque display system which concerns on 5th Embodiment of this invention. A block diagram showing the configuration of a mobile terminal that also constitutes a plaque display system. A block diagram showing the configuration of a server computer that also constitutes a plaque display system.

Next, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. First, the first embodiment will be described with reference to FIGS. 1 to 5.

As shown in FIG. 1, the dental plaque display system S of the present embodiment is configured only with the functions provided by the user's mobile terminal 10. As shown in FIG. 2, the mobile terminal 10 is realized by an image pickup means 11 for imaging the inside of the oral cavity and a built-in processing device 12, and at least the hue value and the saturation value of the image pickup image data captured by the image pickup means 11 are obtained. It is provided with at least an image adjusting means (image adjusting processing unit 12c described later) for changing and making an image in which plaque is emphasized, and an image displaying means 13 for displaying an image adjusted by the image adjusting means. As such a mobile terminal 10, a general-purpose mobile terminal such as a smartphone terminal or a tablet terminal provided with a display such as a CCD camera and a liquid crystal can be used in addition to the dedicated terminal. Those provided with lighting means for illuminating the imaging position are preferable.

The processing device 12 of the mobile terminal 10 is mainly composed of a CPU such as a microprocessor, and transmits and receives various information to and from the image pickup means 11, the image display means 13, and the storage means 14 through an input / output unit and a bus line. .. The storage means 14 is composed of storage memories such as RAMs and ROMs inside and outside the processing device 12, a hard disk, and the like, and stores programs and processing data that define procedures for various processing operations in the processing device 12.

Functionally, as shown in FIG. 2, the processing device 12 acquires the image data of the oral cavity captured by the image pickup means 11 from the image pickup means 11 and stores it in the image pickup image data storage unit 14a of the storage means 14. The captured image storage processing unit 12a, the color space conversion processing unit 12b as a color space conversion means for converting the captured image data from the RGB color space into a color space composed of components including hue and saturation, and the converted hue. An image adjustment processing unit 12c that changes the hue value and the saturation value in a color space composed of components including saturation and stores the changed adjustment image data in the adjustment image data storage unit 14b of the storage means 14, and the adjustment image. It includes at least an adjusted image display processing unit 12d that takes out the adjusted image data stored in the data storage unit 14b and displays it by the image display means 13, and these functions are realized by the above program.

The captured image data for which the color space conversion processing unit 12b converts the color space may be data immediately after being captured by the imaging means 11, or may be past image data stored and managed by the storage means 14. Further, the color space composed of components including hue and saturation, which is converted from the RGB color space by the color space conversion processing unit 12b, corresponds to, for example, an HLS color space or an HSV color space. For example, you can use the Android Support V4 graphics package.

The change of the hue value and the saturation value by the image adjustment processing unit 12c is performed one pixel at a time, and the change amount is, for example, -40 ± 5 of the original hue value and the original saturation value. A predetermined change amount such as +30 ± 5 may be set, or a set of a plurality of change amounts is set in advance for each specific area and changed using the change amount selected and determined by the user's selection operation. It is also preferable to do so. The amount of change of −40 ± 5 of the original hue value and +30 ± 5 of the original saturation value is preferable in that the color is close to the color when dyed with a conventional plaque revealing agent.

Further, as such a change amount, select from a plurality of sets of image adjustment conditions set in advance according to the comparison information between the magnitude of the hue value or saturation value of a specific region of the captured image data and the reference value of that region. It is also a preferable embodiment to use the amount of change of the set determined by the hue value or the saturation value of the specific region. Specifically, for example, a table of a plurality of sets of change amounts is prepared in advance with the hue value of the tooth region as a threshold value, and the hue and saturation are changed using the change amount determined from the hue value of the tooth region. That is the case.

That is, depending on the shooting conditions (bright place, dark place), the plaque may or may not be sufficiently emphasized in the adjusted image obtained by changing a certain amount from the captured image. Therefore, by determining the threshold value of hue or saturation before adjustment of a specific area and allocating the optimum amount of change corresponding to that value, it is possible to sufficiently emphasize plaque in an image of any brightness. You can.

The image adjustment processing unit 12c is not limited to changing only the hue value and the saturation value, and can also change other parameters of the converted color space, such as brightness and brightness.

As shown in FIG. 3, the processing operation procedure of the dental plaque display system S of the present embodiment is acquired by the image pickup image storage processing unit 12a when the oral cavity is first imaged by the image pickup means 11 by the user (S101). The captured image data is stored in the captured image data storage unit 14a of the storage means 14 (S102). Then, the color space conversion processing unit 12b takes out the captured image data from the captured image data storage unit 14a automatically or by the user's operation, and converts the RGB color space into a color space composed of components including hue and saturation. At the same time, the color space data is stored in the converted image data storage unit 14c of the storage means 14 (S103).

Next, the image adjustment processing unit 12c changes the hue value and the saturation value in the color space including the converted hue and saturation component automatically or by the user's operation, and the changed adjustment image. Data is stored in the adjusted image data storage unit 14b of the storage means 14 (S104). Then, the adjustment image display processing unit 12d takes out the adjustment image data stored in the adjustment image data storage unit 14b and displays it by the image display means 13 (S105). 4 and 5 show the original captured image 100 and the adjusted image 101 whose hue value and saturation value are changed by the image adjusting processing unit 12c from the data of the captured image 100 and displayed by the image display means 13. ing.

Next, a second embodiment will be described with reference to FIGS. 6 to 9.

In addition to the first embodiment, the plaque display system S1 of the present embodiment includes a region detecting means for detecting a tooth region from captured image data, and an image adjusting means (image adjusting processing unit 12c) provides captured image data. Of these, the adjusted image in which the plaque is emphasized is obtained by changing at least the hue value and the saturation value only in the detected tooth region. According to this, as shown in FIGS. 8 and 9, only the tooth region is adjusted to the image in which the plaque is emphasized, so that other parts, lips and gums are also changed to obtain an unnatural image. Can be prevented.

The region detection means uses a method of detecting a region consisting of a white hue surrounded by a region consisting of a red hue as a tooth region based on the hue value of the captured image data. Is less likely to occur, which is a preferable example. In addition, a known image recognition processing technique may be used to recognize and detect the tooth region based on the detection of the shape, shape, hue value, and the like.

In this example, as shown in FIG. 6, the processing device 12 of the mobile terminal 10 is provided as an area detection processing unit 12e, and the area detection processing unit 12e extracts the converted image data storage unit 14c of the storage means 14. Based on the hue value of the data, as described above, a white hue region surrounded by a red hue region is specified, and the region is detected as a tooth region. Specifically, for example, a region having a white hue value of 0 to 60 close to plaque or teeth, surrounded by a region having a red hue value of 320 to 360 close to the color of the lips or gums is detected. By emphasizing only these 0 to 60 pixels, the processed image becomes easier to see. As a result, even if a skin portion outside the oral cavity having a hue value of 0 to 60 is present in the image, it is possible to more reliably detect only the tooth without erroneously detecting the tooth.

As shown in FIG. 7, the processing operation procedure of the dental plaque display system S1 of the present embodiment is acquired by the image pickup image storage processing unit 12a when the inside of the oral cavity is first imaged by the image pickup means 11 by the user (S201). The captured image data is stored in the captured image data storage unit 14a of the storage means 14 (S202). Then, the color space conversion processing unit 12b takes out the captured image data from the captured image data storage unit 14a automatically or by the user's operation, and converts the RGB color space into a color space composed of components including hue and saturation. At the same time, the color space data is stored in the converted image data storage unit 14c of the storage means 14 (S203). Up to this point, it is the same as the first embodiment described above.

Next, the area detection processing unit 12e automatically or manually operates the white hue surrounded by the red hue region based on the color space data stored in the converted image data storage unit 14c. A region is specified and the region is detected as a tooth region (S204). Next, the image adjustment processing unit 12c changes the hue value and the saturation value of the detected tooth region, and stores the changed adjustment image data in the adjustment image data storage unit 14b of the storage means 14 (S205). .. Then, the adjustment image display processing unit 12d takes out the adjustment image data stored in the adjustment image data storage unit 14b and displays it by the image display means 13 (S206).

In this example, the adjusted image data is image data of only the tooth region, and as shown in FIGS. 8A and 9A, the adjusted image 101 displayed by the image display means 13 is only the tooth region. As shown in FIGS. 8 (b) and 9 (b), the image of the tooth region of the original captured image 100 is replaced with the adjusted image 101 of the tooth region, that is, the image of the tooth. It is also a preferred embodiment to display the region adjustment image 101 as a composite image 102 combined with the original captured image 100.

Other configurations and modifications are the same as those in the first embodiment described above, the same configurations are designated by the same reference numerals, and the description thereof will be omitted.

Next, a third embodiment will be described with reference to FIGS. 10 to 13.

As shown in FIG. 10, the dental plaque display system S2 of the present embodiment is composed of a user's mobile terminal 10 and a server computer 20 that communicates with and connects to the mobile terminal 10 via a communication network N such as the Internet. be. Then, in the above-mentioned first embodiment and the second embodiment, the conversion of the color space of the captured image and the adjustment of the image are processed in the mobile terminal 10, but in the present embodiment, these processes are processed by the server computer 20. Is performed, and the server computer 20 is configured to manage the mobile terminals 10 of a plurality of users.

As shown in FIG. 11, the mobile terminal 10 includes a communication control unit 15, and the processing device 12 includes a color space conversion processing unit 12b and an image adjustment processing unit 12c provided in the first embodiment and the second embodiment. Omitted, instead, a captured image transmission processing unit 12f for transmitting captured image data to the server computer 20 and an adjusted image receiving processing unit 12g for receiving adjusted image data from the server computer 20 are provided. In the storage means 14, the converted image data storage unit 14c is omitted. Other configurations, modifications, and the like are the same as those of the first embodiment and the second embodiment.

As shown in FIG. 12, the server computer 20 is composed of a general-purpose server computer including a processing device 21, a storage means 22, and a communication control unit 23. The processing device 21 is mainly composed of a CPU such as a microprocessor, and transmits and receives various information to and from the storage means 22 and the communication control unit 23 through the input / output unit and the bus line. The storage means 22 is composed of storage memories such as RAMs and ROMs inside and outside the processing device 21, a hard disk, and the like, and stores programs and processing data that define procedures for various processing operations in the processing device 21.

The processing device 21 functionally receives the captured image data from the mobile terminal 10 through the communication network N, stores the captured image data in the captured image data storage unit 22a of the storage means 22, and stores the captured image data in RGB. The color space conversion processing unit 21b as a color space conversion means for converting a color space into a color space composed of components including hue and saturation, and the hue value and the above-mentioned hue value in the color space composed of components including converted hue and saturation. An image adjustment processing unit 21c that changes the saturation value and stores the changed adjustment image data in the adjustment image data storage unit 22b of the storage means 22, and an adjustment image transmission processing unit 21d that transmits the adjustment image data to the mobile terminal 10. At least, these functions are realized by the above program.

In the present embodiment, the contents of the color space conversion processing unit 21b and the image adjustment processing unit 21c included in the processing device 21 of the server computer 20 are the processing devices 12 of the mobile terminal 10 in the first and second embodiments described above. Since it is the same as the color space conversion processing unit 12b and the image adjustment processing unit 12c provided in the above, the description thereof will be omitted. Similar to the second embodiment, it is also a preferable embodiment that the processing device 21 includes an area detection processing unit and performs the same processing as that of the second embodiment.

As shown in FIG. 13, the processing operation procedure of the toothpaste display system S2 of the present embodiment is as follows: First, when the oral cavity is imaged by the image pickup means 11 of the mobile terminal 10 by the user (S301), the image pickup image storage processing unit The captured image data acquired by 12a is stored in the captured image data storage unit 14a of the storage means 14, and the captured image transmission processing unit 12f transmits the captured image data to the server computer 20 (S302).

The captured image reception processing unit 21a of the server computer 20 receives the captured image data and stores it in the captured image data storage unit 22a of the storage means 22 (S303). Then, the color space conversion processing unit 21b takes out the captured image data from the captured image data storage unit 22a, converts the RGB color space into a color space composed of components including hue and saturation, and stores the color space data in the storage means 22. It is stored in the converted image data storage unit 22c (S304).

Next, the image adjustment processing unit 21c changes the hue value and the saturation value of the converted image data, and stores the changed adjustment image data in the adjustment image data storage unit 22b of the storage means 22 (S305). Then, the adjusted image transmission processing unit 21d transmits the adjusted image data to the mobile terminal 10 (S306).

The adjusted image receiving processing unit 12g of the mobile terminal 10 stores the received adjusted image data in the adjusted image data storage unit 14b (S307). Then, the adjustment image display processing unit 12d takes out the adjustment image data stored in the adjustment image data storage unit 14b and displays it by the image display means 13 (S308).

Next, a fourth embodiment will be described with reference to FIGS. 14 and 15.

In the toothpaste display system S3 of the present embodiment, in addition to the third embodiment, the server computer 20 stores a plurality of the users stored in the adjusted image data storage unit 22b of the storage means 22 as shown in FIG. Based on the adjustment image data of the analysis processing unit 21e as an analysis means for analyzing the problem of the tooth brushing operation of the user, and the tooth brushing instruction to the user based on the problem of tooth brushing analyzed by the analysis processing unit 21e. A guidance information generation processing unit 21f as a guidance information generation means for generating information is provided, and the adjustment image transmission processing unit 21d transmits the generated guidance information together with the adjustment image data to the mobile terminal 10, and the mobile terminal 10 The adjustment image display processing unit 12d displays the generated tooth brushing instruction information in accordance with the display of the adjustment image by the image display means.

In the adjusted image data storage unit 22b, a plurality of adjusted image data of each user is accumulated for each user. Although not shown, an ID is assigned to each user, and the storage means 22 manages the user information together with the attribute information, and also manages the captured image data, the adjusted image data, the converted image data, and the like in the storage means 22. Information is stored and managed for each user.

As shown in FIG. 15, the processing operation procedure of the toothpaste display system S3 of the present embodiment is as follows. The captured image data acquired by 12a is stored in the captured image data storage unit 14a of the storage means 14, and the captured image transmission processing unit 12f transmits the captured image data to the server computer 20 (S402). At this time, the ID information that identifies the user is also transmitted.

The captured image reception processing unit 21a of the server computer 20 receives the captured image data and stores it in the captured image data storage unit 22a of the storage means 22 (S403). Then, the color space conversion processing unit 21b takes out the captured image data from the captured image data storage unit 22a, converts the RGB color space into a color space composed of components including hue and saturation, and stores the color space data in the storage means 22. It is stored in the converted image data storage unit 22c (S404).

Next, the image adjustment processing unit 21c changes the hue value and the saturation value of the converted image data, and stores the changed adjustment image data in the adjustment image data storage unit 22b of the storage means 22 (S405). Next, the analysis processing unit 21e analyzes the problem of the user's tooth brushing operation based on the plurality of adjusted image data of the user stored in the adjusted image data storage unit 22b (S406).

In the analysis by the analysis processing unit 21e, for example, the habit of brushing teeth may be determined from the accumulated past adjustment image data of the user, and the analysis may be expanded to the tooth brushing instruction function. Specifically, if it is detected that a certain part of the tooth has a red (with plaque) hue value every day, it means that there is always unpolished part (habit) in that specific part. Output as a problem. Detection can be detected by performing statistical processing in a certain period. Further, it is also preferable that the plurality of adjusted image data used for the analysis are the adjusted image data before and after tooth brushing obtained from the captured image data before and after tooth brushing in which the user has imaged the oral cavity before and after tooth brushing. As a result, the analysis processing unit 21 can compare the adjusted image data to more accurately analyze the problem of the tooth brushing operation.

Then, the instruction information generation processing unit 21f generates tooth brushing instruction information for the user based on the analyzed tooth brushing problem (S407). Guidance information includes, for example, "this is not polished", "it is better to polish here", etc., and guidance is given by displaying the part. It is also preferable to analyze past problems and instructional information provided in the past by machine learning, make inferences, and generate more suitable instructional information. Any method such as deep learning by a neural network can be used for this machine learning.

Next, the adjusted image transmission processing unit 21d transmits the adjusted image data and the instruction information to the mobile terminal 10 (S408). The adjusted image receiving processing unit 12g of the mobile terminal 10 stores the received adjusted image data and tooth brushing instruction information in the adjusted image data storage unit 14b (S409). Then, the adjustment image display processing unit 12d takes out the adjustment image data and the tooth brushing instruction information stored in the adjustment image data storage unit 14b, and displays them by the image display means 13 (S410).

At this time, it is also preferable to display an image having a different degree of visual enhancement of dental plaque as an adjustment image to be displayed according to the problem by the analysis processing unit 21 and the content of the instruction information. Specifically, the larger the problem analyzed by the analysis processing unit 21, the more preferable it is to display the adjusted image readjusted to the hue and saturation in which the plaque is emphasized, thereby brushing the teeth to the user. You can make a strong impression of the good or bad of.

Other configurations and modifications are the same as those in the third embodiment described above, the same configurations are designated by the same reference numerals, and their description will be omitted.

Next, a fifth embodiment will be described with reference to FIGS. 16 to 18.

In the toothpaste display system S4 of the present embodiment, in addition to the fourth embodiment, as shown in FIG. 18, the server computer 20 uses the storage means 22 as a tooth brushing motion storage means for accumulating the user's tooth brushing motion data. A tooth brushing operation data storage unit 22d is provided, and an analysis processing unit 21e analyzes the problem based on the user's tooth brushing operation data and a plurality of adjusted image data stored in the tooth brushing operation data storage unit 22d. ..

There are various possibilities for accumulating tooth brushing motion data, but in the present embodiment, as shown in FIG. 16, the motion acquisition sensor terminal 30 attached to the tooth brush 3 used by the user and acquiring the tooth brushing motion data is a portable terminal. The tooth brushing operation data is transmitted to the 10 and the mobile terminal 10 is configured to receive the data and transmit the data to the server computer 20. Therefore, as shown in FIG. 17, the mobile terminal 10 has a tooth brushing motion receiving processing unit 12h that receives tooth brushing motion data from the motion acquisition sensor terminal 30 and stores it in the tooth brushing motion data storage section 14d of the storage means 14. Further, a tooth brushing operation data transmission processing unit 12i for transmitting tooth brushing operation data to the server computer 20 is provided.

Such an operation acquisition sensor terminal 30 is the attachment for a toothbrush described in International Publication No. 2016/104443, JP-A-2017-60661, and JP-A-2017-60662, which the applicant has already applied for. As the toothbrushing motion data that can be used and obtained, as described in the same publication, the acceleration data detected by the 3-axis accelerometer that fluctuates with the movement of the toothbrush and the relative that fluctuates with the posture change of the toothbrush. This corresponds to the direction data of various gravitational accelerations, as well as gyro sensor data that detects the rotation angle and angular velocity of the toothbrush, magnetic compass data that detects geomagnetism, GPS data that acquires position information, and hair bundle teeth. The data of the pressure sensor that detects the pressing force, the data of the distance sensor that measures the distance, and the like may be used.

In the present embodiment, as compared with the fourth embodiment described above, the analysis processing unit 21e solves the problem based on the user's tooth brushing operation data and a plurality of adjusted image data stored in the tooth brushing operation data storage unit 22d. It can be analyzed more accurately, and the content of the tooth brushing instruction information generated by the instruction information generation processing unit 21f can also be the instruction content more suitable for the user.

Specifically, when the position where the bristles of the toothbrush is applied is correct but plaque remains, the analysis processing unit 21e may use other factors such as pressing force and operation speed based on the tooth brushing operation data of the user. The operation time and the like can be compared with a preset threshold value as an appropriate value, and the missing parameters can be extracted and output as problems. The instruction content can appropriately reflect the user's usual tooth brushing habits, such as being reflected in the instruction information.

Further, when a music performance or a game linked to a tooth brushing operation is executed on the mobile terminal 10, the server computer 20 has a plurality of stored in the storage means in advance based on the problems analyzed by the analysis processing unit 21. The music data and the result of the game are selected according to the content of the problem, and the data is transmitted to the mobile terminal 10, which is received by the music performance processing unit and the game execution processing unit included in the processing device 12 of the mobile terminal 10. It can be configured to output the music data and the result of the game. As a result, the result of music or game according to the residual status of plaque can be output, and the result of tooth brushing (good or bad) is displayed to the user according to the result of the output music or game as well as the display of the residual status of plaque. It is possible to make a stronger impression.

When playing music or a game in this way, it is particularly preferable that the user images the oral cavity before and after brushing the teeth, and the captured image data and the adjusted image data are the adjusted image data before and after brushing the teeth. As a result, the analysis processing unit 21 can compare the adjusted image data to more accurately analyze the problem of the tooth brushing operation. Further, it is also a preferable embodiment to display an image having a different degree of emphasis of dental plaque as an adjustment image displayed by the image display means according to the problem by the analysis processing unit 21 and the content of the instruction information. ..

Other configurations and modifications are the same as those in the fourth embodiment described above, the same configurations are designated by the same reference numerals, and the description thereof will be omitted.

Although the embodiments of the present invention have been described above, the present invention is not limited to each of these embodiments, and it is needless to say that the present invention can be implemented in various forms without departing from the gist of the present invention.

3 Toothbrush 10 Portable terminal 11 Imaging means 12 Processing device 12a Imaging image storage processing unit 12b Color space conversion processing unit 12c Image adjustment processing unit 12d Adjusted image display processing unit 12e Area detection processing unit 12f Image capture image transmission processing unit 12g Adjusted image reception processing 12h Toothbrushing motion receiving processing section 12i Toothbrushing motion data transmission processing section 13 Image display means 14 Storage means 14a Captured image data storage section 14b Adjusted image data storage section 14c Converted image data storage section 14d Toothbrushing motion data storage section 15 Communication control section 20 Server computer 21 Processing device 21a Captured image reception processing unit 21b Color space conversion processing unit 21c Image adjustment processing unit 21d Adjusted image transmission processing unit 21e Analysis processing unit 21f Guidance information generation processing unit 22 Storage means 22a Captured image data storage unit 22b Adjusted image Data storage unit 22c Converted image Data storage unit 22d Toothbrushing operation data storage unit 23 Communication control unit 30 Operation acquisition sensor terminal 100 Captured image 101 Adjusted image 102 Composite image N Communication network S, S1 to S4 Toothpaste display system

Claims (7)

An imaging means that captures the inside of the oral cavity,
With respect to the captured image data captured by the imaging means, at least the hue value and the saturation value are set according to the magnitude of the hue value or the saturation value in a specific region of the captured image data without extracting the toothpaste portion. An image adjustment means that changes only the amount of change set in advance to create an image with emphasized hue,
An image display means for displaying an image adjusted by the image adjustment means, and an image display means.
Plaque display system with. A color space conversion means for converting the captured image data from the RGB color space into a color space composed of components including hue and saturation is provided.
The plaque display system according to claim 1, wherein the image adjusting means changes the hue value and the saturation value in a color space composed of a component including the converted hue and saturation. A region detecting means for detecting a tooth region from the captured image data is provided.
The image adjusting means is limited to the detected tooth region of the captured image data, and at least the hue value and the saturation value are changed to obtain an adjusted image in which plaque is emphasized. The plaque display system according to 1 or 2 . The plaque display according to claim 3 , wherein the region detecting means detects a region having a white hue surrounded by a region consisting of a red hue as the tooth region based on the hue value of the captured image data. system. An image storage means for storing the adjusted image data adjusted by the image adjustment means, and an image storage means.
An analysis means for analyzing a problem of the user's tooth brushing operation based on a plurality of the adjusted image data of the user stored in the image storage means, and an analysis means.
It is provided with a guidance information generation means for generating tooth brushing guidance information for the user based on the analyzed tooth brushing problem.
The plaque display system according to any one of claims 1 to 4 , wherein the image display means displays the generated tooth brushing instruction information in accordance with the display of the adjustment image. It is equipped with a tooth brushing motion storage means that stores the tooth brushing motion data of the user.
The plaque display system according to claim 5 , wherein the analysis means analyzes the problem based on the tooth brushing motion data of the user and the plurality of adjusted image data stored in the tooth brushing motion storage means. The plaque display system according to claim 5 or 6 , wherein the plurality of adjusted image data are adjusted image data before and after brushing teeth.

Images