WO2021166733A1 - Information processing device, evaluation system, machine learning device, information processing method, and program - Google Patents

Abstract

The objective of the present invention is to provide a technique for inspecting the condition of a fingernail, with which it is possible to determine, simply, whether the fingernail condition is acceptable, without involving a visual observation by a person. An information processing device (20a) is provided with: an acquiring unit (221) for acquiring image data including a fingernail, as a subject; and an evaluating unit (222) for generating information indicating an evaluation result, by evaluating the sanitary state of the fingernail based on the length of the fingernail, with reference to the image data.

Description

Information processing equipment, evaluation system, machine learning equipment, information processing method, and program

The present invention relates to a technique for inspecting the condition of nails.

The nails of human hands continue to grow every day and need to be trimmed regularly. Since dirt tends to accumulate on the inside of the stretched nails, it is required that the nails are always cut to an appropriate length at food manufacturing sites and medical sites. In addition, nails with nail art such as artificial nails and nail polish are not preferable in terms of cleanliness, contamination with foreign substances, and risk of infection in the medical field.

In order to prevent these conditions, enthusiastic guidance has been taken to correct each person's grooming, and to ensure that others double-check when entering or leaving the room. However, this method cannot prevent human error, and when someone else double-checks it, it is necessary to stop the work of the people around it and check it, which is also in terms of work efficiency. There was a problem.

Japanese Patent Application Laid-Open No. 2017-185102

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a technique for inspecting the state of a nail, which can easily determine whether or not the state of the nail is present without visual confirmation by a person. do.

According to the technique described in Patent Document 1, the recommended sports hand protector can be selected based on the measured three-dimensional shape of the hand. However, there has been no technique for easily determining whether or not the nail state is possible without visual confirmation by a person.

In order to solve the above problems, the information processing apparatus according to one aspect of the present invention includes an acquisition unit that acquires image data including a nail as a subject, and the nail based on the length of the nail with reference to the image data. It is configured to include an evaluation unit that generates information indicating the evaluation result by evaluating the hygiene condition.

The information processing device according to each aspect of the present invention may be realized by a computer. In this case, the information processing device is made into a computer by operating the computer as each part (software element) included in the information processing device. The control program of the information processing device to be realized and the computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

According to the present invention, it is possible to provide a technique capable of easily determining whether or not a nail state is present without visual confirmation by a person.

It is a block diagram of the nail state inspection apparatus (information processing apparatus) which concerns on Embodiment 1 of this invention. It is explanatory drawing of the image pickup apparatus which concerns on Embodiment 1 of this invention. This is an example of a schematic configuration of a nail condition inspection device (evaluation system) according to a first embodiment of the present invention. This is an example of a schematic configuration of a nail condition inspection device (evaluation system) according to a first embodiment of the present invention. It is a block diagram which shows the structure of the evaluation system which concerns on Embodiment 2 of this invention. It is a flowchart which shows the flow of the information processing method carried out by the information processing apparatus included in the evaluation system of FIG. It is a schematic diagram explaining an example of the process of the evaluation step shown in the flowchart of FIG. It is a figure which shows an example of the "information which shows the evaluation result" output by the output device on the subject side included in the evaluation system of FIG. It is a figure which shows an example of the "information which shows the evaluation result" output by the output device on the manager side included in the evaluation system of FIG. It is a perspective view which shows the structure of the image pickup apparatus included in the evaluation system of FIG. It is a top view which shows an example of the structure of the guide mechanism provided in the image pickup apparatus of FIG. It is a flowchart which shows the flow of the machine learning method carried out by the machine learning apparatus included in the evaluation system of FIG. It is a block diagram which shows the hardware configuration of the information processing apparatus included in the evaluation system of FIG. It is a block diagram which shows the hardware configuration of the machine learning apparatus included in the evaluation system of FIG. It is a block diagram which shows the structure of the evaluation system which concerns on Embodiment 3 of this invention.

[Embodiment 1]
Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as embodiments) will be described in detail with reference to the accompanying drawings. The same elements are numbered the same throughout the description of the embodiment.

(Structure of Embodiment 1)
As shown in FIG. 1, the claw condition inspection device (evaluation system) 1 according to the first embodiment hardware at least an image pickup device 10, an arithmetic device (information processing device) 20, and an input / output device 30. The arithmetic unit (information processing unit) 20 is provided as a hardware resource, and includes a trained neural network (evaluation model) 21.

The arithmetic unit (information processing device) 20 and the input / output device 30 can be realized by, for example, a computer having a stand-alone configuration (for example, a personal computer (Personal Computer) or the like) or an online configuration in which a server and a computer are combined.

The nail condition inspection device (evaluation system) 1 can be used for determination for all purposes, but in particular, it can be used to determine whether or not a person can enter or leave the room (indoor) R by the above determination. Further, the room (indoor) R is preferably a food manufacturing site or a medical site where a hygienic condition of the hand is required, and a food factory room (indoor) R or a food processing plant R is particularly suitable. ..

The arithmetic unit (information processing unit) 20 includes a trained neural network (evaluation model) 21 learned from the image H of a human hand. The image H of the human hand may be an image obtained by the image pickup device 10 or may be an image obtained by using another image pickup device, but from the viewpoint of the angle of view, color tone, etc., an image pickup having the same configuration as that of the image pickup device 10 may be obtained. The one obtained from the device is preferable. Also, the image of the human hand includes at least the image of the back side of the human hand to inspect the nails.

The trained neural network (evaluation model) 21 can be learned from nail information such as the length of the nail and the color tone of the entire nail and visual judgment result information, whereby the back of the hand of the subject P can be learned. The state of the nail can be determined from the image on the side.

Further, the trained neural network (evaluation model) 21 can determine whether or not there is an deposit on the nail by learning based on the color tone of the entire nail. The deposit refers to, for example, nail art, and includes scalp, nail tip, rhinestone installation, nail polish, gel, and the like.

As the input / output device 30, an output device (output device) such as an audio device such as a display, an indicator light, and a buzzer can be used. Further, an input device such as a mouse, a keyboard, and a touch panel may be provided.

As shown in FIG. 2, the imaging unit (imaging device) 10 according to the present embodiment can have an imaging device 101 installed at the upper part, a bottom surface 103 installed at the lower part, and a light-shielding wall 102. The bottom surface 103 is preferably a single color. The light-shielding wall 102 and the bottom surface 103 can also be configured as a continuous body so that light from the outside does not easily hit the bottom surface 103. Further, as shown in FIG. 10 to be described later, it can also be configured as an imaging box 105 which is a box body formed as a continuous body by adding the top surface 104, and can be configured from other surfaces except one surface to be touched. It can be installed at the entrance / exit of a room (indoor) to prevent the intrusion of light. The subject P can insert the image pickup device with the back of his / her hand facing up and acquire an image.

FIG. 3 shows a configuration example when the nail condition inspection device (evaluation system) 1 has a stand-alone configuration. In this case, the arithmetic unit (information processing device) 20 and the input / output device 30 may be configured as one personal computer. The subject P acquires the image H of the hand (see FIG. 7 described later) by the imaging device 10, and transmits the image H of the hand to the arithmetic unit (information processing device) 20. The arithmetic unit 20 determines the image H of the hand by the trained neural network (evaluation model) 21. The result of the determination may be output to the input / output device 30.

The administrator S may be able to confirm the saved information of the subject P by the personal computer. At that time, information on the physical condition of the subject P, such as body temperature and sighing alcohol information, presence / absence of scratches, which is separately measured or input in association with information uniquely set for the individual such as the employee code of the subject P. Information such as whether or not the hair has come out of the hat may be displayed in a list.

FIG. 4 shows a configuration example when the entrance / exit inspection device (evaluation system) 1 has an online configuration. In this case, the subject P and the manager S may be able to use separate input / output devices 30. The subject P acquires the image H of the hand (see FIG. 7 described later) by the imaging device 10, and transmits the image H of the hand to the arithmetic unit (information processing device) 20. The arithmetic unit (information processing unit) 20 determines the image H of the hand by the trained neural network (evaluation model) 21. The result of the determination may be output to the input / output device 30 on the subject side and / or the administrator side.

The administrator S may be able to confirm the saved information of the subject P by using the input / output device on the administrator side. At that time, information on the physical condition of the subject P, which is separately measured or input in association with information uniquely set for the individual such as the employee code of the subject P, such as body temperature and sighing alcohol information, and the presence or absence of scratches. , Information such as whether or not the hair has come out of the hat may be displayed in a list.

[Embodiment 2]
Other embodiments of the present invention will be described below. For convenience of explanation, the members having the same functions as the members described in the first embodiment are designated by the same reference numerals, and the description thereof will not be repeated.

(Configuration of evaluation system 1a)
The evaluation system 1a according to the second embodiment of the present invention will be described with reference to FIG. FIG. 5 is a diagram showing the configuration of the evaluation system 1a.

The evaluation system 1a is a system for evaluating the hygiene condition based on the length of the nail. As shown in FIG. 5, the evaluation system 1a includes an image pickup device 10a, an information processing device 20a, an output device 30a, and a machine learning device 40. The input / output device 30 is an example of the output device described in the claims. The information processing device 20a is communicably connected to each of the image pickup device 10a, the machine learning device 40, and the output device 30a.

The image pickup device 10a is a device for imaging a part of the human body including the nail and generating image data including the nail as a subject (hereinafter, may be simply referred to as "image data"). The image pickup device 10a is at least different from the image pickup device 10 in the first embodiment in that the image pickup device 10a is provided with the image pickup box 105. The details of the configuration of the image pickup apparatus 10a will be described later instead of the reference drawings. The output device 30a is an output device included in the input / output device 30 according to the first embodiment.

The information processing device 20a uses the first evaluation model 21a and the second evaluation model 21b instead of the trained neural network 21 (FIG. 1), which is different from the arithmetic unit (information processing device) 20 in the first embodiment. At least not. The information processing device 20a is a device for implementing the information processing method M1. The information processing method M1 is a method of evaluating the sanitary state of the nail using the first evaluation model 21a and the second evaluation model 21b based on the image data provided by the image pickup apparatus 10a. The configuration of the information processing device 20a and the details of the flow of the information processing method M1 will be described later instead of the reference drawings.

Here, as the first evaluation model 21a and the second evaluation model 21b, machine learning algorithms using neural networks such as CNN (Convolution Neural Network) and GAN (Generative Adversarial Network) are used, respectively. Can be done. Further, as the first evaluation model 21a and the second evaluation model 21b, machine learning algorithms that do not use a neural network, such as a support vector machine, a random forest, and a gradient boosting tree, can be used. However, the algorithms that realize the first evaluation model 21a and the second evaluation model 21b are not limited to those described above. Further, the algorithms for realizing the first evaluation model 21a and the second evaluation model 21b may be the same or different.

The first evaluation model 21a is constructed by machine learning so as to input image data and output first information indicating the evaluation result of the sanitary condition based on the length of the nail. Here, the "nail length" is, for example, the length of the toe. The "toe" is the part that appears white from the nail bed, and the length of the toe corresponds to the length from the yellow line of the nail to the tip of the nail. The "yellow line" is the boundary between the nail plate and the tip of the nail, the "nail plate" is the area of the nail above the nail plate, and the "nail plate" is the subcutaneous tissue under the nail plate. Is. However, the first evaluation model 21a is not machine-learned using the nail length (for example, the above-mentioned toe length) itself as correct answer data. The first evaluation model 21a is constructed by machine learning using the evaluation result in which the evaluator evaluates the sanitation state from the viewpoint of the length of the nail (for example, the length of the toe described above) as correct answer data. The "nail length" is not limited to the above-mentioned toe length. For example, the "nail length" may be the length of the entire nail or the ratio of the length of the tip of the nail to the length of the entire nail. In addition, "nail length" is a comprehensive viewpoint including the length of the toe, the length of the entire nail, the ratio of the length of the toe to the total length of the nail, and some or all of the other viewpoints. It may be. In this case, the first evaluation model 21a is constructed by machine learning using the evaluation result in which the evaluator evaluates the sanitary condition from the comprehensive viewpoint of the length of the nail as correct answer data. The method of constructing the first evaluation model 21a by machine learning will be described later.

The second evaluation model 21b is constructed by machine learning so as to input image data and output second information indicating the evaluation result of the sanitary condition based on the color of the nail. Here, the "nail color" is the color tone, hue, etc. of the nail. The "nail color" may be at least a part of the color of the nail, but is preferably the color of the entire nail from the viewpoint of hygiene. The method of constructing the second evaluation model 21b by machine learning will be described later.

The image data including the nail as the subject, which is the input of the first evaluation model 21a, and the sanitary condition based on the length of the nail, which is the output of the first evaluation model 21a, are explicitly specified as a relational expression. Although difficult, it is known that there is a certain relationship. Similarly, there is a certain relationship between the image data including the nail as the subject, which is the input of the second evaluation model 21b, and the sanitary condition based on the color of the nail, which is the output of the second evaluation model 21b. It has been known. Therefore, by using the first evaluation model 21a that inputs image data including the nail as a subject, it is possible to accurately evaluate the sanitary state based on the length of the nail. Further, by using the second evaluation model 21b in which the image data including the nail as a subject is input, the sanitary state based on the color of the nail can be evaluated with high accuracy.

The machine learning device 40 is a device for implementing the machine learning method M2. The machine learning method M2 creates a learning data set using image data for learning, and constructs a first evaluation model 21a and a second evaluation model 21b by machine learning using the learning data data set. Is the way to. The configuration of the machine learning device 40 and the details of the flow of the machine learning method M2 will be described later instead of the drawings to be referred to.

The evaluation system 1a reaches the practical phase through the preparation phase and the trial phase. The contents of the preparation phase, trial phase, and practical phase are as follows.

(1) Preparation Phase The preparation phase is a phase in which the machine learning device 40 constructs the first evaluation model 21a and the second evaluation model 21b. In the preparation phase, the evaluator performs an evaluation based on the length of the nail and an evaluation based on the color of the nail for the nail included as a subject in each of the plurality of image data for learning. The machine learning device 40 generates a learning data set consisting of teacher data in which the evaluation results by the evaluator are labeled with respect to the image data for each learning, and the first evaluation is performed by machine learning using the learning data set. A model 21a and a second evaluation model 21b are constructed. The constructed first evaluation model 21a and second evaluation model 21b are transferred from the machine learning device 40 to the information processing device 20a.

(2) Trial Phase In the trial phase, the evaluation system 1a is installed at the entrance indoors (for example, a room such as a food factory room). When an operator who should enter the room inserts the fingernail of the hand into the image pickup box 105 (see FIG. 10 described later) of the image pickup device 10a, the information processing device 20a acquires image data including the nail as a subject from the image pickup device 10a. Then, the hygiene condition of the nail is evaluated. In the trial phase, the evaluator performs an evaluation based on the length of the nail and an evaluation based on the color of the nail with respect to the image data evaluated by the information processing apparatus 20a. The evaluation by the evaluator may be performed every time the evaluation by the information processing device 20a is performed, or the evaluation by the information processing device 20a is performed for a predetermined period (for example, one week, one month, or one year) or a predetermined time. It may be carried out a number of times (for example, 100 times, 1000 times, or 10000 times) and then collectively. The evaluator compares the evaluation result by the evaluator with the evaluation result by the information processing device 20a, and evaluates the evaluation accuracy of the information processing device 20a. If the evaluation accuracy is insufficient, the process returns to the preparation phase. If the evaluation accuracy is sufficient, the process proceeds to the practical phase. The evaluation accuracy may be confirmed by using a part of the learning data set at the stage when the preparation phase is completed. In this case, the trial phase can be omitted.

(3) Practical phase In the practical phase, the information processing device 20a evaluates the hygiene condition of the nail based on the length and color of the nail. The first evaluation model 21a and the second evaluation model 21b used by the information processing apparatus 20a in the practical phase have been confirmed to have sufficient estimation accuracy in the trial phase. In the practical phase, when a subject enters a food factory room, visual evaluation of hygiene based on nail length and color can be omitted. For this reason, the subject is freed from the trouble of evaluating the hygiene condition based on the length and color of the nail, and the hygiene condition based on the length and color of the nail is easily and accurately obtained without visual confirmation by a person. It becomes possible to evaluate.

(Configuration of information processing device 20a)
The configuration of the information processing device 20a will be described with reference to FIG. The information processing device 20a includes a control unit 22 and a storage unit 23. The control unit 22 comprehensively controls the information processing device 20a. In the information processing device 20a, the control unit 22 includes an acquisition unit 221 and an evaluation unit 222. The acquisition unit 221 executes the acquisition step M11 of the information processing method M1. The evaluation unit 222 executes the evaluation step M12 and the output step M13 of the information processing method M1. The details of the flow of the information processing method M1 will be described later. The first evaluation model 21a and the second evaluation model 21b are stored in the storage unit 23.

(Flow of information processing method M1)
The flow of the information processing method M1 will be described with reference to FIG. FIG. 6 is a flowchart showing the flow of the information processing method M1.

The information processing method M1 includes an acquisition step M11, an evaluation step M12, and an output step M13.

(Acquisition step M11)
The acquisition step M11 is a step in which the acquisition unit 221 acquires image data including the nail as a subject. In the acquisition step M11, the acquisition unit 221 acquires image data including the nail generated by the image pickup apparatus 10a as a subject.

(Evaluation step M12)
The evaluation step M12 is a step in which the evaluation unit 222 evaluates the hygiene condition based on the length and color of the nail using the first evaluation model 21a and the second evaluation model 21b. In the evaluation step M12, the evaluation unit 222 evaluates the hygiene state of the nail based on the region of at least two fingernails included in the image data acquired by the acquisition unit 221. Specifically, the evaluation unit 222 inputs the image data including the nail acquired by the acquisition unit 221 as the subject into the first evaluation model 21a, and acquires the first information output from the first evaluation model 21a. .. Further, the evaluation unit 222 inputs the image data including the nail acquired by the acquisition unit 221 as the subject into the second evaluation model 21b, and acquires the second information output from the second evaluation model 21b. Further, the evaluation unit 222 generates information indicating the evaluation result based on the first information and the second information. The evaluation unit 222 writes information indicating the evaluation result in the storage unit 23.

FIG. 7 is a schematic diagram illustrating an example of the process of the evaluation step M12. For example, the evaluation step M12 includes a partial image data extraction process, a first evaluation process, a second evaluation process, and an information generation process indicating an evaluation result. These processes will be described with reference to FIG. 7.

(Extraction process of partial image data)
The evaluation unit 222 extracts the "nail" region of each of the five fingers from the image data H including the nail acquired by the acquisition unit 221 as the subject, and partially image data IMG-1, IMG-2, IMG-3, and IMG. -4, Extract as IMG-5. The partial image data IMG-1 is the area of the thumb claw, the partial image data IMG-2 is the area of the index finger claw, the partial image data IMG-3 is the area of the middle finger claw, and the partial image data IMG- Reference numeral 4 denotes a region of the ring finger claw, and partial image data IMG-5 is a region of the little finger claw. Hereinafter, when it is not necessary to distinguish each of the partial image data IMG1 to IMG5, the partial image data IMG-i (i = 1, 2, 3, 4, 5) is also simply described. In other words, the partial image data IMG-i includes one nail as a subject.

A known object detection algorithm can be used as a method for extracting the nail region from the image data. Examples of such an object detection algorithm include, but are not limited to, YOLO V3 (You Only Look Once V3), MASK R-CNN (MASK Regions with CNN features), and the like.

Further, the number of partial image data IMG-i extracted from the image data H by the evaluation unit 222 is not limited to 5, but is preferably a plurality. For example, the evaluation unit 222 continues the process when n (n is an integer of 2 or more) or more partial image data IMG-i is extracted, and when less than n partial image data IMG-i are extracted. May interrupt the process and output error information. For example, by setting n to 2 or more and less than 5, if all five nails are not included in the image data H due to the position of the hand at the time of shooting, or the direction of the finger at the time of shooting, etc. Therefore, it is possible to flexibly deal with the case where some of the nails are not clearly included in the image data H.

(First evaluation process)
The evaluation unit 222 inputs each of the extracted n partial image data IMG-i into the first evaluation model 21a. As a result, the first score is output from the first evaluation model 21a. Here, the first score is an example of the first information showing the evaluation result of the hygiene state based on the length of the nail. The first score is a numerical value in the range of 0 or more and 1 or less, and indicates that the hygiene condition based on the length of the nail may be good. That is, the closer the first score is to 1, the better the hygiene condition based on the length of the nail is, and the closer it is to 0, the less likely it is to be good. If the average value of the n first scores obtained from the n partial image data IMG-i is equal to or higher than a predetermined threshold value, the evaluation unit 222 determines the hygiene condition based on the nail length of the subject P. Evaluate as "pass". On the other hand, if the average value of the first score is less than a predetermined threshold value, the evaluation unit 222 evaluates the hygiene condition based on the claw length of the subject P as "failed". The evaluation unit 222 stores in the storage unit 23 information indicating the pass / fail of the nail length in association with the identification information (employee code, etc.) of the subject P.

In the first evaluation process, the method of determining the pass / fail of the hygiene condition based on the length of the claws of the subject P by referring to the n first scores is not limited to the above. For example, the evaluation unit 222 may make a determination based on whether or not various statistical values such as the maximum value, the minimum value, and the median value of n first scores are equal to or more than the threshold value. For example, when the minimum value is used, in other words, if there is even one claw whose claw length is not good, it is determined to be rejected. Further, for example, when the maximum value is used, in other words, if there is even one claw having a good claw length, it is determined to pass.

(Second evaluation process)
The evaluation unit 222 inputs each of the extracted n partial image data IMG-i into the second evaluation model 21b. As a result, the second score is output from the second evaluation model 21b. Here, the second score is an example of the second information showing the evaluation result of the hygiene state based on the color of the nail. The second score is a numerical value in the range of 0 or more and 1 or less, and indicates that the hygiene condition based on the color of the nail may be good. That is, the closer the second score is to 1, the better the hygiene condition based on the color of the nail is, and the closer it is to 0, the less likely it is to be good. If the average value of the n second scores obtained from the n partial image data IMG-i is equal to or higher than a predetermined threshold value, the evaluation unit 222 determines the hygiene condition based on the nail color of the subject P. Evaluate as "passed". On the other hand, if the average value of the second score is less than a predetermined threshold value, the evaluation unit 222 evaluates the hygiene condition based on the nail color of the subject P as "failed". The evaluation unit 222 stores the information indicating the pass / fail of the nail color in the storage unit 23 in association with the identification information of the subject.

In the second evaluation process, the method of determining the pass / fail of the hygiene condition based on the color of the nail of the subject P by referring to the n second scores is not limited to the above. For example, the evaluation unit 222 may make a determination based on whether or not various statistical values such as the maximum value, the minimum value, and the median value of n second scores are equal to or more than the threshold value. For example, when the minimum value is used, in other words, if there is even one nail whose nail color is not good, it is determined to be rejected. Further, for example, when the maximum value is used, in other words, if there is even one nail having a good nail color, it is determined to pass. Further, the execution order of the first evaluation process and the second evaluation process is in no particular order. In other words, the evaluation unit 222 may perform the second evaluation process after the first evaluation process, or may perform the first evaluation process after the second evaluation process, and these The evaluation process may be performed in parallel.

(Generation process of information showing evaluation result)
The evaluation unit 222 generates information indicating whether or not to enter or leave the room based on the first information and the second information. For example, the evaluation unit 222 has (i) the number n of the extracted partial images n is 2 or more, and (ii) the evaluation result of the hygiene condition based on the length of the nail and the evaluation result of the hygiene condition based on the color of the nail. If both of the above are "passed", the subject P generates information indicating that he / she can enter and leave the room. In addition, the evaluation unit 222 generates information indicating that the subject P cannot enter or leave the room when neither (i) nor (ii) is satisfied. The evaluation unit 222 stores in the storage unit 23 information indicating whether or not to enter or leave the subject P in association with the identification information of the subject P. The evaluation unit 222 associates the identification information of the subject P with the evaluation date and time when the evaluation step M12 is executed, and records the claw length as a record uniquely identified by the identification information of the subject P and the evaluation date and time. Information indicating pass / fail, pass / fail of nail color, and pass / fail of entry / exit may be stored. Here, "indoor" is the interior of any building, for example, any room in the building. The “building” is not particularly limited, but is preferably a building at a food manufacturing site (for example, a food processing factory) or a building at a medical site (for example, a hospital) where hygiene of hands is required. ..

From the viewpoint of hygiene, the evaluation unit 222 preferably evaluates the hygiene condition of both hands of the subject P based on the length and color of the nails. The image data of both hands of the subject P may be one image data, or may be individual image data acquired by imaging each hand.

From a hygienic point of view, the evaluation unit 222 has a hygienic condition based on the length and color of the nail based on the area of at least two fingernails (partial image data IMG-i) included in the image data. It is preferable to evaluate. Of the partial image data of the nail recognized as one nail by the object detection algorithm, the partial image data of two or more nails may be used as the input data, and the partial image data of which fingernail is selected as the input data. There is no particular limitation as to whether or not to do so. When evaluating the sanitary condition of both hands of the subject P based on the length and color of the nails, partial image data of two or more nails per one hand may be used as input data.

The output step M13 is a step of outputting information indicating the evaluation result of the hygiene condition of the nail evaluated in the evaluation step M12. In the output step M13, the information processing device 20a reads the information indicating the evaluation result from the storage unit 23 and outputs the information to the output device 30a, thereby presenting the information to the user.

FIG. 8 is a diagram showing an example of a screen output by the output device 30a included in the evaluation system 1a of FIG. The screen example 500 shown in FIG. 8 is typically a suitable screen example when the output device 30a is visibly installed by the subject P. Screen example 500 includes information 501 indicating whether or not each subject can enter or leave. The information 501 indicating whether or not entry / exit is possible is output by reading the information stored in the storage unit 23 by the evaluation unit 222 in the evaluation step M12. By visually recognizing the screen example 500, the subject P can easily confirm whether or not to enter or leave the room. The screen example 500 may include other additional information in addition to the information 501 indicating whether or not to enter the room. For example, when the screen example 500 includes information 501 indicating that entry into the room is not possible, the reason why it is evaluated as impossible (for example, the length of the nail is rejected, the color of the nail is rejected, etc.) is also output. May be good.

FIG. 9 is a diagram showing an example of a screen output by the output device 30a included in the evaluation system 1a of FIG. The screen example 600 shown in FIG. 9 is typically a screen example suitable when the output device 30a is visibly installed by the administrator S. Screen example 600 includes information 601 showing the evaluation results for each subject. The information 601 indicating the evaluation result includes the employee code which is the identification information of the subject P, the evaluation date and time, the pass / fail of the nail length, the pass / fail of the nail color, and the information indicating whether or not to enter or leave. .. As for these information, the information stored in the storage unit 23 by the evaluation unit 222 in the evaluation step M12 is read and output. By visually recognizing the screen example 601, the manager S can confirm a list of information indicating the evaluation result of the nail hygiene state of each subject P.

(Configuration of Imaging Device 10a)
The configuration of the image pickup apparatus 10a will be described with reference to FIG. FIG. 10 is a perspective view showing the configuration of the image pickup apparatus 10a included in the evaluation system 1a of FIG. The image pickup apparatus 10a includes an image pickup box 105. In the image pickup apparatus 10a, the image pickup apparatus 101 is installed inside the image pickup box 105. The image pickup box 105 has a bottom surface 103, a side surface 102 as a light-shielding wall, and a top surface 104 on which the image pickup device 101 is installed. The image pickup apparatus 10a is arranged in the three-dimensional coordinate space shown in FIG. The bottom surface 103 of the image pickup box 105 of the image pickup apparatus 10a is arranged on the xy plane, and the direction toward the zenith among the normal directions orthogonal to the xy plane is defined as the z-axis direction.

One of the four side surfaces 102 of the imaging box 105 has an opening 107 inside the imaging box 105 into which a part of the human body including the claw of the subject P can be inserted. The side surface having the opening 107 is particularly referred to as a front surface 106.

The front surface 106 is made of a translucent material so that the inside of the image pickup box 105 can be visually recognized from the outside of the image pickup box 105 and the irradiation of light inside the image pickup box 105 is constant. As a result, the subject P can insert the subject P's hand into the imaging box 105 while visually checking the inside of the imaging box 105 from the outside of the imaging box 105. The material forming the front surface 106 is not particularly limited as long as it is a translucent material. For example, it can be formed from a translucent material such as resin or glass. Here, "semi-transparent" means that the light environment inside the imaging box 105 is less likely to be interfered with by external light during imaging, and the subject P has his or her own hand inside the imaging box 105 from the outside of the imaging box 105. It suffices if it is transparent enough to be visually recognized.

The size of the opening 107 is not particularly limited, but the hand of the subject P can be inserted into the image pickup box 105 from the viewpoint of preventing light from the outside of the image pickup box 105 from entering the inside of the image pickup box 105. It is preferable that the size is the smallest in the above range. Further, it is preferable that the opening 107 is provided on the lower side of the front surface 106 so that the hand inserted inside the image pickup box 105 can be easily placed on the bottom surface 103.

The bottom surface 103 functions as an alignment table that enables horizontal alignment of the subject P's hand with respect to the imaging device 101. The imaging device 101 is installed so as to include the bottom surface 103 of the imaging box 105 as an alignment table in the imaging range. The bottom surface 103 preferably has a single color on the surface facing the imaging device 101, and is preferably a color having a large contrast with the color of the subject P's hand, such as black or blue. By selecting a color having a large contrast with the color of the hand of the subject P as the color of the bottom surface 103, the information processing device 20a recognizes the nail in the "image data including the nail as the subject" generated by the imaging device 101. The accuracy can be improved.

It is preferable that the imaging device 10a is provided with a guide mechanism that enables an image of a part of the human body including the nails in a state where adjacent nails are separated from each other. FIG. 11 is a top view showing an example of the configuration of the guide mechanism included in the imaging device 10a of FIG. 10, and 1101 is a configuration in which the bottom surface 103a of the imaging box 105 includes a plurality of protrusions 109a as a guide mechanism. (Modification 1) is described, and 1102 describes a configuration in which the bottom surface 103b of the imaging box 105 includes a hand-shaped guide line 109b as a guide mechanism (Modification 2).

In the first modification, the plurality of protrusions 109a as the guide mechanism protrude from the bottom surface 103a of the image pickup box 105 in the positive direction of the Z axis. The plurality of protrusions 109a are arranged on the bottom surface 103a at predetermined intervals. After inserting the hand into the imaging box 105, the subject P may place the hand on the bottom surface 103a so that the protrusions 109a are located between the fingers. By providing the imaging device 10a with a plurality of protrusions 109a as a guide mechanism, the subject P can be guided so that the subject P can place his / her hand on the bottom surface 103a in a state suitable for photographing. It is possible to photograph parts of the human body including the nails while they are separated from each other. As a result, in the "image data including the nail as a subject" generated by the imaging device 101, the recognition accuracy of the nail by the information processing device 20a can be improved.

When the guide mechanism is a hand-shaped guide line 109b as in the second modification, the subject P inserts his / her hand into the imaging box 105 and then opens it so as to match the shape of the guide line 109b. The hand may be placed on the bottom surface 103b in this state. In the second modification as well, as in the first modification, the subject P can be guided so that the subject P can place his / her hand on the bottom surface 103b in a state suitable for photographing, so that the adjacent nails are separated from each other. It is possible to photograph parts of the human body including the nails. As a result, in the "image data including the nail as a subject" generated by the imaging device 101, the recognition accuracy of the nail by the information processing device 20a can be improved. Since the front surface 106 is made of a translucent material, the subject P visually confirms the position of the hand in the imaging box 105 through the front surface 106 and places the hand at an appropriate position on the bottom surface 103b. Can be placed.

The configuration of the guide mechanism provided in the image pickup apparatus 10a is not limited to the above-mentioned modification. For example, the bottom surface 103 may be uneven, a rod-shaped object may be placed on a part of the surface of the bottom surface 103, or a pattern using a plurality of colors may be used. Further, for example, in the "image data including a nail as a subject" generated by the imaging device 10a, the degree of opening of the fingers of the subject P's hand is not sufficient, and the information processing device 20a recognizes it as one nail. If this is not possible, the information processing device 20a causes the output device 30a to display a message prompting the subject P to reposition his / her hand on the bottom surface 103 with the adjacent fingers further separated from each other. It may be configured as.

The image pickup apparatus 10a includes a plurality of LEDs (light emission diodes) 108 as illumination inside the image pickup box 105. This makes it possible to irradiate the entire surface of the bottom surface 103 with uniform light. The intensity of the light emitted by the plurality of LEDs 108 may be appropriately adjusted so that the reflection of the light applied to the nails does not become too strong and does not affect the evaluation accuracy of the sanitary condition based on the color of the nails.

(Configuration of machine learning device 40)
The configuration of the machine learning device 40 will be described with reference to FIG. The machine learning device 40 includes a storage unit 41 and a control unit 42. The control unit 42 comprehensively controls the machine learning device 40. In the machine learning device 40, the control unit 42 includes the construction unit 421. The construction unit 421 executes the machine learning method M2.

(Flow of machine learning method M2)
The flow of the machine learning method M2 will be described with reference to FIG. FIG. 12 is a flowchart showing the flow of the machine learning method M2.

The machine learning method M2 includes a learning data set construction step M21 and a trained model construction step M22.

(Training data set construction step M21)
The learning data set construction step M21 is a step in which the processor 401 creates a first learning data set and a second learning data set.

(Creation process of the first learning data set)
The first learning data set is a set of a plurality of first learning data (teacher data) for constructing the first evaluation model 21a.

Each first learning data includes teacher image data including one nail as a subject and correct answer information. The teacher image data may be captured by the imaging device 10a or may be captured by another imaging device. Further, the teacher image data may be partial image data extracted from image data including a plurality of nails as subjects. Further, the teacher image data may be public image data published on the network, or may be partial image data extracted from the public image data. The correct answer information is information indicating the evaluation result of the nail length by the evaluator.

In the learning data set construction step M21, the construction unit 421 acquires the teacher image data and displays it on the display device (not shown). In addition, the construction unit 421 acquires correct answer information indicating the evaluation result of the hygiene condition based on the length of the nail by the evaluator via an input device (not shown). For example, the evaluator may visually recognize the teacher image data displayed on the display device and input the evaluation result of "pass" or "fail" from the input device. Further, the evaluator may visually recognize the teacher image data displayed on the display device and input the sanitation score from the input device.

Further, the construction unit 421 generates the first learning data including the teacher image data and the correct answer information. Further, the construction unit 421 stores the generated first learning data in the storage unit 41. The first learning data set is created by repeating the above process for the plurality of teacher image data by the construction unit 421.

(Creation process of the second training data set)
The second learning data set is a set of a plurality of second learning data (teacher data) for constructing the second evaluation model 21b.

Each second learning data includes teacher image data including one nail as a subject and correct answer information. The teacher image data is as described in the first learning data set creation process. Note that part or all of the teacher data used in the process of creating the second learning data set may be the same as that used in the process of creating the first learning data set. The correct answer information is information indicating the evaluation result of the nail color by the evaluator.

In the learning data set construction step M21, the construction unit 421 acquires the teacher image data and displays it on the display device (not shown). In addition, the construction unit 421 acquires correct answer information indicating the evaluation result of the hygiene condition based on the color of the nail by the evaluator via an input device (not shown). A specific example of the method for inputting correct answer information by the evaluator is as described in the first process of creating the learning data set.

Further, the construction unit 421 generates the second learning data including the teacher image data and the correct answer information. Further, the construction unit 421 stores the generated second learning data in the storage unit 41. The construction unit 421 repeats the above process for a plurality of teacher image data, thereby creating a second learning data set.

(Trained model construction step M22)
The trained model construction step M22 is a step in which the construction unit 421 constructs the first evaluation model 21a and the second evaluation model 21b. In the trained model construction step M22, the construction unit 421 constructs the first evaluation model 21a by supervised learning using the first learning data set. In addition, the construction unit 421 constructs the second evaluation model 21b by supervised learning using the second learning data set. The constructed first evaluation model 21a and second evaluation model 21b are stored in the storage unit 41.

(Other Aspect 1 of Embodiment 2)
As the second embodiment, the embodiment in which the number of evaluation models used by the evaluation unit 222 for evaluating the hygiene condition of the nail is two has been described. However, the number of evaluation models used by the evaluation unit 222 is not limited to two. For example, the evaluation unit 222 may evaluate the hygiene state of the nail using either the first evaluation model 21a or the second evaluation model 21b. Further, the evaluation unit 222 may evaluate the hygiene state of the nail by using another evaluation model in addition to the first evaluation model 21a and the second evaluation model 21b. In this case, the other evaluation model is constructed by machine learning so as to input image data including the nail as a subject and output information indicating the evaluation result of the sanitation state based on another viewpoint of the nail.

(Other Aspect 2 of Embodiment 2)
As the second embodiment, the aspect in which the input of the first evaluation model 21a and the second evaluation model 21b is the partial image data IMG-i including one nail as a subject has been described. However, the input of the first evaluation model 21a and the second evaluation model 21b may be image data H including two or more fingernails as subjects.

In this case, the construction unit 421 includes first learning including image data for learning including a plurality of claws as subjects and correct answer information indicating an evaluation result in which the evaluator evaluates the sanitation state based on the lengths of the plurality of claws. Generate data for. As a result, the first evaluation model 21a is machine-learned using the first learning data set including such first learning data. In addition, the construction unit 421 includes second learning data including image data for learning including a plurality of nails as subjects and correct answer information indicating an evaluation result in which an evaluator evaluates a sanitary condition based on the colors of the plurality of nails. To generate. As a result, the second evaluation model 21b is machine-learned using the second learning data set including such second learning data.

Further, in this case, the evaluation unit 222 omits the partial image data extraction process in the evaluation step M12, and uses the image data H acquired by the acquisition unit 221 as the first evaluation model 21a and the second evaluation model 21b, respectively. You may enter in.

When the input of the first evaluation model 21a and the second evaluation model 21b is the partial image data IMG-i, the first evaluation model 21a and the second evaluation are evaluated using a relatively small amount of image data for learning. It is possible to construct the model 21b with high accuracy. On the other hand, when the input of the first evaluation model 21a and the second evaluation model 21b is the image data H, there is an advantage that the processing cost of the evaluation unit 222 can be reduced.

(Other Aspect 3 of Embodiment 2)
In the second embodiment, the evaluation system 1a may include a subject terminal, a subject-side output device, an administrator terminal, and an administrator-side output device instead of the output device 30a. In this case, the information processing device 20a is communicably connected to each of the subject terminal and the administrator terminal via the network. For example, the subject terminal, the subject-side output device, and the image pickup device 10a are installed near the entrance of the food factory. Further, for example, the administrator terminal and the output device on the administrator side are installed in the office of the administrator.

In this embodiment, in the output step M13, the evaluation unit 222 outputs the information for the subject as shown in FIG. 8 to the output device on the subject side, and outputs the information for the administrator as shown in FIG. It may be output to the output device on the administrator side. Further, the evaluation unit 222 executes the process of outputting the information for the administrator to the output device on the administrator side instead of or in addition to executing the process in the output step M13 in response to the request from the administrator. May be good. The request by the administrator is input via an input device (not shown) of the administrator terminal.

(Other Aspect 4 of Embodiment 2)
As the second embodiment, the mode in which the first evaluation model 21a and the second evaluation model 21b are stored in the storage unit 23 of the information processing device 20a has been described. However, the information processing device 20a does not have to include the first evaluation model 21a and the second evaluation model 21b. For example, the first evaluation model 21a and the second evaluation model 21b may be built in another computer (for example, a computer constituting a cloud server) connected to the information processing device 20a via a network. In this case, the evaluation unit 222 of the information processing device 20a uses the first evaluation model 21a and the second evaluation model 21b via the network to generate an evaluation result of the sanitary condition based on the length and color of the nail. do it.

(Effect of Embodiment 2)
In this embodiment, since image data is input and the hygiene state of the nail is evaluated using AI (Artificial Intelligence), image analysis is not required, and the hygiene state of the nail is evaluated based on the analysis result. It has the advantage that it is not necessary to set up evaluation rules in advance. This can be said to have the same effect as the evaluation system 1 of the first embodiment using AI. Further, in the present embodiment, the evaluation unit 222 evaluates the hygiene condition based on the length and color of the nail, so that the hygiene condition of the nail is evaluated as compared with the case of evaluating the hygiene condition based only on the length of the nail. It is possible to evaluate with high accuracy and from various angles. Further, in the present embodiment, since the evaluation unit 222 uses two types of evaluation models, the first evaluation model 21a and the second evaluation model 21b, the length and color of the nail are used by using one type of evaluation model. Compared with the case of evaluating the sanitary condition based on the above, the sanitary condition of the nail can be evaluated more accurately.

(Example of hardware configuration of information processing device 20a)
The information processing device 20a may have a hardware configuration as shown in FIG. 13, for example. FIG. 13 is a block diagram showing an example of the hardware configuration of the information processing apparatus 20a.

The information processing device 20a is realized by using a general-purpose computer, and includes a processor 201, a primary memory 202, a secondary memory 203, an input / output interface 204, a communication interface 205, and a bus 206. The processor 201, the primary memory 202, the secondary memory 203, the input / output interface 204, and the communication interface 205 are connected to each other via the bus 206.

The information processing program and one or more evaluation models are stored in the secondary memory 203. The secondary memory 203 is an example of a configuration that realizes the storage unit 23. The processor 201 develops the information processing program and one or more evaluation models stored in the secondary memory 203 on the primary memory 202. Then, the processor 201 functions as the control unit 22 by executing each step included in the information processing method M1 according to the instructions included in the information processing program expanded on the primary memory 202. One or a plurality of evaluation models (for example, the first evaluation model 21a and the second evaluation model 21b) developed on the primary memory 202 are used when the processor 201 executes the evaluation step M12 of the information processing method M1. Will be done. The fact that the information processing program is stored in the secondary memory 203 means that the source code or the execution format file obtained by compiling the source code is stored in the secondary memory 203. Further, the fact that one or more evaluation models are stored in the secondary memory 203 means that the parameters defining the one or more evaluation models are stored in the secondary memory 203.

Devices that can be used as the processor 201 include, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), and a PPU. (Physics Processing Unit), a microcontroller, or a combination thereof.

Further, as a device that can be used as the primary memory 202, for example, RAM (Random Access Memory) can be mentioned. The primary memory 202 is sometimes referred to as the "main storage device". Examples of the device that can be used as the secondary memory 203 include a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an ODD (Optical Disk Drive), or a combination thereof. .. The secondary memory 203 is sometimes called an "auxiliary storage device". The secondary memory 203 may be built in the information processing device 20a, or constitutes another computer (for example, a cloud server) connected to the information processing device 20a via the input / output interface 204 or the communication interface 205. It may be built into the computer. In the present embodiment, the storage in the information processing apparatus 20a is realized by two memories (primary memory 202 and secondary memory 203), but the present invention is not limited to this. That is, the memory in the information processing device 20a may be realized by one memory. In this case, for example, a certain storage area of the memory may be used as the primary memory 202, and the other storage area of the memory may be used as the secondary memory 203.

An input device and / or an output device is connected to the input / output interface 204. Examples of the input / output interface 204 include interfaces such as USB (Universal Serial Bus), ATA (Advanced Technology Attachment), SCSI (Small Computer System Interface), and PCI (Peripheral Component Interconnect). Examples of the input device connected to the input / output interface 204 include an image pickup device 10a. The image data acquired from the image pickup apparatus 10a in the information processing method M1 is stored in the primary memory 202. Examples of the input device connected to the input / output interface 204 include a keyboard, a mouse, a touch pad, a microphone, or a combination thereof. Examples of the output device connected to the input / output interface 204 include an output device 30a (display, projector, printer, speaker, headphones, or a combination thereof). The information provided to the user in the information processing method M1 is output from the information processing device 20a via these output devices. The information processing device 20a may include a keyboard that functions as an input device and a display that functions as an output device, such as a laptop computer. Alternatively, the information processing device 20a may have a built-in touch panel that functions as both an input device and an output device, such as a tablet computer.

Another computer is connected to the communication interface 205 by wire or wirelessly via a network. Examples of the communication interface 205 include interfaces such as Ethernet (registered trademark) and Wi-Fi (registered trademark). Available networks include, for example, PAN (Personal Area Network), LAN (Local Area Network), CAN (Campus Area Network), MAN (Metropolitan Area Network), WAN (Wide Area Network), and GAN (Global Area Network). , Or an interconnect network including these networks. Internetwork may be an intranet, an extranet, or the Internet. In the information processing method M1, the information processing device 20a acquires data (for example, one or more evaluation models) from another computer (for example, the machine learning device 40), and in the information processing method M1, the information processing device 20a is another. The data provided to the computer is transmitted and received via these networks.

Although the present embodiment employs a configuration in which the information processing method M1 is executed using a single processor (processor 201), the present invention is not limited to this. That is, a configuration may be adopted in which the information processing method M1 is executed using a plurality of processors. For example, the acquisition step M11 may be executed in the first processor and the evaluation step M12 may be executed in the second processor. In this case, a plurality of processors that cooperate to execute the information processing method M1 may be provided in a single computer and may be configured to be able to communicate with each other via a bus, or may be distributed to the plurality of computers. It may be provided and configured to be able to communicate with each other via a network. As an example, a mode in which a processor built in a computer constituting a cloud server and a processor built in a computer owned by a user of the cloud server cooperate to execute the information processing method M1 can be considered.

Further, in the present embodiment, the configuration in which the evaluation model is stored in the memory (secondary memory 203) built in the same computer as the processor (processor 201) that executes the information processing method M1 is adopted, but the present invention. Is not limited to this. That is, a configuration may be adopted in which one or a plurality of evaluation models are stored in a memory built in a computer different from the processor that executes the information processing method M1. In this case, a computer having a built-in memory for storing one or a plurality of evaluation models is configured to be able to communicate with each other via a network with a computer having a built-in processor that executes the information processing method M1. As an example, an embodiment in which one or more evaluation models are stored in the memory built in the computer constituting the cloud server, and the processor built in the computer owned by the user of the cloud server executes the information processing method M1. Can be considered.

Further, in the present embodiment, a configuration in which one or more evaluation models are stored in a single memory (secondary memory 203) is adopted, but the present invention is not limited to this. That is, a configuration may be adopted in which one or a plurality of evaluation models are distributed and stored in a plurality of memories. In this case, the plurality of memories for storing one or more evaluation models may or may not be a computer having a built-in processor that executes the information processing method M1. It may be provided, or may be distributed among a plurality of computers (which may or may not include a computer having a processor that executes the information processing method M1). .. As an example, a configuration in which one or a plurality of evaluation models are distributed and stored in a memory built in each of a plurality of computers constituting a cloud server can be considered.

(Hardware configuration of machine learning device 40)
The machine learning device 40 may have a hardware configuration as shown in FIG. 14, for example. FIG. 14 is a block diagram showing an example of the hardware configuration of the machine learning device 40.

The machine learning device 40 is realized by using a general-purpose computer, and includes a processor 401, a primary memory 402, a secondary memory 403, an input / output interface 404, a communication interface 405, and a bus 406. The processor 401, the primary memory 402, the secondary memory 403, the input / output interface 404, and the communication interface 405 are connected to each other via the bus 406.

The secondary memory 403 stores a machine learning program and a learning data set. The secondary memory 403 is an example of a configuration that realizes the storage unit 41. A training data set is a set of a plurality of training data. The processor 401 expands the machine learning program stored in the secondary memory 403 on the primary memory 402. Then, the processor 401 functions as the control unit 42 by executing each step included in the machine learning method M2 according to the instructions included in the machine learning program expanded on the primary memory 402. The learning data set stored in the secondary memory 403 is created in the learning data set construction step M21 of the machine learning method M2, and is used in the trained model construction step M22 of the machine learning method M2. Further, one or a plurality of evaluation models constructed in the trained model construction step M22 of the machine learning method M2 are also stored in the secondary memory 403. The fact that the machine learning program is stored in the secondary memory 403 means that the source code or the execution format file obtained by compiling the source code is stored in the secondary memory 403. Further, the fact that one or more evaluation models are stored in the secondary memory 403 means that the parameters defining the one or more evaluation models are stored in the secondary memory 403.

Specific examples of the devices that can be used as the processor 401, the primary memory 402, or the secondary memory 403 are as described in the description of the hardware configuration of the information processing device 20a. The secondary memory 403 may be built in the machine learning device 40, or constitutes another computer (for example, a cloud server) connected to the machine learning device 40 via the input / output interface 404 or the communication interface 405. It may be built into the computer. In the present embodiment, the storage in the machine learning device 40 is realized by two memories (primary memory 402 and secondary memory 403), but the present invention is not limited to this. That is, the memory in the machine learning device 40 may be realized by one memory. In this case, for example, a certain storage area of the memory may be used as the primary memory 402, and the other storage area of the memory may be used as the secondary memory 403.

An input device and / or an output device is connected to the input / output interface 404. A specific example of the input / output interface 404 is as described in the description of the hardware configuration of the information processing apparatus 20a. Examples of the input device connected to the input / output interface 404 include an image pickup device 10a. The data acquired from the image pickup apparatus 10a in the machine learning method M2 is input to the machine learning apparatus 40 and stored in the primary memory 402. Further, a specific example of the input device connected to the input / output interface 404 is as described in the description of the hardware configuration of the information processing device 20a. The data acquired from the user in the machine learning method M2 is input to the machine learning device 40 via these input devices and stored in the primary memory 402. Further, a specific example of the output device connected to the input / output interface 404 is as described in the description of the hardware configuration of the information processing device 20a. The information provided to the user in the machine learning method M2 is output from the machine learning device 40 via these output devices. The machine learning device 40 may include a keyboard that functions as an input device and a display that functions as an output device, such as a laptop computer. Alternatively, the machine learning device 40 may have a built-in touch panel that functions as both an input device and an output device, such as a tablet computer.

Another computer is connected to the communication interface 405 by wire or wirelessly via a network. A specific example of the communication interface 405 is as described in the description of the hardware configuration of the information processing device 20a. Internetwork may be an intranet, an extranet, or the Internet. Data (eg, one or more evaluation models) provided by the machine learning device 40 to another computer (eg, information processing device 20a) is transmitted and received via these networks.

Although the present embodiment employs a configuration in which the machine learning method M2 is executed using a single processor (processor 401), the present invention is not limited to this. That is, a configuration may be adopted in which the machine learning method M2 is executed using a plurality of processors. For example, a configuration may be adopted in which the training data set construction step M21 is executed in the first processor and the trained model construction step M22 is executed in the second processor. In this case, a plurality of processors that collaborate to execute the machine learning method M2 may be provided in a single computer and may be configured to be able to communicate with each other via a bus, or may be distributed to a plurality of computers. It may be provided and configured to be able to communicate with each other via a network. As an example, a mode in which a processor built in a computer constituting a cloud server and a processor built in a computer owned by a user of the cloud server cooperate to execute the machine learning method M2 can be considered.

Further, in the present embodiment, a configuration is adopted in which the learning data set is stored in the memory (secondary memory 403) built in the same computer as the processor (processor 401) that executes the machine learning method M2. The present invention is not limited to this. That is, a configuration may be adopted in which the learning data set is stored in a memory built in a computer different from the processor that executes the machine learning method M2. In this case, the computer having the built-in memory for storing the learning data set is configured to be able to communicate with each other via the network with the computer having the built-in processor that executes the machine learning method M2. As an example, a mode in which a learning data set is stored in a memory built in a computer constituting a cloud server and a processor built in the computer owned by the user of the cloud server executes the machine learning method M2 is considered. Be done.

Further, in the present embodiment, a configuration in which the learning data set is stored in a single memory (secondary memory 403) is adopted, but the present invention is not limited to this. That is, a configuration may be adopted in which the learning data set is distributed and stored in a plurality of memories. In this case, the plurality of memories for storing the learning data set are provided in a single computer (which may or may not be a computer having a processor that executes the machine learning method M2). It may be distributed to a plurality of computers (which may or may not include a computer having a processor for executing the machine learning method M2). As an example, a configuration in which a learning data set is distributed and stored in a memory built in each of a plurality of computers constituting a cloud server can be considered.

Further, in the present embodiment, the configuration in which the information processing method M1 and the machine learning method M2 are executed by using different processors (processor 201 and processor 401) is adopted, but the present invention is not limited to this. That is, the information processing method M1 and the machine learning method M2 may be executed using the same processor. In this case, by executing the machine learning method M2, one or more evaluation models are stored in the memory built in the same computer as this processor. Then, this processor uses one or more evaluation models stored in this memory when executing the information processing method M1.

[Embodiment 3]
(Configuration of evaluation system 1b)
The evaluation system 1b according to the third embodiment of the present invention will be described with reference to FIG. FIG. 15 is a block diagram showing the configuration of the evaluation system 1b. As shown in FIG. 15, the evaluation system 1a includes an image pickup device 10a, an information processing device 20b, and an output device 30a. The image pickup device 10a and the output device 30a are as described in the second embodiment.

The information processing device 20b is a device for executing the information processing method M1b. The information processing device 20b is at least different from the second embodiment in that it evaluates the hygiene state of the nail without using the evaluation model constructed by machine learning.

(Configuration of information processing device 20b)
The configuration of the information processing device 20b will be described with reference to FIG. The information processing device 20b includes a control unit 22b and a storage unit 23b. The control unit 22b comprehensively controls the information processing device 20b. In the information processing device 20b, the control unit 22b includes an acquisition unit 221 and an evaluation unit 222b. The acquisition unit 221 executes the acquisition step M11 of the information processing method M1b. The evaluation unit 222b executes the evaluation step M12b and the output step M13 of the information processing method M1b.

(Flow of information processing method M1b)
The information processing method M1b is different from the information processing method M1 according to the second embodiment in that the evaluation step M12b is included instead of the evaluation step M12. In the following, only the differences will be described.

(Evaluation step M12b)
In the evaluation step M12b, the evaluation unit 222b uses a known image analysis technique to measure the length of the nail in the image data including the nail acquired by the acquisition unit 221 as a subject. Further, the evaluation unit 222b refers to the information indicating the correspondence between the nail length and the sanitary condition stored in advance in the storage unit 23b, and based on the measurement result of the nail length, the sanitary condition based on the nail length. Is evaluated, and data indicating the evaluation result is written in the storage unit 23b. For example, the evaluation unit 222 may measure the relative length of the tip of the nail as the "length of the nail" with respect to the length of the region of the entire nail in the image data including the nail as a subject. In this case, the storage unit 23b stores information indicating the correspondence between the relative length of the nail and the hygiene state.

Similarly, the evaluation unit 222b measures the color of the nail in the image data including the nail acquired by the acquisition unit 221 as a subject by using a known image analysis technique. With reference to the information indicating the correspondence between the nail color and the hygiene state stored in advance in the storage unit 23b, the hygiene state based on the nail color is evaluated from the measurement result of the nail color, and the evaluation result is shown. The data is written in the storage unit 23b.

(Effect of Embodiment 3)
In the present embodiment, the information processing device 20b evaluates the sanitary state of the nail without using the evaluation model constructed by machine learning, so that the machine learning device is unnecessary. Therefore, there is an advantage that the construction cost of the evaluation system 1b can be reduced.

[Example of realization by hardware]
In each of the above-described embodiments, the control blocks (particularly the control units 22 and 22b) of the information processing devices 20, 20a and 20b are realized by reading the program stored in the secondary memory 203 and executing the processor 201. Aspects have been described. Not limited to this, a part or all of the control blocks of the information processing devices 20, 20a and 20b may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. Further, in the second embodiment, the mode realized by the control block (particularly the control unit 42) of the machine learning device 40 reading the program stored in the secondary memory 403 and executing the processor 401 has been described. Not limited to this, a part or all of the control block of the machine learning device 40 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It will also be apparent to those skilled in the art that such changes or improvements can be made. Further, it is clear from the description of the scope of claims that such a modified or improved form may be included in the technical scope of the present invention.

〔summary〕
The present invention
(1) Information indicating the evaluation result is generated by evaluating the sanitary state of the nail based on the length of the nail with reference to the acquisition unit that acquires the image data including the nail as the subject and the image data. An information processing device equipped with an evaluation unit,
(2) The information processing apparatus according to (1), wherein the evaluation unit generates information including whether or not the person having the claw can enter or leave the room as information indicating the evaluation result.
(3) The information processing device of (2), wherein the indoor is a food factory room or a food processing plant.
(4) The evaluation unit evaluates the hygiene state of the nail based on the region of at least two or more fingernails included in the image data, any one of (1) to (3). Two information processing devices,
(5) The information processing device according to any one of (1) to (4), wherein the evaluation unit further evaluates the sanitary state of the nail based on the color of the nail with reference to the image data.
(6) Using the image data as an input, the first evaluation model constructed by machine learning is used to output the first information indicating the evaluation result of the hygiene condition based on the length of the claw, and the evaluation unit uses the first evaluation model. , An information processing apparatus according to any one of (1) to (5), which generates information indicating the evaluation result based on the first information.
(7) Using the image data as input, the second evaluation model constructed by machine learning to output the second information indicating the evaluation result of the hygiene condition based on the color of the nail is used, and the evaluation unit uses the second evaluation model. The information processing apparatus of (6), which generates information indicating the evaluation result based on the first information and the second information.
(8) An information processing device according to any one of (1) to (7), an imaging device that images a part of the human body including the nail and generates the image data, and outputs information indicating the evaluation result. Output device and evaluation system, including
(9) The image pickup apparatus includes an image pickup device, an alignment table that enables alignment of the portion with respect to the image pickup device, and a light-shielding wall that blocks light incident on the alignment table from the outside. The imaging device is installed so as to include the alignment table in the imaging range, the alignment table has a single color on the surface facing the imaging device, and the light-shielding wall is installed around the alignment table. (8) Evaluation system,
(10) The image pickup device is installed inside the image pickup box, and the image pickup box has a bottom surface as the alignment table, a side surface as the light-shielding wall, and a top surface on which the image pickup device is installed. The evaluation system of (9), wherein the side surface has an opening into which the part can be inserted inside the imaging box.
(11) The evaluation system according to (9) or (10), wherein the imaging device includes a guide mechanism capable of photographing the portion in a state where adjacent nails are separated from each other.
(12) A construction unit for constructing a first evaluation model for evaluating the hygiene condition based on the length of the nail by machine learning using the learning data set is provided, and the input of the first evaluation model is The machine learning device, which is image data including a nail as a subject, and the output of the first evaluation model is the first information indicating an evaluation result of a sanitary condition based on the length of the nail.
(13) The construction unit further constructs a second evaluation model for evaluating the hygiene condition based on the color of the nail by machine learning using the learning data set, and the input of the second evaluation model is the nail. The machine learning device of (12), which is the image data including the subject, and the output of the second evaluation model is the second information indicating the evaluation result of the sanitary condition based on the color of the nail.
(14) Information indicating the evaluation result is generated by the acquisition step of acquiring the image data including the nail as the subject and the evaluation of the hygiene state of the nail based on the length of the nail with reference to the image data. Information processing methods, including evaluation steps,
(15) A program for operating a computer as an information processing device according to any one of (1) to (7).
Is.

1 Nail condition inspection device (evaluation system)
1a, 1b Evaluation system 10 (10') Imaging device (imaging terminal)
10a Imaging device (imaging terminal)
20 Arithmetic logic unit (information processing device)
20a, 20b Information processing device 21 Learned neural network 21a First evaluation model 21b Second evaluation model 22,22b Control unit 23, 23b Storage unit 30 (30') Input / output device (input / output terminal) (output device)
30a Output device 40 Machine learning device 101 Imaging device (imaging device)
102 Shading wall (side)
103, 103a, 103b Bottom surface (alignment table)
104 Top surface 105 Imaging box 106 Front surface 107 Opening 108 LED
109a protrusion (guide mechanism)
109b Guide wire (guide mechanism)
201 Processor 202 Primary memory 203 Secondary memory 204 I / O interface 205 Communication interface 206 Bus 221 Acquisition unit 222, 222b Evaluation unit 401 Processor 402 Primary memory 403 Secondary memory 404 I / O interface 405 Communication interface 406 Bus 500 Screen example 501 Indoor Information indicating that entry is not possible 600 Screen example 601 Information indicating that entry is not possible indoors R Room (indoor)
H Hand image P Subject S Administrator

Claims (15)

An acquisition unit that acquires image data that includes the nail as a subject,
An evaluation unit that generates information indicating an evaluation result by evaluating the sanitary state of the nail based on the length of the nail with reference to the image data.
Information processing device equipped with. The information processing device according to claim 1, wherein the evaluation unit generates information including whether or not the person having the claw can enter or leave the room as information indicating the evaluation result. The information processing device according to claim 2, wherein the indoor is a food factory room or a food processing plant. The information according to any one of claims 1 to 3, wherein the evaluation unit evaluates the sanitary state of the nail based on the region of at least two or more fingernails contained in the image data. Processing equipment. The information processing device according to any one of claims 1 to 4, wherein the evaluation unit further evaluates the sanitary state of the nail based on the color of the nail with reference to the image data. Using the image data as an input, the first evaluation model constructed by machine learning to output the first information indicating the evaluation result of the sanitary condition based on the length of the nail is used.
The information processing apparatus according to any one of claims 1 to 5, wherein the evaluation unit generates information indicating the evaluation result based on the first information. Using the image data as an input, a second evaluation model constructed by machine learning is used to output a second information indicating the evaluation result of the sanitation state based on the color of the nail.
The information processing apparatus according to claim 6, wherein the evaluation unit generates information indicating the evaluation result based on the first information and the second information. The information processing device according to any one of claims 1 to 7.
An imaging device that images a part of the human body including the nail and generates the image data,
An output device that outputs information indicating the evaluation results and
Evaluation system including. The image pickup device
Imaging equipment and
An alignment table that enables alignment of the part with respect to the imaging device,
A light-shielding wall that blocks light incident on the alignment table from the outside,
Is equipped with
The imaging device is installed so as to include the alignment table in the imaging range.
The alignment table has a single color on the surface facing the imaging device.
The evaluation system according to claim 8, wherein the light-shielding wall is installed around the alignment table. The imaging device is installed inside the imaging box and is installed.
The imaging box
The bottom surface as the alignment table and
The side surface as the light-shielding wall and
It has a top surface on which the imaging device is installed.
The evaluation system according to claim 9, wherein the side surface has an opening into which the portion can be inserted inside the imaging box. The evaluation system according to claim 9 or 10, wherein the imaging device includes a guide mechanism that enables imaging of the portion in a state where adjacent nails are separated from each other. It is equipped with a construction unit that builds a first evaluation model that evaluates hygiene based on nail length by machine learning using a learning data set.
The input of the first evaluation model is image data including the nail as a subject, and is
The output of the first evaluation model is a machine learning device which is the first information indicating the evaluation result of the hygiene state based on the length of the nail. The construction unit further constructs a second evaluation model for evaluating the hygiene condition based on the color of the nail by machine learning using the learning data set.
The input of the second evaluation model is image data including the nail as a subject, and the output of the second evaluation model is the second information indicating the evaluation result of the sanitation state based on the color of the nail. Item 12. The machine learning device according to Item 12. The acquisition step to acquire image data including the nail as the subject,
An evaluation step of generating information indicating an evaluation result by evaluating the sanitary state of the nail based on the length of the nail with reference to the image data, and an evaluation step.
Information processing methods, including. A program for operating a computer as the information processing device according to any one of claims 1 to 7.

Images