JP7225551B2 - Biting detection device and program - Google Patents

Description

The present invention relates to a biting detection device and program.

For example, Patent Literature 1 discloses a recognition data transmission device that recognizes actions including the movements of a person based on user information representing the physical characteristics of a photographed person and motion information including the movements or gestures of the person. Are listed. This recognition data transmission device detects a feature amount of a part or the whole of a person's body from a photographed image, and identifies a person in the image from the detected feature amount. Also, this recognition data transmission device detects user information representing the physical characteristics of the person from the feature amount of the specified person, and detects motion information including user movement or gestures from the captured image. Then, this recognition data transmission device recognizes actions including human movements from the detected motion information and user information, divides the recognized action information for each human action, and generates the divided action information as block data. and transmits the generated block data to the outside for each block.

International Publication No. 2015/093330 pamphlet

By the way, for example, in nursery schools, kindergartens, and other facilities that take care of young children, biting occurs frequently between young children, and it is not uncommon for the facilities to take responsibility for supervision. A nursery teacher or the like in a facility, while monitoring the actions of many infants alone, also performs complicated paperwork, so in many cases, even if infants bite each other, they cannot notice it. Conventionally, techniques for watching the behavior of animals such as humans have been proposed, but no proposals have been found for techniques for detecting the occurrence of biting between a plurality of animals.

SUMMARY OF THE INVENTION An object of the present invention is to provide a biting detection device and program capable of detecting the occurrence of biting between a plurality of animals.

In order to achieve the above object, the bite detection device according to claim 1 comprises a photographing unit for photographing a plurality of animals to obtain image information, and a mouth of one animal based on the image information obtained by the photographing unit. a notification unit that notifies occurrence of biting when contact with a part of the other animal is detected; and when the difference between the depth of the one animal and the depth of the other animal obtained from the image information is equal to or less than a first threshold, it is determined that the bite has occurred. and a determination unit .

Further, the biting detection device according to claim 2 comprises: a photographing unit for photographing a plurality of animals to obtain image information; and a notification unit that notifies occurrence of biting when it is detected that the mouth of the one animal is in contact with a part of the other animal from the image information. and a determination unit that determines that a bite has occurred when a distance between the one animal and the other animal obtained from the image information in a direction perpendicular to the depth direction is equal to or less than a second threshold. , is equipped with

Further, the biting detection device according to claim 3 comprises: a photographing unit for photographing a plurality of animals to obtain image information; and a notification unit that notifies occurrence of biting when it is detected that the mouth of the one animal is in contact with a part of the other animal from the image information. and a determination unit that determines that a bite has occurred when it is detected from the image information that the physical change of the other animal is a predetermined change.

A fourth aspect of the present invention is a biting detection device according to the third aspect of the invention, wherein the physical change is a change in facial expression, and the predetermined change is a change to a facial expression expressing pain. It is said that there is

Further, the biting detection device according to claim 5 comprises: a photographing unit for photographing a plurality of animals and obtaining image information; a notification unit that notifies occurrence of biting when it is detected that the animal is in contact with the mouth of the one animal, a voice acquisition unit that acquires voice information from each of the plurality of animals, and the mouth of the one animal from the image information is detected to be in contact with a part of the other animal, and when it is detected that the sound indicated by the sound information acquired by the sound acquisition unit is a predetermined sound, and a determination unit that determines that biting has occurred.

Further, in the bite detection device according to claim 6 , in the invention according to claim 5 , the predetermined sound is at least one of a scream, a word expressing pain, and a word prompting to stop biting. It is said that

Further, the biting detection device according to claim 7 comprises: a photographing unit for photographing a plurality of animals to obtain image information; and a recording unit for recording image information obtained by the imaging unit, wherein the recording unit is configured to record the image information A difference between the depth of the one animal and the depth of the other animal obtained from is equal to or less than a first threshold, and is perpendicular to the depth direction of the one animal and the other animal obtained from the image information If the distance in the direction is less than or equal to a second threshold, start recording the image information.

According to an eighth aspect of the present invention, there is provided a biting detection device according to the seventh aspect of the invention, further comprising a storage unit including a predetermined temporary storage area and a non-temporary storage area, wherein the recording unit starts recording. After that, the image information obtained until the bite occurs is overwritten and recorded in the temporary storage area, and when the bite occurs, the image information overwritten and recorded in the temporary storage area is recorded in the non-temporary storage area. , and the image information obtained after the occurrence of the biting is recorded in the non-temporary storage area.

Further, the biting detection device according to claim 9 is characterized in that, in the invention according to claim 8 , the recording unit detects the mouth of the one animal from the image information obtained after the occurrence of the bite. The recording of the image information is ended when it is no longer detected that the part is touched.

Further, in order to achieve the above object, the program according to claim 10 is a program according to claim 10, wherein a computer is provided with the reporting unit and determination unit of the biting detection device according to any one of claims 1 to 6 , or It functions as a notification unit and a recording unit provided in the biting detection device according to any one of items 7 to 9 .

According to the inventions of claims 1 and 10 , occurrence of biting between a plurality of animals can be detected.
In addition, it is possible to detect the occurrence of biting between a plurality of animals with high accuracy compared to the case where the difference in depth is not taken into account.

According to the second aspect of the invention, it is possible to accurately detect the occurrence of biting between a plurality of animals, compared to the case where the distance in the direction orthogonal to the depth direction is not considered.

According to the third aspect of the invention, it is possible to detect the occurrence of biting between a plurality of animals with high accuracy, compared with the case where physical changes are not considered.

According to the fourth aspect of the invention, it is possible to detect the occurrence of biting between a plurality of animals with high accuracy, compared to the case where a change to a facial expression expressing pain is not detected.

According to the fifth aspect of the present invention, it is possible to accurately detect the occurrence of biting between a plurality of animals, compared to the case where only image information is used for biting determination.

According to the sixth aspect of the invention, it is possible to accurately detect the occurrence of biting between a plurality of animals accompanied by at least one of a scream, a word expressing pain, and a word prompting to stop biting.

According to the seventh aspect of the invention, it is possible to confirm later how biting occurs.
In addition, compared to the case where the depth difference and the distance in the direction perpendicular to the depth direction are not taken into account, it is possible to record image information with a high possibility of biting.

According to the eighth aspect of the invention, the storage area of the storage unit can be used more efficiently than when overwrite recording is not performed.

According to the ninth aspect of the invention, it is possible to prevent image information from being recorded after the biting is finished.

1 is a block diagram showing an example of the configuration of a bite detection system according to a first embodiment; FIG. 1 is a block diagram showing an example of an electrical configuration of a biting detection device according to a first embodiment; FIG. It is a block diagram showing an example of functional composition of a biting detection device concerning a 1st embodiment. FIG. 10 is a diagram showing an example of an image determined as occurrence of a bite according to the embodiment; It is a schematic diagram which shows an example of the depth difference and adjacent distance which concern on embodiment. FIG. 10 is a schematic diagram showing another example of the depth difference and the adjacent distance according to the embodiment; It is a figure which shows an example of the determination condition table which concerns on 1st Embodiment. 4 is a flow chart showing an example of the flow of processing of a biting detection program according to the first embodiment; FIG. 5 is a front view showing an example of a determination condition selection screen according to the embodiment; FIG. 7 is a block diagram showing an example of a functional configuration of a biting detection device according to a second embodiment; FIG. FIG. 11 is a diagram showing an example of a determination condition table according to the second embodiment; FIG.

Hereinafter, an example of a mode for carrying out the present invention will be described in detail with reference to the drawings.

[First embodiment]
FIG. 1 is a block diagram showing an example of the configuration of a biting detection system 90 according to the first embodiment.
As shown in FIG. 1 , a bite detection system 90 according to this embodiment includes a bite detection device 10 and a terminal device 50 .

A biting detection device 10 according to this embodiment is connected to a terminal device 50 via a network N. As shown in FIG. The network N may be, for example, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), or the like. The biting detection device 10 is installed, for example, in facilities such as nursery schools and kindergartens that take care of young children. In this case, the biting detection device 10 detects the occurrence of biting between children, the biting of a child by a nursery teacher, and the like from image information obtained by photographing the behavior of a child, which is an example of a person. The bite detection device 10 may be installed in a facility that takes care of patients with dementia, developmental disorders, or the like. Also, in the present embodiment, the biting detection target is human, but dogs, cats, and the like are also included. Targets for biting detection according to the present embodiment can be all animals having at least a mouth.

As an example of the terminal device 50 according to the present embodiment, a general-purpose computer such as a personal computer (PC) or a portable terminal device such as a smart phone or a tablet terminal is applied. The terminal device 50 is a terminal device used by a supervisor (in this embodiment, a nursery teacher or the like) who monitors a person's behavior. An alert message of occurrence of a bite is notified.

FIG. 2 is a block diagram showing an example of the electrical configuration of the biting detection device 10 according to the first embodiment.
As shown in FIG. 2, the biting detection device 10 according to the present embodiment includes a control unit 12, a storage unit 14, an imaging unit 16, a microphone 18, a display unit 20, an operation unit 22, and a communication unit 24. and have.

The control unit 12 includes a CPU (Central Processing Unit) 12A, a ROM (Read Only Memory) 12B, a RAM (Random Access Memory) 12C, and an input/output interface (I/O) 12D. are connected to each other.

Functional units including the storage unit 14, the imaging unit 16, the microphone 18, the display unit 20, the operation unit 22, and the communication unit 24 are connected to the I/O 12D. Each of these functional units can mutually communicate with the CPU 12A via the I/O 12D.

The control unit 12 may be configured as a sub-control unit that controls part of the operation of the bite detection device 10, or as a part of the main control unit that controls the overall operation of the bite detection device 10. good. An integrated circuit such as LSI (Large Scale Integration) or an IC (Integrated Circuit) chipset is used for part or all of each block of the control unit 12 . An individual circuit may be used for each of the above blocks, or a circuit in which a part or all of them are integrated may be used. The blocks may be provided integrally, or some of the blocks may be provided separately. Moreover, in each of the above blocks, a part thereof may be separately provided. The integration of the control unit 12 is not limited to LSI, and a dedicated circuit or general-purpose processor may be used.

For example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like is used as the storage unit 14 . A biting detection program 14A according to the present embodiment is stored in the storage unit 14 . The biting detection program 14A may be stored in the ROM 12B. The storage unit 14 also stores a determination condition table 14B, which will be described later. This determination condition table 14B may also be stored in the ROM 12B.

The bite detection program 14A may be pre-installed in the bite detection device 10, for example. The bite detection program 14A may be stored in a non-volatile storage medium or distributed via the network N and installed in the bite detection device 10 as appropriate. Examples of nonvolatile storage media include CD-ROMs (Compact Disc Read Only Memory), magneto-optical discs, HDDs, DVD-ROMs (Digital Versatile Disc Read Only Memory), flash memories, memory cards, and the like. be.

For example, a video camera equipped with a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) is used as the imaging unit 16 . The photographing unit 16 photographs the behavior of a person (an infant in the present embodiment) whose biting is to be detected as a moving image. In this embodiment, as an example, a plurality of video cameras are installed at predetermined intervals in order to measure the depth (described later) indicating the distance to the subject using the photographing unit 16 . It should be noted that the number of video cameras does not necessarily have to be plural, and even a single video camera can be used as long as the distance to the subject can be measured. Further, the photographing unit 16 may be provided separately from the biting detection device 10 or may be provided integrally with the biting detection device 10 .

The microphone 18 converts the voice uttered by a person whose biting is to be detected into an electric signal and inputs the electric signal. The microphone 18 may also be provided separately from the biting detection device 10 , or may be provided integrally with the biting detection device 10 , like the imaging unit 16 .

For the display unit 20, for example, a liquid crystal display (LCD), an organic EL (Electro Luminescence) display, or the like is used. The display unit 20 integrally has a touch panel. The operation unit 22 is provided with various operation keys such as numeric keys and alphabet keys. The display unit 20 and the operation unit 22 receive various instructions from the user of the biting detection device 10 (here, the user is the observer). The display unit 20 displays various kinds of information such as the results of processing executed in accordance with instructions received from the observer and notifications regarding the processing.

The communication unit 24 is connected to the network N, and can communicate with the terminal device 50 via the network N. In this example, the communication unit 24 and the network N are connected by wire, but they may be connected wirelessly.

The biting detection device 10 according to the present embodiment detects the occurrence of biting between people as described above. Therefore, the CPU 12A of the biting detection device 10 according to the present embodiment writes the biting detection program 14A stored in the storage unit 14 into the RAM 12C and executes it, thereby functioning as each unit shown in FIG.

FIG. 3 is a block diagram showing an example of the functional configuration of the biting detection device 10 according to the first embodiment.
As shown in FIG. 3, the CPU 12A of the biting detection device 10 according to this embodiment functions as an image acquisition section 30, a detection section 32, a determination section 34, a notification section 36, and a recording section .

The image acquisition unit 30 according to this embodiment acquires image information captured by the imaging unit 16 . This image information captures the actions of a plurality of people who are targets for biting detection.

The detection unit 32 according to this embodiment detects multiple people from the image information acquired by the image acquisition unit 30 . Specifically, the detection unit 32 detects a plurality of persons using known person recognition technology. Further, when detecting a plurality of persons, the detection unit 32 detects whether or not the mouth of one person is in contact with part of the other person. Specifically, there is a method of constructing a learned model by performing machine learning using a plurality of images obtained in advance by photographing the state of biting between people as teacher images. This machine learning includes, for example, Faster-RCNN (Region-based Convolutional Neural Network) related to object detection, SSD (Single Shot multibox Detector), FPGA (Field-Programmable Gate Array), etc. Built-in CNN (Convolutional Neural Network) ) and other techniques are applied. The method of detecting mouth contact from image information is not limited to these methods using machine learning, and other methods such as pattern matching may be applied.

The determination unit 34 according to the present embodiment determines whether or not a bite is occurring between people based on the detection result of the detection unit 32 . Specifically, the determination unit 34 determines that biting has occurred when the detection unit 32 detects that one person's mouth is in contact with part of the other person's mouth. FIG. 4 shows an example of an image determined as occurrence of a bite.

Further, in addition to detection of contact with the mouth, other conditions may be combined to further improve the accuracy of biting determination. Specifically, the determination unit 34 detects from the image information that the mouth of one person is in contact with a part of the other person, and determines the depth of the one person and the depth of the other person obtained from the image information. It may be determined that a bite has occurred when the difference from the depth of the person is equal to or less than the first threshold. It should be noted that the "depth" referred to here indicates the distance from the photographing unit 16 to the subject, as described above. For this depth measurement, for example, the plurality of video cameras described above are used, but an infrared sensor, an ultrasonic sensor, or the like may be used instead of the plurality of video cameras. In this embodiment, the difference between the depth of one person and the depth of the other person is called "depth difference". This depth difference is measured by the detector 32 . Also, as the first threshold value, for example, an appropriate value is set that is highly likely to distinguish the occurrence of a bite when applied to past cases of occurrence of a bite.

Further, the determination unit 34 detects that one person's mouth is in contact with a part of the other person from the image information, and determines the depth direction of the one person and the other person obtained from the image information. If the distance in the direction orthogonal to is equal to or less than the second threshold, it may be determined that biting has occurred. Note that the “depth direction” here indicates the direction from the photographing unit 16 toward the subject. For example, in a three-dimensional space with X-axis (horizontal direction), Y-axis (vertical direction), and Z-axis (depth direction), when an image is projected onto the XY plane, the depth direction is the Z-axis direction. The orthogonal direction is the direction of the XY plane. In this embodiment, the distance in the direction orthogonal to the depth direction is called "adjacent distance". This adjacent distance is measured, for example, based on the number of pixels between people obtained from image information. This adjacent distance is measured by the detector 32 . Also, as the second threshold, an appropriate value with a high possibility of classifying the occurrence of biting is set as in the case of the first threshold.

FIG. 5 is a schematic diagram showing an example of the depth difference and the adjacent distance according to this embodiment.
As shown in FIG. 5, the depth D _A is represented as the distance between the imaging unit 16 and the person A in the Z-axis direction, and the depth D _B is represented as the distance between the imaging unit 16 and the person B in the Z-axis direction. be. Also, the adjacent distance L _AB is expressed as the distance between person A and person B in the direction of the XY plane.

In the _{example shown in FIG. 5, when the absolute value of the difference |D A -D B} | presumed to be in Then, when the adjacent distance _LAB is equal to or less than the second threshold, it is highly possible that the person A and the person B are in contact with each other and biting occurs between them.

FIG. 6 is a schematic diagram showing another example of the depth difference and the adjacent distance according to this embodiment.
In the example _{shown in FIG. 6, when the absolute value of the difference |D A -D B |} presumed to be in a state of absence. In this case, even if the adjacent distance _LAB is equal to or less than the second threshold value, the person A and the person B may be separated from each other in the depth direction, which may result in an erroneous determination.

As can be understood from the examples shown in FIGS. 5 and 6, the determination unit 34 detects that one person's mouth is in contact with a part of the other person from the image information, and the depth D When the absolute value of the difference |D _A -D _B | (hereinafter referred to as depth difference D _AB ) between _A and depth D _B is equal to or less than a first threshold and the adjacent distance L _AB is equal to or less than a second threshold Furthermore, it is more desirable to determine that biting has occurred.

Further, the determination unit 34 detects from the image information that one person's mouth is in contact with a part of the other person, and, as an example, as shown in FIG. If teeth are detected in the mouth of the person, it may be determined that biting has occurred. This tooth is detected by the detector 32 . To detect teeth, for example, the above-described method using machine learning, the pattern matching method, or the like is applied.

Further, the determination unit 34 detects from the image information that one person's mouth is in contact with a part of the other person, and determines from the image information that the other person's physical change is a predetermined change. It may be determined that biting occurs when it is detected that Specifically, the physical change is, for example, a change in facial expression as shown in FIG. 4 described above, and the predetermined change is, for example, a change to a facial expression expressing pain. This change in facial expression is detected by the detection unit 32 . Note that the above-described method using machine learning, the pattern matching method, or the like is applied as an example to detect a change in facial expression. When using machine learning, machine learning may be performed in advance using a plurality of images obtained by photographing facial expressions expressing pain as teacher images, and a trained model may be constructed. It should be noted that a change to an angry facial expression may be employed instead of the facial expression to express pain. In this case, the condition is that the change in facial expression is a change to an angry facial expression. Also, as a physical change, a change in the contact position of the mouth may be employed instead of the change in facial expression. In this case, the condition is that the contact position of the mouth changes abruptly. Also, as a physical change, a change in skin color due to bleeding due to a bite may be adopted. In this case, the condition is that the change in skin color is a change to red.

Further, as an example, the determination unit 34 selectively uses any one of the conditions 1, 2, 3, 4, . . . registered in the determination condition table 14B shown in FIG. It is good also as a form to carry out.

FIG. 7 is a diagram showing an example of the determination condition table 14B according to the first embodiment.
As shown in FIG. 7, in the determination condition table 14B according to the present embodiment, conditions 1, conditions 2, conditions 3, conditions 4, . . In addition, in FIG. 7, it is shown that the determination conditions marked with white circles are associated with each other.

In the determination condition table 14B shown in FIG. 7, each determination condition or combination of determination conditions of "mouth contact", "depth difference", "adjacent distance", "teeth", and "expression change" described above is registered. It is "Mouth contact" is a condition that "one person's mouth is in contact with another person's part." The "depth difference" is a condition that "the difference between the depth of one person and the depth of the other person is equal to or less than a first threshold". The "adjacent distance" is a condition that "the distance between one person and the other person in the direction perpendicular to the depth direction is equal to or less than the second threshold". "Teeth" is a condition that "the teeth are visible in one person's mouth." "Facial expression change" is a condition that "the facial expression of the other person changes to a facial expression expressing pain".

In the example shown in FIG. 7, condition 1 is associated with "mouth contact" as a determination condition. Further, condition 2 is associated with "mouth contact", "depth difference", and "adjacent distance" as determination conditions. Further, condition 3 is associated with "mouth contact" and "teeth" as determination conditions. In addition, condition 4 is associated with "mouth contact" and "expression change" as determination conditions. The conditions registered in the determination condition table 14B are not limited to conditions 1, 2, 3, 4, . . . ”, “teeth”, and “change in facial expression” are appropriately combined.

The notification unit 36 according to the present embodiment notifies occurrence of biting when the determining unit 34 determines that biting has occurred. Specifically, the notification unit 36 causes the terminal device 50 used by the monitor to display by transmitting an alert message such as, for example, "Bite has occurred." Also, instead of this alert message, it may be notified by voice or by an alarm sound.

On the other hand, the recording unit 38 according to this embodiment records the image information obtained by the imaging unit 16 . Specifically, the recording unit 38 starts recording the image information when the depth difference _DAB is equal to or less than the first threshold and the adjacent distance _LAB is equal to or less than the second threshold. The recording start condition may be other than this condition, and may be only one of the depth difference D _AB and the adjacent distance L _AB , or may be combined with other conditions as appropriate.

Here, the storage unit 14 includes a predetermined temporary storage area 15A and a non-temporary storage area 15B. The recording unit 38 overwrites and records the image information obtained between the start of recording and the occurrence of the biting in the temporary storage area 15A. Then, when biting occurs, the recording unit 38 moves the image information overwritten and recorded in the temporary storage area 15A to the non-temporary storage area 15B, and non-temporarily stores the image information obtained after the biting occurs. Record in area 15B. Copying may be applied instead of moving image information.

In addition, the recording unit 38 ends the recording of the image information when it is no longer detected that one person's mouth is in contact with part of the other person's mouth from the image information obtained after the occurrence of the biting. . Here, if the biting occurs as described above, the recording unit 38 may record the image information obtained after the biting occurs in a recording area different from the temporary storage area 15A and the non-temporary storage area 15B. good. In this case, for example, when the mouth contact is no longer detected as a trigger, the image information overwritten and recorded in the temporary storage area 15A is moved to the non-temporary storage area 15B, and the image recorded in the recording area is moved to the non-temporary storage area 15B. The information is moved to the non-temporary storage area 15B. It should be noted that copying may be applied in place of moving image information in the same manner as described above.

Next, the operation of the biting detection device 10 according to the first embodiment will be described with reference to FIG. FIG. 8 is a flow chart showing an example of the processing flow of the biting detection program 14A according to the first embodiment.

First, when the power of the bite detection device 10 is turned on, the bite detection program 14A is started and the following steps are executed.

At step 100 in FIG. 8 , the image acquisition section 30 acquires image information captured by the imaging section 16 . Note that, in the present embodiment, each frame of a moving image is acquired as image information.

At step 102 , the detection unit 32 determines whether or not a plurality of people have been detected from the image information acquired at step 100 . When it is determined that a plurality of people are detected (in the case of affirmative determination), the process proceeds to step 104, and when it is determined that a plurality of people is not detected (in the case of negative determination), the process returns to step 100 and repeats the process.

At step 104, the detection unit 32 measures the depth difference D _AB and the adjacent distance L _AB for the plurality of persons detected at step 102, as described above with reference to FIGS.

At step 106, the determination unit 34 determines whether or not the recording start condition is satisfied. The recording start condition here is, as described above, the condition that the depth difference D _AB is equal to or less than the first threshold and the adjacent distance L _AB is equal to or less than the second threshold. If it is determined that the recording start condition is satisfied (affirmative determination), the process proceeds to step 108, and if it is determined that the recording start condition is not satisfied (negative determination), the process returns to step 100 and repeats the process.

At step 108, the recording unit 38 overwrites and stores the image information satisfying the recording start condition at step 106 in the temporary storage area 15A.

At step 110, the detection unit 32 detects from the image information recorded in the temporary storage area 15A at step 108 whether the mouth of one person is in contact with part of the other person. If mouth contact is detected (positive determination), the process proceeds to step 112, and if mouth contact is not detected (negative determination), the process returns to step 100 and repeats the process. It should be noted that it may be detected whether or not the mouth of one person is in contact with part of the other person for a certain period of time or longer. In this case, for example, if the contact is for 10 seconds or longer, it is highly likely that biting has occurred.

At step 112, the determination unit 34 determines whether or not a combination condition is specified, based on, for example, the contents of the selection specification on the determination condition selection screen shown in FIG. When it is determined that the combination condition is specified (in the case of affirmative determination), the process proceeds to step 114, and when it is determined that the combination condition is not specified (in the case of a negative determination), the process proceeds to step 116.

FIG. 9 is a front view showing an example of the determination condition selection screen 20A according to this embodiment.
A judgment condition selection screen 20A shown in FIG. 9 is a screen displayed on the display unit 20 in accordance with an operation input by the observer before the biting detection device 10 starts the bite judgment operation. The observer selects and designates a desired condition via the judgment condition selection screen 20A. Here, as an example, "Condition 2", which is a combination condition, is selected and specified. Condition 1, Condition 2, Condition 3, Condition 4, . . . in the judgment condition selection screen 20A shown in FIG.・Compatible with That is, when "Condition 2" is selected and specified on the determination condition selection screen 20A, "mouth contact", "depth difference", and "adjacent distance" are the determination conditions from the determination condition table 14B shown in FIG. is specified. In this case, the biting determination operation is performed using these "mouth contact", "depth difference", and "adjacent distance" as determination conditions.

At step 114, the determination unit 34 determines whether or not the combination condition is satisfied. For example, as shown in the judgment condition selection screen 20A shown in FIG. 9, if the combination condition is "condition 2", it is judged whether or not each of the "depth difference" and the "adjacent distance" is satisfied. If it is determined that the combination condition is satisfied (positive determination), the process proceeds to step 116, and if it is determined that the combination condition is not satisfied (negative determination), the process returns to step 100 and is repeated.

At step 116, the determination unit 34 determines that a bite is occurring between people.

In step 118, the notification unit 36 transmits an alert message to the terminal device 50 used by the monitor to indicate that a bite has occurred between humans. Note that, as described above, instead of the alert message, the notification may be made by voice or warning sound.

At step 120, the recording unit 38 moves the image information obtained before the occurrence of the biting, overwritten and recorded in the temporary storage area 15A, to the non-temporary storage area 15B. Copying may be applied instead of moving image information.

At step 122, the recording section 38 stores the image information obtained after the occurrence of the bite in the non-temporary storage area 15B. It should be noted that the order of steps 120 and 122 may be changed. In this case, step 120 is not executed in the second and subsequent loops.

At step 124, the determination unit 34 determines whether or not the recording end condition is satisfied. The recording end condition here is, as described above, a condition that the contact of the mouth is no longer detected from the image information obtained after the occurrence of the biting. If it is determined that the recording end condition is satisfied (in the case of affirmative determination), the process proceeds to step 126, and if it is determined that the recording end condition is not satisfied (in the case of a negative determination), the process returns to step 122 and the process is repeated.

At step 126, the recording unit 38 ends the recording of the image information, and the series of processing by the bite detection program 14A ends.

As described above, according to the present embodiment, the occurrence of human-to-human bites is detected with high accuracy and notified to the supervisor, thereby reducing the burden on the supervisor.
In addition, since the situations before and after the occurrence of the bite are recorded, it is possible to check the history of the occurrence of the bite later.

In the present embodiment, a plurality of people are targeted, but for example, patients with dementia, developmental disorders, or the like may bite their arms or the like due to self-harm. Therefore, the present embodiment is similarly applied even when one person is targeted. Specifically, the detection unit 32 detects one person using a known person recognition technology. Further, when detecting one person, the detection unit 32 detects whether or not the person's mouth is in contact with a part of the person (such as an arm). Specifically, there is a method of constructing a trained model by performing machine learning using a plurality of images obtained by photographing a person's biting state in advance as teacher images. Note that the method of detecting mouth contact from image information is not limited to the method using machine learning, and other methods such as pattern matching may be applied.

[Second embodiment]
In the above-described first embodiment, the form of detecting the occurrence of biting using only image information has been described. In this embodiment, an embodiment will be described in which the occurrence of biting is detected using audio information in addition to image information.

FIG. 10 is a block diagram showing an example of the functional configuration of the biting detection device 11 according to the second embodiment.
As shown in FIG. 10 , the biting detection device 11 according to this embodiment includes an image acquisition section 30 , a detection section 32 , a determination section 34 , a notification section 36 , a recording section 38 and a voice acquisition section 40 . The same components as those of the biting detection device 10 shown in the first embodiment are denoted by the same reference numerals, and repeated explanations are omitted.

The voice acquisition unit 40 according to the present embodiment acquires voice information collected via the above-described microphone 18 shown in FIG. 2 for each of a plurality of people.

The determination unit 34 according to the present embodiment detects that one person's mouth is in contact with a part of the other person from the image information, and is indicated by the audio information acquired by the audio acquisition unit 40 When it is detected that the sound is a predetermined sound, it is determined that biting has occurred. The predetermined sound is at least one of a scream, a word of pain, and a word that encourages a person to stop biting. Specifically, the detection unit 32 detects at least one of a scream such as "Cheer!", a word expressing pain such as "Ouch!" detect.

As an example, the determination unit 34 according to the present embodiment selectively uses any one of conditions 1, 2, 3, 4, and 5 registered in a determination condition table 14C shown in FIG. make a judgment.

FIG. 11 is a diagram showing an example of the determination condition table 14C according to the second embodiment.
As shown in FIG. 11, conditions 1 to 5 associated with one or more determination conditions are registered in the determination condition table 14C according to the present embodiment. In FIG. 11, white circles indicate that determination conditions are assigned.

In the example shown in FIG. 11, condition 1 is associated with "mouth contact" as a determination condition. Further, condition 2 is associated with "mouth contact", "depth difference", and "adjacent distance" as determination conditions. Further, condition 3 is associated with "mouth contact" and "teeth" as determination conditions. In addition, condition 4 is associated with "mouth contact" and "expression change" as determination conditions. Further, condition 5 is associated with “mouth contact” and “voice” as determination conditions.

Any one of the conditions 1 to 5 registered in the judgment condition table 14C is used as the judgment condition for biting judgment. Which one of these conditions 1 to 5 is to be used is selected and specified by the observer via the judgment condition selection screen shown in FIG. 9, as an example.

As described above, according to the present embodiment, biting is determined using audio information in addition to image information, so occurrence of biting can be detected with higher accuracy than when only image information is used.

As described above, the biting detection device has been exemplified as an embodiment. The embodiment may be in the form of a program for causing a computer to execute the function of each part provided in the biting detection device. Embodiments may be in the form of a computer-readable storage medium storing this program.

In addition, the configuration of the biting detection device described in the above embodiment is merely an example, and may be changed according to the situation without departing from the gist of the invention.

Further, the flow of processing of the program described in the above embodiment is also an example, and unnecessary steps may be deleted, new steps added, or the processing order changed without departing from the scope of the invention. good.

Further, in the above embodiment, a case has been described in which the processing according to the embodiment is realized by a software configuration using a computer by executing a program, but the present invention is not limited to this. Embodiments may be implemented by, for example, a hardware configuration or a combination of hardware and software configurations.

10, 11 biting detection device 12 control unit 12A CPU
12B ROM
12C RAM
12D I/O
14 storage unit 14A biting detection program 14B, 14C determination condition table 15A temporary storage area 15B non-temporary storage area 16 photographing unit 18 microphone 20 display unit 22 operation unit 24 communication unit 30 image acquisition unit 32 detection unit 34 determination unit 36 notification unit 38 Recording unit 40 Sound acquisition unit 50 Terminal device 90 Biting detection system

Claims (10)

a photographing unit that obtains image information by photographing a plurality of animals;
a notification unit that notifies occurrence of biting when it is detected that the mouth of one animal is in contact with a part of the other animal from the image information obtained by the photographing unit;
It is detected from the image information that the mouth of the one animal is in contact with a part of the other animal, and the depth of the one animal and the depth of the other animal obtained from the image information. A determination unit that determines that biting occurs when the difference is less than or equal to the first threshold;
Bite detection device with. a photographing unit that obtains image information by photographing a plurality of animals;
a notification unit that notifies occurrence of biting when it is detected that the mouth of one animal is in contact with a part of the other animal from the image information obtained by the photographing unit;
It is detected from the image information that the mouth of the one animal is in contact with a part of the other animal, and the depth direction of the one animal and the other animal obtained from the image information. a determination unit that determines that biting occurs when the distance in the orthogonal direction is equal to or less than a second threshold;
Bite detection device with. a photographing unit that obtains image information by photographing a plurality of animals;
a notification unit that notifies occurrence of biting when it is detected that the mouth of one animal is in contact with a part of the other animal from the image information obtained by the photographing unit;
It is detected from the image information that the mouth of the one animal is in contact with a part of the other animal, and the physical change of the other animal is a predetermined change from the image information. a determination unit that determines that biting has occurred when it is detected that
Bite detection device with. The physical change is a change in facial expression,
4. The biting detection device according to claim 3, wherein said predetermined change is a change to a facial expression expressing pain. a photographing unit that obtains image information by photographing a plurality of animals;
a notification unit that notifies occurrence of biting when it is detected that the mouth of one animal is in contact with part of the other animal from the image information obtained by the photographing unit;
a voice acquisition unit that acquires voice information from each of the plurality of animals;
It is detected from the image information that the mouth of the one animal is in contact with a part of the other animal, and the sound indicated by the sound information acquired by the sound acquisition unit is a predetermined sound. a determination unit that determines that biting has occurred when it is detected that
Bite detection device with. 6. The biting detector according to claim 5, wherein said predetermined sound is at least one of a scream, a word expressing pain, and a word prompting to stop biting. a photographing unit that obtains image information by photographing a plurality of animals;
a notification unit that notifies occurrence of biting when it is detected that the mouth of one animal is in contact with a part of the other animal from the image information obtained by the photographing unit;
a recording unit for recording image information obtained by the imaging unit;
with
The recording unit is configured such that a difference between the depth of the one animal obtained from the image information and the depth of the other animal is equal to or less than a first threshold, and the depth of the one animal and the other animal obtained from the image information A bite detection device that starts recording the image information when a distance from the animal in a direction orthogonal to a depth direction is equal to or less than a second threshold. Further comprising a storage unit including a predetermined temporary storage area and a non-temporary storage area,
The recording unit overwrites and records in the temporary storage area the image information obtained from the start of recording until the occurrence of the biting, and when the biting occurs, the recording is overwritten in the temporary storage area. 8. The biting detection device according to claim 7, wherein image information is moved to said non-temporary storage area, and image information obtained after occurrence of a bite is recorded in said non-temporary storage area. The recording unit records the image information when it is no longer detected that the mouth of the one animal is in contact with a part of the other animal from the image information obtained after the occurrence of the biting. 9. The bite detection device of claim 8, wherein the bite detection device terminates. A computer, a reporting unit and a determination unit provided in the biting detection device according to any one of claims 1 to 6 , or a reporting unit and a determination unit provided in the bite detection device according to any one of claims 7 to 9 A program to function as a recording part .

Images