WO2023223376A1 - Human detection device, human detection system, human detection method, and human detection program - Google Patents

Abstract

A human detection device (10) comprises: a human shape extraction unit (11) that extracts human shape information (D11) indicating a human shape in an image (D60) obtained by capturing an imaging range with a visible image camera (60); a heat generation region extraction unit (12) that extracts heat generation region information (D12) that indicates regions with a large amount of heat generation in a thermal image (D70) captured by an infrared camera (70); a human probability calculation unit (13) that selects a predetermined specific-shaped part of the human shape, and calculates a human probability (D13), which is a numerical value that represents the certainty that the selected specific-shaped part is a human part actually existing within the imaging range, on the basis of the human shape information (D11) and the heat generation region information (D12); and a human determination unit (14) that determines whether the human shape indicated by the human shape information (D11) is the shape of an actually existing human or the shape of a person in a medium, on the basis of the human probability (D13) calculated for the selected specific-shaped part.

Description

Human detection device, human detection system, human detection method, and human detection program

The present disclosure relates to a human detection device, a human detection system, a human detection method, and a human detection program.

For example, Patent Document 1 discloses that a first photoelectric conversion element generates first image data from a visible light component from a subject, a second photoelectric conversion element generates second image data from an infrared component from a subject, an imaging unit having a second photoelectric conversion element, a filter provided on the subject side of the second photoelectric conversion element and on the optical axis from the subject, which removes visible light components and transmits infrared components; a position specifying unit that specifies the position of the subject image within the screen; an intensity detection unit that detects the infrared intensity from the subject in the second image data; This paper proposes a technology related to an imaging device having a living body detection unit that detects a living body.

JP2010-136223A (Page 18, Figure 4)

However, the device of Patent Document 1 detects a living body (for example, an actually existing person) using a thermal image as second image data taken with an infrared camera as a second photoelectric conversion element. However, there has been a problem in that a person (also referred to as a "person in the medium") displayed on a display device such as a television that generates heat may be erroneously detected as a living person.

The present disclosure makes it possible to determine with high accuracy whether a person photographed by a visible image camera is an actually existing person or a person displayed or drawn on the medium. The purpose of the present invention is to provide a human detection device, a human detection system, a human detection method, and a human detection program.

The human detection device of the present disclosure receives an image obtained by photographing a photographing range with a visible image camera that photographs an image in at least a wavelength range of visible light, and receives human shape information indicating the shape of a person in the image. a human shape extraction unit that extracts a human shape, a heat generation area extraction unit that receives a thermal image taken by an infrared camera and extracts heat generation area information indicating a region with a large amount of heat generation in the thermal image, and Selecting a part with a predetermined specific shape, and based on the human shape information and the heat generation area information, the selected part with the specific shape is a part of a person actually existing within the imaging range. The shape of the person indicated by the human shape information is based on the human-likeness calculation unit that calculates the human-likeness, which is a numerical value representing the likelihood of The present invention is characterized by comprising a person determining unit that determines whether the shape is the shape of the person who actually exists or the shape of a person in the medium who is a person displayed or drawn on the medium.

The human detection method of the present disclosure is a method executed by a human detection device, which receives an image obtained by photographing a photographing range with a visible image camera that photographs an image in at least a wavelength range of visible light, and a step of extracting human shape information indicating the shape of a person in the image; a step of receiving a thermal image taken with an infrared camera and extracting heat generating area information indicating a region with a large amount of heat generation in the thermal image; Selecting a predetermined specific shaped part of the human shape, and determining whether the selected specific shaped part actually exists within the imaging range based on the human shape information and the heat generation area information. calculating the human-likeness, which is a numerical value representing the certainty that the part is a human part, and the human-likeness indicated by the human-shaped information based on the human-likeness calculated for the selected part of the specific shape. The method is characterized by the step of determining whether the shape is the shape of the person who actually exists or the shape of a person in the medium who is a person displayed or drawn on the medium.

According to the present invention, it is possible to determine with high accuracy whether the shape of a person photographed by a visible image camera is the shape of an actually existing person or the shape of a person displayed or drawn on the medium. It can be determined by

1 is a block diagram schematically showing the configuration of a human detection device and a human detection system according to Embodiment 1. FIG. 1 is a diagram illustrating an example of a hardware configuration of a human detection device according to Embodiment 1. FIG. (A) to (C) are diagrams showing examples of information acquired by each block of the human detection device according to Embodiment 1, in which (A) is a visible image taken by a visible image camera; ) shows human shape information extracted from a visible image, and (C) shows heat generation area information extracted from a thermal image taken with an infrared camera. 3 is a flowchart showing the operation of the human detection device according to the first embodiment. 7 is a flowchart showing the operation of the spherical part human-likeness calculation unit in the first embodiment. (A) and (B) are schematic diagrams showing the operation of the spherical part human-likeness calculation unit in the first embodiment. 7 is a flowchart showing the operation of a line segment part human-likeness calculation unit in the first embodiment. (A) and (B) are schematic diagrams showing the operation of the line segment part human-likeness calculation unit in the first embodiment. FIG. 2 is a block diagram schematically showing the configuration of a human detection device and a human detection system according to a second embodiment. FIG. 3 is a block diagram schematically showing the configuration of a human detection device and a human detection system according to a third embodiment. FIG. 3 is a block diagram schematically showing the configuration of a human detection device and a human detection system according to a fourth embodiment.

Below, a person detection device, a person detection system, a person detection method, and a person detection program according to embodiments will be described with reference to the drawings. The following embodiments are merely examples, and the embodiments can be combined as appropriate and each embodiment can be changed as appropriate.

<<1>> Embodiment 1
<<1-1>> Configuration FIG. 1 is a block diagram schematically showing the configuration of a human detection device 10 and a human detection system 1 according to the first embodiment. As shown in FIG. 1, the human detection system 1 includes a human detection device 10, a visible image camera 60, and an infrared camera 70. The human detection device 10 is a device that can implement the human detection method according to the first embodiment.

The human detection device 10 receives a visible image (also simply referred to as an "image") D60 obtained by photographing a shooting range with a visible image camera 60, and human shape information indicating the shape of a person in this visible image D60. A human shape extraction unit 11 that extracts D11 receives a thermal image (that is, an infrared image) D70 taken by an infrared camera 70, and extracts heat generation area information D12 indicating a region with a large amount of heat generation in this thermal image D70. It has a heat generating area extracting section 12 that performs the following. Examples of the visible image D60, the human shape information D11, and the heat generating area information D12 are shown in FIGS. 3A to 3C, which will be described later. The visible image camera 60 is a camera that captures images in at least the wavelength range of visible light, and is also called a visible light camera. The infrared camera 70 is a camera that takes images in an infrared wavelength region, and is also called a thermal image camera or a thermo camera. The visible image camera 60 and the infrared camera 70 are configured such that the photographing range, which is an area photographed by the visible image camera 60, and the photographing range, which is an area photographed by the thermal imaging camera, overlap each other (that is, they share a common photographing range. ), it will be installed. Further, the human detection device 10 may receive visible images D60 from a plurality of visible image cameras 60. Further, the human detection device 10 may receive thermal images D70 from a plurality of infrared cameras 70. The visible image camera 60 and the infrared camera 70 do not need to be dedicated to the human detection system 1, and may be cameras used as part of existing equipment.

In addition, the human detection device 10 selects a predetermined specific shape part of the human shape, and determines whether the specific shape part is actually within the imaging range based on the human shape information D11 and the heat generation area information D12. The human-likeness calculation unit 13 calculates the human-likeness D13, which is a numerical value representing the certainty that the part is a part of an existing person, and the human-likeness calculation unit 13 calculates the human-likeness D13, which is a numerical value representing the certainty that the part is a part of an existing person. A person who determines whether it is the shape of a person who is doing the job, or the shape of a person displayed or drawn on a medium (for example, a screen of a display device or a printed matter such as a poster), and outputs the determination result D14. The determination unit 14 has a determination unit 14.

The human-likeness calculation unit 13 selects a spherical spherical part (e.g., head, hand, etc.) as a part of the body with a specific shape (i.e., a specific part), and calculates the human-likeness (" The spherical part human-likeness calculation unit 13a calculates D13a, and selects a linear segment part (for example, a part between joints) as a part of a specific shape of the body. and a line segment part human-likeness calculation unit 13b that calculates human-likeness (also referred to as "second human-likeness") D13b based on the line segment parts. In this case, the person determining unit 14 determines whether the overall shape of the person indicated by the human shape information D11 is the shape of an actually existing person, based on the humanness D13a and D13b of specific parts of the human shape information D11. It is determined whether the shape is that of a person in the medium, and a determination result D14 is output. Note that the determination of the shape of the entire person indicated by the human shape information D11 is referred to as "determination of the human-likeness of the entire human shape information D11." Further, the human-likeness calculation section 13 may include only one of the spherical part human-likeness calculation section 13a and the line segment human-likeness calculation section 13b.

The spherical part human-likeness calculation unit 13a calculates the human-likeness D13a of a specific part of a person when the specific part of the person is considered to be spherical, based on the human shape information D11 and the heat generation area information D12. The calculated human-likeness D13a is used by the human-judgment unit 14 to determine the human-likeness of the entire human shape information D11. Specifically, the spherical part human-likeness calculation unit 13a creates a histogram in which the horizontal axis is the temperature value T and the vertical axis is the degree #N for a circular region including the spherical part, and based on the shape of this histogram, The likelihood D13a is calculated. The spherical part human-likeness calculation unit 13a calculates the human-likeness in the case where the histogram shape shows bimodality to a higher value (i.e., the human-likeness in the case where the histogram shape does not show bimodality) (i.e., set to a value indicating that the person is likely to be a person). A specific example of the histogram will be described later using FIGS. 6(A) and (B).

The line segment part human-likeness calculation unit 13b calculates the human-likeness D13b of a specific part of a person when the specific part of the person is regarded as a line segment shape, based on the human shape information D11 and the heat generation area information D12. The calculated human-likeness D13b of the specific part is used by the person determining unit 14 to determine the human-likeness of the entire human shape information D11. Specifically, the line segment part human-likeness calculation unit 13b creates a temperature distribution graph with the horizontal axis as the position and the vertical axis as the temperature value T for the line segment part, and calculates the human-likeness based on the shape of this temperature distribution graph. Calculate D13b. The line segment part human-likeness calculation unit 13b sets the human-likeness D13b when the shape of the temperature distribution graph is unimodal to a higher value than the humanlikeness D13b when the shape of the temperature distribution graph is not unimodal. (In other words, a value indicating that the person is likely to actually exist). A specific example of the temperature distribution graph will be described later using FIGS. 8(A) and 8(B).

FIG. 2 is a diagram showing an example of the hardware configuration of the human detection device 10 according to the first embodiment. The human detection device 10 is, for example, a computer. The human detection device 10 is a device that can execute the human detection program according to the first embodiment. The human detection device 10 includes a processor 91 and a memory 92 that is a volatile storage device. The human detection device 10 also includes a nonvolatile storage device 93 such as a hard disk drive (HDD) or a solid state drive (SSD), and an interface 94. The memory 92 is, for example, a semiconductor memory such as a RAM (Random Access Memory). Each block of the human detection device 10 shown in FIG. 1 is configured by a processor that executes a human detection program, which is a software program installed from a recording medium or via a communication line and stored in the memory 92, for example.

Each function of the human detection device 10 may be realized by a processing circuit. The processing circuitry may be dedicated hardware or may be a processor 91 executing a person detection program stored in memory 92. The processor 91 may be any one of a processing device, an arithmetic device, a microprocessor, a microcomputer, and a DSP (Digital Signal Processor). When the processing circuit is dedicated hardware, the processing circuit is, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

Incidentally, part of the human detection device 10 may be realized by hardware, and the other part may be realized by software or firmware. In this way, the processing circuit can implement each of the above-mentioned functions using hardware, software, firmware, or any combination thereof.

FIGS. 3A to 3C are diagrams showing examples of information acquired by each block of the human detection device 10. 3(A) shows a visible image D60 taken by the visible image camera 60, FIG. 3(B) shows human shape information D11 extracted from the visible image D60, and FIG. 3(C) shows a visible image D60 taken by the infrared camera 70. The heat generating area information D12 extracted from the photographed thermal image D70 is shown.

For example, when the human detection device 10 is part of a monitoring system that watches over people (e.g., elderly people), the imaging range photographed by the visible image camera 60 may be, for example, the residential area where the person to be monitored lives (e.g. , inside the residence). In this case, the visible image D60 often includes a display device that is a display medium such as a television. Therefore, as shown in FIG. 3(A), the visible image D60 acquired by the visible image camera 60 includes an image of the person 100 actually existing within the shooting range and the image displayed on the screen of the display device 200. A person (that is, a person in the medium that is a person in the display medium) 210 may be included.

The human shape extraction unit 11 extracts human shape information D11 of the person in the visible image D60 from the visible image D60 acquired by the visible image camera 60 using a known posture detection algorithm such as MoveNet. The human shape information D11 includes coordinates of body parts. Further, the human shape information D11 may include coordinates of body parts and reliability of the body parts. Examples of body parts include the head, both shoulders, both elbows, both wrists, hips, the bases of both feet, both knees, and both ankles. If the reliability of a body part cannot be explicitly obtained, the subsequent processing is performed with the reliability of all parts set to the same value (for example, 1). The human shape information D11 also includes a person depicted on a poster who is a person in the medium existing in the visible image D60. Therefore, as shown in FIG. 3(B), in the acquired human shape information D11, the spherical part 101 indicated by the human shape information D11 of the actually existing person 100 and the medium displayed on the screen of the display device 200 The medium internal region 211 indicated by the human shape information D11 of the person 210 may be included. Line segments shown as solid lines in FIG. 3(B) represent parts of the human body by connecting coordinate points of each specific part included in the human shape information D11 with lines. Furthermore, a line segment indicated by a dotted line in FIG. 3B represents a case where the reliability of points at both ends of the line segment is low. In the example of FIG. 3B, since the lower body of the person in the medium 210 displayed on the screen of the display device is not displayed, the reliability of the points representing the lower body is low.

The infrared camera 70 photographs the inside of a residential area where people live, and outputs a thermal image D70 that expresses high-temperature areas with high brightness values. In the thermal image D70, objects expressed as high temperature include living things such as humans and animals, and devices that generate heat (for example, displays such as televisions, home appliances, heating devices, etc.). In addition, objects represented as having low temperatures in the thermal image D70 include non-heat generating objects (for example, posters pasted on walls) including walls and floors.

The heat generating area extracting unit 12 outputs heat generating area information D12 by extracting a set of pixels with high temperature using techniques such as lump detection in the thermal image D70 acquired by the infrared camera 70. The heat generating area information D12 expresses the shapes of people and heat generating objects in the living room, and processing is performed to exclude small heat generating objects and noise values generated by the infrared camera 70. As shown in FIG. 3C, the acquired heat generation area information D12 includes a person area 102 which is heat generation area information D12 of the person 100 who actually exists, and a device area 201 which is the heat generation area information D12 of the display device 200. and is included. In FIG. 3C, the gray portion indicates a portion that is higher in temperature than the white region and has a higher pixel value in the heat generation area information D12.

The person determining unit 14 inputs the human shape information D11 and the heat generation area information D12, and causes the humanlikeness calculation unit 13 to calculate the humanlikeness D13 (D13a, D13b) from the correspondence between the human shape information D11 and the heat generation area information D12. Then, the human shape information D11 is filtered. The calculation of the human-likeness D13 (D13a, D13b) is performed using at least one of the spherical part human-likeness calculating section 13a and the line segment human-likeness calculating section 13b. Note that the details of the method of filtering the human shape information D11 by calculating the human-likeness D13 (D13a, D13b) will be described later. The person determination unit 14 can exclude people in the medium, such as people depicted on posters and people displayed on the screen of the display device 200, from the obtained human shape information D11. As a result, the person determining unit 14 can acquire only the human shape information D11 of the person actually present in the living room.

<<1-2>> Operation FIG. 4 is a flowchart showing the operation of the person determination unit 14 of the person detection device 10. The person determination unit 14 filters each person represented by the person shape information D11 using the person-likeness D13 (D13a, D13b) to remove the person in the medium and identify the person who actually exists. Perform the detection process.

In step S1, the person determination unit 14 estimates (that is, calculates) the distance of the person represented by the human shape information D11 from the visible image camera 60 based on the size of the human shape information D11 in the image. do.

In the next steps S2 to S4, the person determining unit 14 selects a body part in the human shape information D11, and causes the person-likeness calculating unit 13 to calculate the human-likeness D13 (D13a, D13b) of the selected specific part. Examples of body parts of the human shape information D11 to be selected include the head, upper arm, lower arm, hand, torso, thigh, and lower leg. Moreover, for each specific part, a part whose center position is estimated to be outside the area of the visible image D60 may be excluded from the human-likeness evaluation. The processes of steps S2 to S4 are repeatedly executed for each specific part of the person.

The calculation of the human-likeness D13 (D13a, D13b) is performed by the spherical part human-likeness calculation unit 13a and the line segment human-likeness calculation unit 13b, but the calculation of the human-likeness calculation unit 13a for the spherical part or the human-likeness calculation unit for line segment 13b. Calculations may be made using only one. Alternatively, both the human-likeness D13a and D13b may be calculated and the average human-likeness may be used. Alternatively, a selection may be made such as using the one with higher humanness among the two humanness D13a and D13b. Furthermore, it may be determined whether to use either the spherical part human-likeness computing section 13a or the line segment human-likeness computing section 13b depending on the region to be selected. For example, a method may be adopted in which the spherical part human-likeness calculating section 13a is used for the head and hands, and the line segment human-likeness calculating section 13b is used for the limbs and the torso.

In step S5, the person determining unit 14 calculates the human-likeness E of the entire human-shaped information D11 by combining the human-likeness D13a, D13b of each specific part and the reliability of the human-shaped information D11. The person determination unit 14 calculates the average value of the values obtained by multiplying the humanness D13a, D13b of each specific part by the reliability of the human shape information D11, and calculates the humanness E of the whole human shape information D11. It may also be used as Regarding the reliability of the human shape information D11, the person determination unit 14 uses the reliability of the human shape information D11 as it is for a spherical part, and calculates the average reliability of the human shape information D11 at both ends for a line segment part. May be used.

In step S6, the person determining unit 14 determines whether the human-likeness E of the entire human shape information D11 calculated in step S5 is greater than or equal to a threshold value. If E is greater than or equal to the threshold (YES in step S6), the process proceeds to step S7, and if E is less than the threshold (NO in step S6), the process proceeds to step S8.

In step S7, the person determining unit 14 considers the human shape information D11 to be determined to be an actually existing person, and adopts the human shape information D11. In step S8, the person determining unit 14 considers the human shape information D11 to be determined to be a person in the medium, and excludes the human shape information D11. This completes the process in which the person determining unit 14 filters the human shape information D11 by calculating the humanness of each person represented by the human shape information D11.

FIG. 5 is a flowchart showing the operation of the spherical part human-likeness calculation unit 13a in the first embodiment. FIG. 5 shows a method for calculating the human-likeness D13a when it is assumed that the specific part selected by the spherical-part human-likeness calculation unit 13a is a spherical part. Further, FIGS. 6A and 6B are schematic diagrams showing the operation of the spherical part human-likeness calculation unit 13a.

In step S31 of FIG. 5, the spherical part human-likeness calculation unit 13a obtains the center point G of the position of the spherical part of the human shape information D11. If the specific part is a part composed of one mass, such as the head and hand, the coordinates of the center of the part are used as the center point G. Note that if the specific part is a part expressed by a line segment, such as an extremity or a torso, the coordinates of the center of the line segment are used as the center point G.

In the next step S32, the spherical part human-likeness calculation unit 13a cuts out a circular area around the center point G (101a in FIG. 6(A), 201a in FIG. 6(B)) in the heat generation area information D12, and Obtain the temperature value T in the surrounding area. Note that the circular radius of the peripheral area is an area that is slightly larger than the size of the spherical part of the person (101 in FIG. 6(A), 201 in FIG. 6(B)) expressed by the human shape information D11, for example. , a coefficient dependent on the spherical part, is determined based on the resolution of the visible image camera 60 and the distance estimated in step S1.

In the next step S33, the spherical part human-likeness calculation unit 13a calculates the human-likeness from the shape of the distribution of the obtained temperature values T. The graphs in FIGS. 6A and 6B are histogram distributions in which the horizontal axis is the temperature value T and the vertical axis is the number of temperature values (that is, the number of degrees #N). For example, the radius of the circular area determined in step S32 is set to √2 times the radius of the spherical part of the person (the head in the figure) calculated from the distance to the person. In this case, if the ideal state is that the spherical part of the person is perfectly spherical and the entire spherical part is at high temperature, then within the circular area of the surrounding area, the area of the high temperature part originating from the spherical part of the person and the area other than the person (i.e., the area around the spherical part) are the same, and the histogram theoretically consists of two peaks with equal heights, as shown in Figure 6(A). It is thought that it will have a shape like this. In reality, the temperature of the part of the head where hair is located is a little lower, the shoulders, arms, and other heat-generating objects are included in the circle in addition to the head, and the shape of the hands is often far from spherical. Therefore, the height and spread of the two mountain-like shapes are not equal. However, the histogram is considered to have a bimodal shape.

On the other hand, in the case of a person displayed on the screen of a display device, areas other than the spherical part of the person can be uniformly heated by the display device, so the histogram is The graph becomes unimodal. Furthermore, a person displayed on a non-high temperature object such as a poster also shows a single peak, although the temperature is different.

From the above, we assume that the histogram is expressed as the sum of two normal distributions, and use the mean and variance, which are the parameters of each normal distribution, when fitting the normal distribution, as a numerical value representing humanness. It is possible that From now on, of the two normal distributions obtained through fitting, the normal distribution with a small variance and a sharp peak will be referred to as the first normal distribution, and the normal distribution with a large variance and a gentle peak will be referred to as the second normal distribution. . At this time, if bimodality is exhibited, the values of the first normal distribution and the second normal distribution will be close to each other, and if unimodality is exhibited, the values of the first normal distribution and the second normal distribution will be close to each other. It is conceivable that the values will be significantly different. Therefore, for example, the value obtained by calculating (variance of first normal distribution/dispersion of second normal distribution) may be used as the humanness D13a of the spherical part. The above completes the process of calculating the human-likeness when the part selected by the spherical-part human-likeness calculation unit 13a is assumed to be spherical.

FIG. 7 is a flowchart showing the operation of the line segment part human-likeness calculation unit 13b to determine the human-likeness D13b. FIG. 7 shows a method for calculating the human-likeness D13b when it is assumed that the specific site selected by the line-segment site human-likeness calculation unit 13b is a linear segment site. Moreover, FIGS. 8A and 8B are schematic diagrams showing the operation of the line segment part human-likeness calculation unit 13b.

In step S34 of FIG. 7, the line segment part human-likeness calculation unit 13b acquires parameters expressing the selected line segment part as human shape information D11. Line segments indicated by thick solid lines in FIGS. 8(A) and 8(B) correspond to line segment sites. Specifically, a line segment part is specified by coordinate values representing both ends of the line segment. For example, if the right upper arm is selected as a line segment part, the coordinate values of the right shoulder and the coordinates of the right elbow are specified. The combination with the value is a parameter that expresses the line segment site.

In step S35, the line segment part human-likeness calculation unit 13b calculates a line segment that is perpendicular to the selected line segment (for example, a line segment that passes through the center position of the selected line segment and is perpendicular to the selected line segment). . The thin solid lines in FIGS. 8(A) and 8(B) correspond to this. The length of this vertical line segment is determined, for example, by the selected linear site, the resolution of the visible image camera 60, and the distance estimated from step S1.

In step S36, the line segment part human-likeness calculation unit 13b acquires the temperature value T from the heat generation area information D12 for the coordinates on the vertical line segment calculated in step S35, and from the temperature value T on the line segment. Calculate your humanity. In the temperature distribution graphs in FIGS. 8(A) and 8(B), the horizontal axis is the value (i.e., position) in the horizontal axis direction within the image on the line segment, and the vertical axis is the temperature value T. It is a graph for obtaining changes in temperature values.

Here, as an ideal state, if a line segment part of a person does not overlap with other line segment parts of the same person, the change in temperature value on the line segment will be a single peak, as shown in FIG. 8(A). This is considered to be a graph that shows the In reality, the base of the mountain may spread or the temperature may not decrease at the end of the line because it overlaps with other parts of the same person or with other heat-generating objects. However, in most cases, at least at one end of the line segment, it descends. On the other hand, in the case of a person in a medium displayed on the screen of a display device, areas other than the line segments of the person may also become hot due to the display device, so the temperature value on the line segment may be The shape of the change is a uniform graph, as shown in the lower right corner of FIG. 7(B). Similarly, a uniform graph is shown for a person displayed on a non-high temperature object such as a poster, although the temperature is different.

From the above, we assume that changes in temperature values on a line segment are expressed by one normal distribution as numerical values representing human-likeness, and we calculate the mean and variance, which are the parameters of the normal distribution when performing normal fitting. It is possible to use it. At this time, if a normal distribution is fitted to a uniform graph, the value of variance may be a large value exceeding 10,000. On the other hand, in the case of a graph exhibiting unimodal characteristics, the value of variance is likely to be a small value of 10 or less. From this, for example, the Min function is a function that selects the smaller of the given arguments, the variance value with K as the reference (10 in the above example), Min(1,√(K/[normal distribution The value obtained by calculating the variance])) may be used as the "humanity of the line segment part." The above completes the human-likeness calculation process when it is assumed that the body part selected by the line-segment-part human-likeness calculation unit 13b is a line segment.

<<1-3>> Effects As explained above, according to the first embodiment, the human detection device 10 uses the human shape information D11 extracted from the visible image D60 obtained from the visible image camera 60 to By filtering using the heat generation area information D12 extracted from the thermal image D70, it is possible to exclude the human shape information D11 corresponding to the person in the medium. Therefore, according to the first embodiment, by excluding people in the medium displayed on posters, televisions, etc. that are present in the visible image D60, it is possible to detect people who actually exist with high accuracy. Furthermore, according to the first embodiment, it is also possible to take the posture of an actually existing person.

<<2>> Embodiment 2
FIG. 9 is a block diagram schematically showing the configuration of a person detection device 20 and a person detection system 2 according to the second embodiment. In FIG. 9, components that are the same as or correspond to those shown in FIG. 1 are given the same reference numerals as those shown in FIG. The person detection device 20 according to the second embodiment includes a device usage information acquisition unit 21 that acquires device usage information indicating the usage status of one or more devices 300 that generate heat existing within a shooting range; A device learning unit 22 that learns the relationship between the heat generating area information D12, and a heat generating area information D23 corrected by excluding a thermal area caused by heat generation of the device 300 from the heat generating area information D12 when the device 300 is in use. This embodiment differs from the human detection device 10 according to the first embodiment in that it further includes a device exclusion section 23 that outputs the following.

The device usage information acquisition unit 21 acquires device usage information indicating the usage status of one or more devices 300 existing within the shooting range. The device usage information is, for example, when the device 300 can transmit power-on or power-off information using a communication method such as HEMS (Home Energy Management System), which is a system for managing and controlling home appliances, etc. Information on whether the device 300 is powered on or off can be collected from the HEMS. In addition, if a smart meter that is a power meter with a communication function (i.e., a power consumption meter) and the human detection device 20 can communicate with each other, the device 300 can be turned on or off based on the power usage information of the device 300. It is possible to obtain the status of

The device learning unit 22 compares the heat generation area information D12 of the heat generation area extraction unit 12 with the device usage information of the device usage information acquisition unit 21, and determines if any of the electrical devices is in use in the device usage information. Next, the relationship between the device and the area showing high temperature in the heat generation area information D12 is learned. Immediately after the person detection device 20 is installed, this learning of relationships is not completed, but by continuing to learn to a certain extent, it becomes accurate like the known technology of image background subtraction. It can be expected that a strong association will be made. Note that background subtraction is a process of extracting objects that do not exist in the previously acquired image by comparing the observed image and the previously acquired image.

The device exclusion unit 23 corrects by excluding the heat area associated with the device 300 from the heat generation area information D12 of the heat generation area extraction unit 12 when one or more devices 300 existing within the imaging range is in use. The generated heat generating area information D23 is acquired. The person determining unit 14, the spherical part human-likeness calculation unit 13a, and the line segment human-likeness calculation unit 13b of the second embodiment obtain benefits by excluding the thermal region associated with the device 300 output by the device exclusion unit 23. The corrected heat generating area information D23 is used. Therefore, when the spherical part human-likeness calculation unit 13a and the line segment human-likeness calculation unit 13b calculate the human-likeness of each specific part of the human shape information D11, they ignore the heat generation area information of devices other than humans that generate heat. It can be calculated by Therefore, the human-likeness calculation unit 13 can more accurately calculate the human-likeness.

As explained above, according to the second embodiment, when the human detection device 20 calculates the humanness of the human shape information D11, it detects a device that generates heat that is unrelated to an actually existing person. Since the configuration is such that heat generation area information can be ignored, more accurate human-likeness calculations can be performed. Therefore, according to the second embodiment, by excluding the person in the medium who is present in the visible image D60, the person who actually exists can be detected with high accuracy.

Note that the second embodiment is the same as the first embodiment except for the above.

<3> Embodiment 3
FIG. 10 is a block diagram schematically showing the configuration of a person detection device 30 and a person detection system 3 according to the third embodiment. In FIG. 10, components that are the same as or correspond to those shown in FIG. 1 are given the same reference numerals as those shown in FIG. The person detection device 30 according to the third embodiment includes a device usage information input unit 31 through which a user inputs usage information of one or more devices 300 that generate heat that are present within the imaging range, The device learning unit 22 learns the relationship between the area information D12 and the corrected heat generation obtained by excluding the heat area caused by the heat generation of the device 300 from the heat generation area information D12 when the device 300 is in use. This embodiment differs from the human detection device 10 according to the first embodiment in that it further includes a device exclusion section 23 that outputs area information D23. Further, in the human detection device 30 according to the third embodiment, the user performs an input operation on the device usage information acquisition unit 21 of FIG. This differs from the person detection device 20 according to the second embodiment in that it is replaced with a device usage information input section 31. However, the human detection device 30 according to the third embodiment may include a device usage information input section 31 in addition to the configuration of the human detection device 20 in FIG. 9 .

When one or more devices 300 existing within the shooting range are in use, the device usage information input unit 31 inputs the device usage information by the user inputting the fact that the device 300 is in use. is transmitted to the device learning section 22. This may be input using an operation unit (for example, a button) attached to the housing of the human detection device 30, or the information may be input via a communication terminal such as a mobile terminal owned by the user of the human detection device 30. It may also be in the form of input. The operations of the device learning section 22 and the device exclusion section 23 are explained in the second embodiment.

Although the device usage information input unit 31 functions effectively even in a form where the user inputs information to the device 300 during actual life, it is possible to turn on the device 300 one by one immediately after installing the human detection device 10, More effectiveness can be achieved by providing a phase in which the device learning section 22 learns the association with the region indicating high temperature in the heat generation region information D12 of the heat generation region extraction section 12. Further, when it is detected that a new device 300 has been purchased and installed in the living room, the device 300 is turned on, and the device learning section 22 detects a high temperature using the heat generation area information D12 of the heat generation area extraction section 12. It can also be more effective by providing a phase in which the user learns the relationship with the indicated area.

As explained above, according to the third embodiment, the user can input the usage information of the device 300, so that even when the device in the living room cannot transmit the usage status of the device by communication, it is possible to detect a person. When the device 30 calculates the human-likeness of the human shape information D11, it becomes possible to ignore heat generation area information of devices that generate heat that are unrelated to the person. Therefore, the human detection device 30 can calculate the human-likeness with higher accuracy. According to the third embodiment, by excluding the person in the medium that exists in the visible image D60, it is possible to detect the person who actually exists with high accuracy.

Note that, except for the above, the third embodiment is the same as the first or second embodiment.

<4> Embodiment 4
FIG. 11 is a block diagram schematically showing the configuration of a person detection device 40 and a person detection system 4 according to the fourth embodiment. In FIG. 11, components that are the same as or correspond to those shown in FIG. 1 are given the same reference numerals as those shown in FIG. The human detection device 40 according to the fourth embodiment includes a communication unit 41 as a first communication unit that communicates with the visible image camera 60 by wire or wirelessly, and a second communication unit that communicates with the infrared camera 70 by wire or wirelessly. The visible image D60 taken by the visible image camera 60 mounted on the camera-equipped device 61 is input to the human shape extraction section 11 via the communication section 41, and the camera-equipped device The human detection device 10 is different from the human detection device 10 according to the first embodiment in that a thermal image D70 taken by an infrared camera 70 mounted on the device 71 is input to the heat generation area extraction unit 12 via the communication unit 42.

In the fourth embodiment, although the visible image camera 60 exists outside the human detection device 40, the human detection device 10 acquires the visible image D60 via the communication unit 41, and similarly to the first embodiment, the human detection device 10 acquires the visible image D60. It is possible to calculate the person-likeness D13 (D13a, D13b) of the information D11 and perform processing to exclude people in the medium. Furthermore, the human detection device 40 according to the fourth embodiment is provided with information indicating the relationship between the coordinates of the human shape information D11 and the heat generating area information D12 in advance, so that it is possible to link the coordinates of both. Can be done.

As explained above, according to the fourth embodiment, the same effects as those explained in the first embodiment can be obtained.

Note that the fourth embodiment is the same as the first embodiment except for the above. Furthermore, it is also possible to apply the communication units 41 and 42 described in the fourth embodiment to the second or third embodiment. Furthermore, the human detection device 40 may have a built-in one of a visible image camera 60 and an infrared camera 70, and may be configured to communicate with the other by wire or wirelessly via a communication unit.

<<5>> Modifications The person detection devices 10, 20, 30, 40 and the person detection systems 1 to 4 according to Embodiments 1 to 4 are installed in facilities or homes for the purpose of monitoring elderly people, for example. The present invention can be applied to a monitoring system that notifies a predetermined notification destination (for example, a facility manager or a family member) of the occurrence of an abnormality when an abnormality occurs in the behavior of an elderly person. By employing the person detection devices 10, 20, 30, and 40 and the person detection systems 1 to 4 that can detect an actually existing person with high accuracy, false notifications of abnormality occurrences can be reduced. By using the function of the human-likeness calculation unit 13 of the human detection devices 10, 20, 30, and 40, it is possible to detect the posture of the person, and therefore it is also possible to notify the person's condition when an abnormality occurs. .

1 to 4 Human detection system, 10, 20, 30, 40 Human detection device, 11 Human shape extraction unit, 12 Heat generation area extraction unit, 14 Human determination unit, 13 Human-likeness calculation unit, 13a Spherical part human-likeness calculation unit, 13b Line segment human-likeness calculation unit, 21 Equipment usage information acquisition unit, 22 Equipment learning unit, 23 Equipment exclusion unit, 31 Equipment usage information input unit, 41, 42 Communication unit, 60 Visible image camera, 70 Infrared camera, 61 Camera installed Equipment, 71 Camera-equipped equipment, 101 Spherical part, 101a Surrounding area, 300 Equipment, D11 Human shape information, D12 Heat generation area information, D13a Humanity, D13b Humanity, D23 Corrected heat generation area information, D60 Visible image (image) , D70 thermal image, D14 judgment result.

Claims (14)

a human shape extraction unit that receives an image obtained by photographing a photographing range with a visible image camera that photographs an image in at least a wavelength range of visible light, and extracts human shape information indicating the shape of a person in the image;
a heat generating area extraction unit that receives a thermal image taken by an infrared camera and extracts heat generating area information indicating a region with a large amount of heat generation in the thermal image;
Selecting a predetermined specific shaped part of the human shape, and determining whether the selected specific shaped part actually exists within the imaging range based on the human shape information and the heat generation area information. a human-likeness calculation unit that calculates human-likeness, which is a numerical value representing the certainty that it is a human part;
Based on the human-likeness calculated for the selected part of the specific shape, it is possible to determine whether the shape of the person indicated by the human shape information is the shape of the person who actually exists, or whether it is displayed on the medium or not. a person determination unit that determines whether the shape of the person in the medium is the depicted person;
A person detection device characterized by having. The human-likeness calculation section includes a spherical part human-likeness calculation section that selects a spherical spherical part as the part of the specific shape and calculates the human-likeness based on the spherical part and a surrounding area surrounding the spherical part. The person detection device according to claim 1, characterized in that it has. The spherical part human-likeness calculation unit creates a histogram in which the horizontal axis is a temperature value and the vertical axis is a degree with respect to the surrounding area, and calculates the human-likeness based on the shape of the histogram. 2. The human detection device according to 2. The spherical part human-likeness calculation unit sets the human-likeness when the histogram shows a bimodal distribution to a higher value than the human-likeness when the shape of the histogram shows a unimodal distribution. The human detection device according to claim 3, characterized in that: The human-likeness calculating unit is characterized by having a line-segment-part human-likeness calculating unit that selects a linear segment site as the part of the specific shape and calculates the human-likeness based on the line segment site. The human detection device according to claim 1. The line segment part human-likeness calculation unit creates a temperature distribution graph with respect to the line segment part in which the horizontal axis is a position perpendicular to the line segment part and the vertical axis is a temperature value, and the shape of the temperature distribution graph is The person detection device according to claim 5, wherein the person-likeness is calculated based on the person-likeness. The line segment part human-likeness calculation unit sets the human-likeness when the temperature distribution graph shows a unimodal distribution to a higher value than the humanlikeness when the temperature distribution graph does not show a unimodal distribution. The human detection device according to claim 6, wherein the human detection device is set to . The human-likeness calculation unit is
a spherical part human-likeness calculation unit that selects a spherical spherical part as the specific-shaped part and calculates a first human-likeness based on the spherical part;
a line segment part person-likeness calculation unit that selects a linear line segment part as the part of the specific shape and calculates a second person-likeness based on the line segment part;
has
The person determining unit determines whether the shape of the person indicated by the human shape information is the shape of the person who actually exists, based on the first person-likeness and the second person-likeness, and determines whether the shape of the person indicated by the human shape information is the shape of the person who actually exists The person detection device according to claim 1, wherein the person detection device determines whether the shape is that of an internal person. a device usage information acquisition unit that acquires usage information of devices that generate heat installed within the photographing range;
a device learning unit that learns the relationship between the usage information and the heat generation area information;
further comprising a device exclusion unit that, when the device is in use, excludes information on a heat generating part caused by the device from the heat generating area information to generate corrected heat generating area information;
The human-likeness calculation unit calculates the human-likeness based on the human-like shape information and the corrected heat-generating region information. Device. an equipment usage information input unit that acquires usage information of equipment that generates heat installed within the imaging range;
a device learning unit that learns the relationship between the usage information and the heat generation area information;
further comprising a device exclusion unit that, when the device is in use, excludes information on a heat generating part caused by the device from the heat generating area information to generate corrected heat generating area information;
The person detection according to any one of claims 1 to 8, wherein the person-likeness calculation unit calculates the person-likeness based on the human shape information and the corrected heat generation area information. Device. a first communication unit that receives the image transmitted from the visible image camera;
a second communication unit that receives the thermal image transmitted from the infrared camera;
It further has
The human shape extraction unit receives the image via the first communication unit,
The person detection device according to any one of claims 1 to 10, wherein the heat generation area extraction unit receives the thermal image via the second communication unit. A human detection device according to any one of claims 1 to 10,
the visible image camera;
the infrared camera;
A person detection system characterized by having. A person detection method performed by a person detection device, the method comprising:
receiving an image obtained by photographing a photographing range with a visible image camera that photographs an image in at least a wavelength range of visible light, and extracting human shape information indicating the shape of a person in the image;
receiving a thermal image taken by an infrared camera, and extracting heat generation area information indicating a region with a large amount of heat generation in the thermal image;
Selecting a predetermined specific shaped part of the human shape, and determining whether the selected specific shaped part actually exists within the imaging range based on the human shape information and the heat generation area information. a step of calculating human-likeness, which is a numerical value representing the certainty that it is a human part;
Based on the human-likeness calculated for the selected part of the specific shape, it is possible to determine whether the shape of the person indicated by the human shape information is the shape of the person who actually exists, or whether it is displayed on the medium or not. determining whether the shape of the person in the medium is the depicted person;
A person detection method characterized by having the following. receiving an image obtained by photographing a photographing range with a visible image camera that photographs an image in at least a wavelength range of visible light, and extracting human shape information indicating the shape of a person in the image;
receiving a thermal image taken by an infrared camera, and extracting heat generation area information indicating a region with a large amount of heat generation in the thermal image;
Selecting a predetermined specific shaped part of the human shape, and determining whether the selected specific shaped part actually exists within the imaging range based on the human shape information and the heat generation area information. a step of calculating human-likeness, which is a numerical value representing the certainty that it is a human part;
Based on the human-likeness calculated for the selected part of the specific shape, it is possible to determine whether the shape of the person indicated by the human shape information is the shape of the person who actually exists, or whether it is displayed on the medium or not. determining whether the shape of the person in the medium is the depicted person;
A human detection program that causes a computer to execute.

Images