WO2019220752A1 - Automatic tracking and recording system and recording control device - Google Patents

Abstract

Provided is an automatic tracking and recording system with which it is possible to track a target with high accuracy and capture an image of the target. This automatic tracking and recording system 1 is provided with: a first camera 10 for detecting an object in three dimensions, as well as specifying and tracking a tracking target, and generating and outputting position information pertaining to the tracking target; a recording control device for generating imaging control information on the basis of the position information pertaining to the tracking target and outputting the generated information; and a second camera 20 for controlling the drive mechanism of an imaging unit on the basis of the imaging control information ,generating a captured image, and outputting the captured image to the recording control device.

Description

Automatic tracking recording system and recording control device

The present invention relates to an automatic tracking recording system and a recording control device.

In recent years, video distribution systems that capture lessons and distribute videos in real time, and video distribution systems that capture and record classes and distribute recorded videos in response to viewing requests from students have become widespread. It's getting on. In such a moving image distribution system, a PTZ camera that automatically adjusts pan, tilt, and zoom and automatically tracks and captures imaging targets such as teachers and students is used. Panning is a function that changes the direction of the camera in the horizontal direction. Tilt is a function that changes the orientation of the camera up and down. Zoom is a function for changing the angle of view to a telephoto or wide angle.

Patent Document 1 discloses a moving body imaging apparatus using a PTZ camera. When detecting an intruder, the moving body imaging device disclosed in Patent Document 1 adjusts the pan, tilt, and zoom of the PTZ camera so that the detected intruder is captured near the center of the imaging screen.

JP 2011-77617 A

The moving body imaging device disclosed in Patent Document 1 tracks and captures an intruder, and predicts the position of the intruder based on the difference image and the movement (moving direction and speed) of the intruder. The pan and tilt are adjusted based on the predicted value. Thus, since it was based on a predicted value, when tracking of the tracking target failed, there was a problem that it was impossible to capture and capture the tracking target.

The present invention has been made in view of the above circumstances, and an object thereof is to provide an automatic tracking recording system and a recording control apparatus capable of tracking and imaging an object with high accuracy.

In order to solve the above-mentioned problem, the automatic tracking recording system of the present invention includes:
A first camera that detects an object in three dimensions, identifies and tracks a tracking target, and generates and outputs position information of the tracking target;
A recording control device that generates and outputs imaging control information based on the position information of the tracking target;
And a second camera that generates a captured image by controlling a drive mechanism of the imaging unit based on the imaging control information, and outputs the captured image to the recording control device.

The recording control device of the present invention is
A recording control device communicably connected to the first camera and the second camera,
A position information acquisition unit for detecting the object in three dimensions, specifying and tracking the tracking target, generating the positional information of the tracking target and outputting the positional information from the first camera;
An imaging control information generating unit that generates imaging control information for controlling a driving mechanism of the imaging unit in the second camera based on the position information, and outputs the imaging control information to the second camera;
A captured image acquisition unit configured to acquire a captured image generated by the imaging unit of the second camera based on the imaging control information.

According to such a configuration, the first camera that can detect an object in three dimensions generates position information of the tracking target with high accuracy, and the recording control device uses the second camera based on the position information. Imaging control information for controlling the driving mechanism is generated and output to the second camera. For this reason, the second camera can control the drive mechanism based on the imaging control information and track the target with high accuracy and perform imaging. Thereby, the captured image which caught the tracking object with high precision can be obtained.

In the configuration of the present invention, the first camera generates and outputs the behavior information of the tracking target,
The recording control device preferably generates the imaging control information based on the position information and the behavior information of the tracking target.

According to such a configuration, imaging control information that matches the behavior of the tracking target is generated. For this reason, the second camera can image the tracking target by controlling the drive mechanism in accordance with the behavior of the tracking target. Thereby, it is possible to obtain a captured image that is adjusted and imaged according to the behavior of the tracking target.

In the above-described configuration of the present invention,
The first camera is a camera capable of measuring a three-dimensional distance to an object in an imaging space;
The second camera is preferably a camera including at least the driving mechanism for adjusting pan, tilt, and zoom.

According to such a configuration, a captured image that captures the tracking target with high accuracy can be obtained.

According to the present invention, it is possible to provide an automatic tracking recording system and a recording control device that can track and image an object with high accuracy.

1 is an overall view showing a configuration example of an automatic tracking recording system according to a first embodiment of the present invention. 2 is a diagram illustrating a configuration example of a first camera, where (a) is a block diagram and (b) is a diagram illustrating a hardware configuration example. FIG. 2 is a diagram illustrating a configuration example of a second camera, where (a) is a block diagram and (b) is a diagram illustrating a hardware configuration example. FIG. 2 is a diagram illustrating a configuration example of a recording control apparatus, where (a) is a block diagram, (b) is a diagram illustrating a hardware configuration example, and (c) is a block diagram of a control unit. FIG. It is a figure which shows the example of installation similarly, (a) is the figure seen from the upper surface, (b) is the figure seen from the side surface. It is a flowchart which shows an operation example similarly. 4 is a flowchart showing an example of automatic tracking recording processing. It is a figure which shows the example of a coordinate transformation similarly. It is a flowchart which shows another example of an automatic tracking video recording process similarly. It is a figure which shows the example of the action information produced | generated with the automatic tracking video recording system which concerns on the 2nd Embodiment of this invention. 4 is a flowchart showing an example of automatic tracking recording processing. It is a figure which shows the specific example of an action state and imaging control information. 4 is a flowchart for explaining generation of imaging control information.

Hereinafter, in order to explain the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is an overall view showing a configuration example of an automatic tracking recording system 1 according to the first embodiment. Hereinafter, a case where the automatic tracking recording system 1 automatically tracks a person will be described as an example. However, the tracking target is not limited to a person, and may be a mobile object or the like.
The automatic tracking recording system 1 includes a first camera 10, a second camera 20, and a recording control device 30. The first camera 10 and the second camera 20 are communicably connected to the recording control device 30 via the network 50. The network 50 is realized by the Internet, a wired LAN (Local Area Network), a wireless LAN (Local Area Network), a WAN (Wide Area Network), or a combination thereof. The first camera 10 includes a communication unit 13 described later. The second camera 20 includes a communication unit 24 described later. The recording control device 30 includes a communication unit 34 described later. Hereinafter, transmission / reception of information in the automatic tracking recording system 1 is performed via the communication unit 13, the communication unit 24, and the communication unit 34.

FIG. 2A is a block diagram illustrating a configuration example of the first camera 10.
The first camera 10 includes an information acquisition unit 11, an information control unit 12, and a communication unit 13. The information acquisition unit 11 includes a light source unit 11a, a light receiving unit 11b, a drive circuit 11c, and a signal processing circuit 11d.
The first camera 10 is a camera capable of measuring a three-dimensional distance to an object (the surface of the object) in the imaging space, and is, for example, a stereo camera or a 3D sensor camera.
A stereo camera is a camera that can generate parallax data of an image, measure the distance from the camera to a target object, and detect the target object in three dimensions. The stereo camera can detect the position, size, unevenness information and the like of the target object (three-dimensional object). The stereo camera can also generate a two-dimensional image.
The 3D sensor camera is, for example, a camera equipped with a three-dimensional distance image sensor. The 3D sensor camera uses TOF (Time
This is a camera that can measure the three-dimensional distance information (depth, height, shape, positional relationship, etc.) to the target object in the imaging space and detect the target object in three dimensions by the Of Flight method.
Further, the first camera 10 has a tracking function for tracking the same detected object (such as a person).

Hereinafter, a case where a 3D sensor camera is used as the first camera 10 and three-dimensional distance information is measured (phase difference method) will be described. The light source unit 11a includes, for example, at least one light source element. The light source element is, for example, an infrared LED. The drive circuit 11c outputs a light projection signal to the light source unit 11a based on the request acquired from the recording control device 30. When the light source unit 11a acquires the light projection signal from the drive circuit 11c, the light source unit 11a emits a ranging wave that is infrared pulsed light, for example.
The light receiving unit 11b is, for example, a plurality of light receiving elements arranged in a two-dimensional array. The light receiving element is, for example, a CMOS or a CCD. The light received from the light source unit 11a and reflected by the object (that is, reflected light) is incident on each light receiving element of the light receiving unit 11b. Each light receiving element of the light receiving unit 11b performs photoelectric conversion, and outputs a light receiving signal indicating the intensity of incident light to the element to the signal processing circuit 11d.

The signal processing circuit 11d acquires a light projection signal from the drive circuit 11c and acquires a light reception signal from the light receiving unit 11b. The signal processing circuit 11d obtains a phase difference φ between the light projection signal and the light reception signal for each of the plurality of light receiving elements, and based on the obtained phase difference φ, from the light source unit 11a to each reflection point (person). Find the spatial distance. The signal processing circuit 11d outputs a collection of spatial distances obtained for each light receiving element to the information control unit 12 as distance image information at a predetermined frame rate. The distance image information has the spatial distance as a pixel value.
The information control unit 12 analyzes the distance image information acquired from the signal processing circuit 11d and detects a tracking target (person). The memory of the first camera 10 stores information (feature information) indicating features for each object type. The feature information may be acquired from the recording control device 30. The information control unit 12 sequentially assigns an ID to the detected person and tracks the person to whom the ID is assigned. The information control unit 12 identifies an ID corresponding to a predetermined condition as a tracking target ID, and generates position information of a person to whom the tracking target ID is assigned. The information control unit 12 transmits the generated position information to the recording control device 30.

FIG. 2B is a diagram illustrating a hardware configuration example of the first camera 10.
The CPU 201 controls the overall operation of the first camera 10 via the bus. The ROM 202 is a non-volatile memory that stores a control program necessary for the CPU 201 to execute processing. The RAM 203 is used as a temporary storage area such as a main memory and a work area for the CPU 201. That is, the CPU 201 loads various programs and the like from the ROM 202 to the RAM 203 when executing the processing, and implements various functional operations by executing the programs and the like. The input unit 204 is a button or the like that receives an operation input. The communication unit 205 corresponds to the communication unit 13 in FIG. The light source unit 206 corresponds to the light source unit 11a in FIG. The light receiving unit 207 corresponds to the light receiving unit 11b in FIG. The drive circuit 208 corresponds to the drive circuit 11c in FIG. The signal processing circuit 209 corresponds to the signal processing circuit 11d in FIG.
Some or all of the functions of each element of the first camera 10 shown in FIG. 2A are realized by the CPU 201 executing a program. However, at least some of the elements of the first camera 10 shown in FIG. 2A may operate as dedicated hardware. In that case, the dedicated hardware operates based on the control of the CPU 201.

FIG. 3A is a block diagram illustrating a configuration example of the second camera 20.
The second camera 20 includes an imaging unit 21, an imaging control unit 22, an encoding unit 23, a communication unit 24, and the like. The second camera 20 is, for example, a PTZ (Pan Tilt Zoom) camera. A PTZ camera is a camera that can adjust at least pan, tilt, and zoom. Hereinafter, although the case where a PTZ camera is used is demonstrated, arbitrary cameras can be used if the same visual field control is possible.

The imaging unit 21 includes a lens, an imaging element, a drive mechanism, and the like. The image sensor has a CMOS (Complementary
Metal Oxide Semiconductor) and CCD (Charge Coupled Device). Examples of the drive mechanism include a pan drive mechanism, a tilt drive mechanism, and a zoom drive mechanism. The pan driving mechanism is a mechanism that changes the orientation (imaging region) of the imaging unit 21 in the horizontal direction (left-right direction). The tilt drive mechanism is a mechanism that changes the orientation (imaging region) of the imaging unit 21 in the vertical direction (up and down direction). The zoom drive mechanism is a mechanism that can change the imaging region of the imaging unit 21 in the zoom direction and changes the angle of view of the imaging unit 21 to a telephoto or wide angle. Examples of zoom include optical zoom and digital zoom. The imaging unit 21 generates a captured image and outputs the generated captured image to the encoding unit 23.

The imaging control unit 22 controls imaging of the imaging unit 21 based on information acquired from the recording control device 30. The encoding unit 23 compresses and encodes the captured image acquired from the imaging unit 21 to generate image data, and outputs the generated image data to the recording control device 30.

FIG. 3B is a diagram illustrating a hardware configuration example of the second camera 20.
The CPU 301 controls the overall operation of the second camera 20 via the bus. The ROM 302 is a non-volatile memory that stores a control program necessary for the CPU 301 to execute processing. A RAM 303 is used as a temporary storage area such as a main memory and work area of the CPU 301. That is, the CPU 301 loads various programs and the like from the ROM 302 to the RAM 303 when executing the processing, and implements various functional operations by executing the programs and the like. The imaging unit 304 corresponds to the imaging unit 21 in FIG. The input unit 305 is a button or the like that receives an operation input. The communication unit 306 corresponds to the communication unit 24 in FIG.
Some or all of the functions of each element of the second camera 20 shown in FIG. 3A are realized by the CPU 301 executing a program. However, at least some of the elements of the second camera 20 shown in FIG. 3A may operate as dedicated hardware. In that case, the dedicated hardware operates based on the control of the CPU 301.

FIG. 4A is a block diagram illustrating a configuration example of the recording control device 30.
The recording control device 30 includes a control unit 31, a storage unit 32, an input unit 33, a communication unit 34, and the like. The control unit 31 controls automatic tracking recording processing. The input unit 33 is a part that receives an operation input by a user. The input unit 33 is, for example, a touch panel or a hardware key. Upon receiving an operation input, the input unit 33 outputs a request (signal) to the control unit 31. The request includes, for example, a recording request. Although FIG. 1 shows the case where the input unit 33 is integrated with the recording control device 30, the input unit 33 may be separate from the recording control device 30.
The storage unit 32 is a part capable of storing information. The storage unit 32 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. The storage unit 32 stores captured images and the like acquired by the recording control device 30. Although FIG. 1 shows a case where the storage unit 32 is built in the recording control device 30, the storage unit 32 may be externally attached to the recording control device 30.

FIG. 4B is a diagram illustrating a hardware configuration example of the recording control device 30.
The CPU 401 comprehensively controls the operation in the recording control device 30 via the bus. The ROM 402 is a non-volatile memory that stores a control program necessary for the CPU 401 to execute processing. The RAM 403 is used as a temporary storage area such as a main memory and a work area for the CPU 401. That is, the CPU 401 implements various functional operations by loading a program or the like necessary from the ROM 402 into the RAM 403 and executing the program. The ROM 402 or RAM 403 corresponds to the storage unit 32 in FIG. The input unit 404 corresponds to the input unit 33 in FIG. The communication unit 405 corresponds to the communication unit 34 in FIG.
Part or all of the functions of each element of the recording control device 30 shown in FIG. 4A is realized by the CPU 401 executing a program. However, at least some of the elements of the recording control apparatus 30 shown in FIG. 4A may operate as dedicated hardware. In that case, the dedicated hardware operates based on the control of the CPU 401.
FIG. 4C is a block diagram illustrating a configuration example of the control unit 31. The control unit 31 includes a position information acquisition unit 31a, a position information correction unit 31b, an imaging control information generation unit 31c, a captured image acquisition unit 31d, and the like.

FIG. 5A and FIG. 5B are diagrams showing an example when the automatic tracking recording system 1 according to the first embodiment is installed in a classroom A. FIG. 5A is a view of the classroom A as viewed from above. FIG.5 (b) is the figure which looked at the classroom A from the side.
As shown in FIGS. 5 (a) and 5 (b), a substantially rectangular platform 2 is arranged in the vicinity of one of the four side walls B surrounding the classroom A, as viewed from above. Yes. The platform 2 is a trapezoidal part formed so as to be one step higher than the floor surface. The platform 2 is arranged such that one side surface in the longitudinal direction contacts the side wall B.
On the upper surface of the platform 2, the teacher 3 is arranged at a position away from the side wall B by a predetermined distance. Note that the teaching desk 3 may be arranged directly on the floor without using the teaching platform 2. The teacher 3 is used by teachers when conducting classes. In the classroom A, a plurality of seats 4 are arranged side by side so as to face the teaching desk 3. Students sit on the seat 4.
Hereinafter, a case where the automatic tracking recording system 1 is installed in the classroom A and teachers and students use the classroom A will be described. However, the classroom A may be a facility such as a hall, a gymnasium, or a theater. In that case, lecturers and spectators will use the facility.
Also, in the following, the teaching platform 2 side of classroom A, that is, the front side viewed from the student seated in seat 4, is referred to as “front”, the rear side is “rear”, the right side is “right”, and the left side is “left " Further, the ceiling side of classroom A is “upper” and the floor side is “lower”.

5A and 5B show an example in which the first camera 10 and the second camera 20 are installed in the classroom A at two locations. However, the number of installed first cameras 10 and second cameras 20 is not limited to this, and may be appropriately determined according to the size of the facility.
One first camera 10 (first camera 10A) and one second camera 20 (second camera 20A) have a ceiling in the center in the front-rear direction of classroom A, and the camera lens is in front of the classroom. It is arranged to face the side. The first camera 10A and the second camera 20A mainly capture an area on the front side (teacher side) of the classroom. The other first camera 10 (first camera 10B) and the other second camera 20 (second camera 20B) have a ceiling on the front side of the classroom A and a camera lens on the rear side of the classroom A. It is installed to face. The 1st camera 10B and the 2nd camera 20B mainly image the area (student side) on the back side of the classroom. The recording control device 30 is installed in the vicinity of the platform 2 in front of the classroom A.

Further, as shown in FIG. 4A, a display unit 40 on which a captured image output from the recording control device 30 is displayed may be further provided. The display unit 40 is, for example, a liquid crystal display. When the display unit 40 is installed in a separate room from the classroom A, the state in the classroom A can be monitored in the separate room. Further, the captured image stored in the storage unit 32 may be displayed on the display unit 40.

Next, an operation example of the automatic tracking recording system 1 will be described using the flowchart shown in FIG.
When the recording control device 30 acquires a recording request via the input unit 33, the recording control device 30 starts the system (step ST1).
Next, the recording control device 30 starts detection by the first camera 10 (step ST2). Specifically, when the drive circuit 11c of the first camera 10 acquires a detection request from the recording control device 30, the drive circuit 11c outputs a projection signal to the light source unit 11a and the signal processing circuit 11d. The light receiving unit 11b outputs a light reception signal indicating the intensity of incident light to each light receiving element to the signal processing circuit 11d. The signal processing circuit 11d calculates a spatial distance based on the light projection signal acquired from the drive circuit 11c and the light reception signal acquired from the signal processing circuit 11d to generate distance image information, and the generated distance image information is information. Output to the control unit 12.
Next, the recording control device 30 starts an automatic tracking recording process (step ST3). Details of the automatic tracking recording process will be described later.
Next, when the automatic recording process is completed, the recording control device 30 shuts down the system (step ST4).

Next, details of the automatic tracking recording process will be described with reference to the flowchart shown in FIG. In FIG. 7, it is assumed that the subject of automatic tracking recording is a teacher who performs a class in classroom A.
The control unit 31 of the recording control device 30 determines whether or not a recording request has been acquired (step ST21). After acquiring the recording request via the input unit 33, the control unit 31 determines that the recording request has been acquired until the recording stop request is acquired.
When it is determined that the recording request has been acquired (step ST21: YES), the control unit 31 proceeds to the process of step ST22.
On the other hand, when it is determined that the recording request has not been acquired, that is, the recording stop request has been acquired (step ST21: NO), the control unit 31 ends the automatic tracking recording process and returns to FIG.

The information control unit 12 analyzes the distance image information acquired from the signal processing circuit 11d to detect the tracking target (person), assigns an ID to the detected person in order, and tracks the person to which the ID is assigned. (Step ST22). Next, the information control unit 12 determines whether or not an ID that first enters an area C described later exists in the area C (step ST23).
In step ST23, when the information control unit 12 determines that the ID first entered in the area C exists in the area C (step ST23: YES), the information control unit 12 identifies the ID as a tracking target ID (teacher ID), The position information of the person to whom the tracking target ID is assigned is generated, and the generated position information is transmitted to the recording control device 30 (step ST24). The position information is expressed by coordinate information, for example. The information of area C may be stored in the memory of the first camera 10 or may be acquired from the recording control device 30.
On the other hand, when the information control unit 12 determines in step ST23 that the ID first entered in area C does not exist in area C (step ST23: NO), the process returns to step ST21. At this time, the information control unit 12 sets the ID that has entered the area C next as the tracking target ID (teacher ID).

Here, an example of the area C will be described with reference to FIGS. 5 (a) and 5 (b).
Area C is provided on the front side of classroom A. The width in the front-rear direction and the width in the left-right direction of the area C are set to be slightly wider than the width occupied by the platform 2 when the classroom A is viewed from above. Further, the vertical width of the area C is set so as to include the upper half of the person (teacher) standing on the platform 2 when the classroom A is viewed from the side. The information of area C can be arbitrarily set by the user.

Next, the control unit 31 of the recording control device 30 corrects the position information acquired from the first camera 10 and sets the image to be set in the second camera 20 based on the corrected position information (corrected position information). Control information is generated, and the corrected position information and imaging control information are transmitted to the second camera 20 (step ST25).
The position information acquisition unit 31a of the control unit 31 acquires the position information, and the position information correction unit 31b acquires the position information of the tracking target acquired from the first camera 10 from the installation position of the second camera 20. The position information is corrected so as to be the position information of the tracking target in the case. The storage unit 32 stores position coordinate information of the installation position of the first camera 10 and position coordinate information of the installation position of the second camera 20. The storage may be manual or automatic. The position information correction unit 31b performs arbitrary coordinate conversion using the position coordinate information stored in the storage unit 32 to correct the position information. FIG. 8 is a diagram illustrating an example of coordinate conversion, and illustrates an example in which 3DXY coordinates are converted into PTZ polar coordinates (r, θ).
The imaging control information generated by the imaging control information generation unit 31c of the control unit 31 includes visual field control information. The visual field control information includes information for determining pan, tilt, and zoom of the imaging unit 21 in the second camera 20. The imaging control information generation unit 31c generates imaging control information so that the tracking target falls within the angle of view.

Next, the imaging control unit 22 of the second camera 20 starts imaging while controlling the drive mechanism of the imaging unit 21 based on the corrected position information and imaging control information acquired from the recording control device 30, and generates them. The captured image is transmitted to the recording control device 30 (step ST26).
Next, the control unit 31 (captured image acquisition unit 31d) of the recording control device 30 stores the captured image acquired from the second camera 20 in the storage unit 32 (step ST27). In the above description, the captured image acquired from the second camera 20 is stored in the storage unit 32. However, the captured image may be live-distributed to an arbitrary terminal. Recording is a concept that includes not only storing captured images but also live distribution of captured images.

In the above description, the example of imaging the teacher in the area C has been described. However, the present invention is not limited to this. For example, when the teacher leaves the area C and moves to the rear of the classroom A, the information of the tracking target ID (teacher ID) specified by the first camera 10A is the first information. The first camera 10B may generate position information of the person to whom the teacher ID is assigned, and the second camera 20B may track the teacher and take an image. Further, when the first camera 10A and the second camera 20A are installed on the ceiling behind the classroom A and the teacher leaves the area C, the first camera 10A has the position information of the person to whom the teacher ID is given. And the second camera 20A may track and image the teacher.

Next, another example of the automatic tracking recording process will be described using the flowchart shown in FIG. In FIG. 9, it is assumed that the subject of automatic tracking recording is a teacher who conducts a lesson in classroom A.
The control unit 31 of the recording control device 30 determines whether or not a recording request has been acquired (step ST41). When it is determined that the recording request is acquired (step ST41: YES), the control unit 31 of the recording control device 30 proceeds to the process of step ST42.
On the other hand, when it is determined that the recording request has not been acquired, that is, the recording stop request has been acquired (step ST41: NO), the control unit 31 ends the automatic tracking recording process and returns to FIG.

The information control unit 12 analyzes the distance image information acquired from the signal processing circuit 11d to detect the tracking target (person), assigns an ID to the detected person in order, and tracks the person to which the ID is assigned. (Step ST42).
Next, the information control unit 12 performs face recognition of the person to whom the ID is assigned, and determines whether there is an ID having face information that matches the face information of the tracking target (teacher) stored in advance. (Step ST43).
The information control unit 12 analyzes the distance image information, extracts the face information of the person given the ID, and performs face recognition by comparing the face information with the face information of the teacher stored in advance. . The face information of the teacher may be stored in the memory of the first camera 10 or may be acquired from the recording control device 30.
In step ST43, when the information control unit 12 determines that there is an ID having face information that matches face information stored in advance (step ST43: YES), the information control unit 12 identifies the ID as a tracking target ID (teacher ID). Then, the position information of the person to whom the tracking target ID is assigned is generated, and the generated position information is transmitted to the recording control device 30 (step ST44).
On the other hand, if it is determined in step ST43 that there is no ID having face information that matches face information stored in advance (step ST43: NO),
The process returns to step ST41.

Step ST45 to step ST47 are the same as step ST25 to step ST27 shown in FIG. 7 and 9, the case where the subject of automatic tracking recording is a teacher has been described. However, the present invention is not limited to this, and a specific student may be subject to automatic tracking recording.

As described above, according to the automatic tracking recording system 1 according to the first embodiment, the first camera 10 capable of detecting an object three-dimensionally generates position information of a tracking target with high accuracy and performs recording control. The device 30 generates imaging control information for controlling the drive mechanism of the second camera 20 based on the position information, and outputs the imaging control information to the second camera 20. For this reason, the second camera 20 can control the drive mechanism based on the imaging control information and track and image the target with high accuracy. Thereby, the captured image which caught the tracking object with high precision can be obtained.

(Second Embodiment)
In the second embodiment, a case will be described in which the information control unit 12 of the first camera 10 further generates tracking target behavior information. Hereinafter, parts that are the same as or correspond to the components of the automatic tracking recording system 1 according to the first embodiment are denoted by the same reference numerals as those used in the first embodiment, and redundant descriptions are omitted or simplified. .

First, an example of tracking target behavior information generated by the information control unit 12 will be described with reference to FIG. FIG. 10 shows a case where the tracking target is a student. The action information includes seating information, raising hand information, standing information, and the like. The seating information is information indicating a state in which the student is seated. Raised hand information is information indicating a state in which a seated student is raising a hand. The standing information is information indicating a state where the student is standing.

Next, an example of automatic tracking recording processing according to the second embodiment will be described using the flowchart shown in FIG. Hereinafter, a case where the subject of automatic tracking recording is a student will be described.
The control unit 31 of the recording control device 30 determines whether or not a recording request has been acquired (step ST61). When it is determined that the recording request has been acquired (step ST61: YES), the control unit 31 proceeds to the process of step ST62.
On the other hand, when it is determined that the recording request has not been acquired, that is, the recording stop request has been acquired (step ST61: NO), the control unit 31 ends the automatic tracking recording process and returns to FIG.

The information control unit 12 analyzes the distance image information acquired from the signal processing circuit 11d to identify the tracking target (student), generates position coordinates and behavior information of the tracking target, and uses the generated position coordinates and behavior information. It transmits to the video recording control apparatus 30 (step ST62).

Next, the control unit 31 of the recording control device 30 corrects the position information acquired from the first camera 10 and sets the imaging control information to be set in the second camera 20 based on the corrected position information and the action information. And the corrected position information and imaging control information are transmitted to the second camera 20 (step ST63).

Here, generation of imaging control information performed by the control unit 31 in step ST63 will be described in detail with reference to FIG. The control unit 31 of the second embodiment further includes a behavior state determination unit (not shown), and the behavior state determination unit determines a behavior state based on the behavior information acquired from the first camera 10 and determines the behavior state. It outputs to the imaging control information generation part 31c.
FIG. 12 is a diagram illustrating a specific example of the behavior state and the imaging control information. For example, the behavioral state includes a behavioral state a to a behavioral state d.
(1) Action state a
The control unit 31 determines that the behavior state is a when the raising information and the standing information are not included in the behavior information, that is, when all the information is seating information. The action state a is a state in which all the students are seated and there are no students raising their hands or standing students.
(2) Action state b
The control unit 31 determines that the behavior state is b when the behavior information includes at least one raising hand information and does not include the standing information. The behavioral state b is a state in which at least one student raising his hand and no student standing up.
(3) Behavior information c
The control unit 31 determines that the behavior state is c when the behavior information includes at least one standing information and does not include the raising hand information. The behavioral state c is a state in which at least one student is standing up and no student is raising his hand.
(4) Action state d
The control unit 31 determines that the behavior state is d when the behavior information includes at least one raising hand information and at least one standing information. The action state d is a state in which at least one student raising his hand and at least one student standing up.

FIG. 13 is a flowchart for explaining generation of imaging control information by the control unit 31.
First, the control unit 31 determines whether or not the hand raising information is acquired (step ST101). In step ST101, when it is determined that the control unit 31 has acquired the hand raising information (step ST101: YES), the control unit 31 determines whether or not the standing information has been acquired (step ST102).
In step ST102, when it is determined that the standing information is acquired (step ST102: YES), the control unit 31 determines that the action state is d (step ST103). Next, the control unit 31 determines the pan / tilt direction and zoom magnification so that both the raised hand and the standing student enter the angle of view based on the acquired position information. The imaging control information d (shown in FIG. 12) is generated (step ST104).
On the other hand, if it is determined in step ST102 that the standing information has not been acquired (step ST102: NO), the control unit 31 determines that the action state is b (step ST105). Next, based on the acquired position information, the control unit 31 determines the panning and tilting directions and the zooming control information b so that the student raising his hand enters the angle of view (FIG. 12). (Step ST106). In this case, when there is only one student raising a hand, the imaging is controlled so that the student falls within the angle of view, and when there are a plurality of students raising a hand, the plurality of students fall within the angle of view. Thus, the image pickup is controlled. However, even when there are a plurality of students raising their hands, the imaging control information may be generated so that only one specific student falls within the angle of view.

In step ST101, when it is determined that the control unit 31 has not acquired the raising hand information (step ST101: NO), the control unit 31 determines whether the standing information is acquired (step ST107).
In step ST107, when it is determined that the standing information is acquired (step ST107: YES), the control unit 31 determines that the action state is c (step ST108). Next, based on the acquired position information, the control unit 31 determines the panning, tilting direction, and zooming magnification so that the standing student enters the angle of view, and the imaging control information c (see FIG. 12). (Shown) is generated (step ST109). In this case, when there is one standing student, the imaging is controlled so that the student falls within the angle of view, and when there are a plurality of standing students, imaging is performed so that the plurality of students fall within the angle of view. Will be controlled. However, even when there are a plurality of standing students, the imaging control information may be generated so that only one specific student falls within the angle of view.
On the other hand, when determining in step ST107 that the standing information has not been acquired (step ST107: NO), the control unit 31 determines that the action state is a (step ST110). Next, the control unit 31 generates the imaging control information a (shown in FIG. 12) in which the pan and tilt directions and the zoom magnification are determined so that all students enter the angle of view based on the acquired position information. (Step ST111).

Returning to FIG.
The imaging control unit 22 of the second camera 20 starts imaging while controlling the drive mechanism of the imaging unit 21 based on the corrected position information and imaging control information acquired from the recording control device 30, and displays the generated captured image. It transmits to the video recording control apparatus 30 (step ST64).
Next, the control unit 31 (captured image acquisition unit 31d) of the recording control device 30 stores the captured image acquired from the second camera 20 in the storage unit 32 (step ST65).

The case where the control unit 31 generates student behavior information and the behavior information is output to the recording control device 30 has been described above.
The control unit 31 may recognize a student's face and generate attended student information. In this case, the attended student information is output to the recording control device 30. The control unit 31 of the recording control device 30 compares the attended student information with the name information stored in advance, and generates and manages student attendance information. Thereby, attendance management can be performed automatically.

In addition, when the control unit 31 performs face recognition of a person in the classroom A and face information other than the person stored in advance is detected, outsider information may be generated. In this case, the outsider information is output to the recording control device 30. When acquiring the outsider information, the control unit 31 of the recording control device 30 performs control such as emitting a warning sound via a sound output unit (not shown). Thereby, the illegal attendance of a class, etc. can be prevented.

The information control unit 12 of the first camera 10 may generate face information that detects the direction of the line of sight, the direction of the face, the facial expression, and the like from the face of the student. In this case, the face information is output to the recording control device 30. The control unit 31 of the recording control device 30 generates and manages evaluation information for evaluating the student's lesson attitude based on the face information. Thereby, a student's lesson attitude can be automatically evaluated. In addition, the control unit 31 statistically aggregates student responses based on the face information, and generates and manages evaluation information for evaluating the teacher's lesson content. Thereby, a teacher's lesson content can be evaluated automatically.

The information control unit 12 of the first camera 10 may generate hand raising information indicating that the student has raised his hand and speech information indicating that the student has made his / her speech. In this case, raised hand information and utterance information are output to the recording control device 30. In this case, the control unit 31 of the recording control device 30 statistically counts the number of raising hands and the number of utterances of the student, and generates and manages evaluation information for evaluating the student's class attitude. Thereby, a student's lesson attitude can be automatically evaluated.

As described above, according to the automatic tracking recording system 1 according to the second embodiment, the imaging control information according to the behavior to be tracked is generated. For this reason, the second camera 20 can control the drive mechanism in accordance with the behavior of the tracking target and image the tracking target. Thereby, it is possible to obtain a captured image that is adjusted and imaged according to the behavior of the tracking target.

In a facility where a PTZ camera having a tracking function is already installed, the PTZ camera is used as the second camera 20, and the first camera 10 and the recording control device 30 are further installed in the first embodiment. The automatic tracking recording system 1 shown in the second embodiment can also be constructed. In this case, the second camera 20 may be subjected to a measure such as a specification for tracking imaging according to information from the recording control device 30, for example.

DESCRIPTION OF SYMBOLS 1 Automatic tracking recording system 10 1st camera 20 2nd camera 30 Recording control apparatus 31a Position information acquisition part 31b Position information correction | amendment part 31c Imaging control information generation part 31d Captured image acquisition part

Claims (4)

A first camera that detects an object in three dimensions, identifies and tracks a tracking target, and generates and outputs position information of the tracking target;
A recording control device that generates and outputs imaging control information based on the position information of the tracking target;
An automatic tracking recording system comprising: a second camera that generates a captured image by controlling a driving mechanism of an imaging unit based on the imaging control information, and outputs the captured image to the recording control device. The first camera generates and outputs the tracking target behavior information,
2. The automatic tracking recording system according to claim 1, wherein the recording control device generates the imaging control information based on the position information and the behavior information of the tracking target. The first camera is a camera capable of measuring a three-dimensional distance to an object in an imaging space;
The automatic tracking recording system according to claim 1 or 2, wherein the second camera is a camera including at least the driving mechanism for adjusting pan, tilt, and zoom. A recording control device communicably connected to the first camera and the second camera,
A position information acquisition unit for detecting the object in three dimensions, specifying and tracking the tracking target, generating the positional information of the tracking target and outputting the positional information from the first camera;
An imaging control information generating unit that generates imaging control information for controlling a driving mechanism of the imaging unit in the second camera based on the position information, and outputs the imaging control information to the second camera;
A recording control apparatus comprising: a captured image acquisition unit configured to acquire a captured image generated by the imaging unit of the second camera based on the imaging control information.

Images