JP2007266652A - Moving object detection apparatus, moving object detection method, moving object detection program, video decoding apparatus, video encoding apparatus, imaging apparatus, and video management system - Google Patents

Abstract

A moving object detection method and apparatus capable of detecting a moving object at high speed, high accuracy, and low processing load.
A band division method for dividing an image into a reduced image, a horizontal component, a vertical component, and a diagonal component, and motion information from a video stream that has been encoded using motion prediction compensation encoding. A motion information extracting unit 102 for extracting, an edge information extracting unit 103 for extracting information on a horizontal component, a vertical component and a diagonal component from one or more bit planes in order from the most significant bit plane of the video stream; By having a moving object detection means 106 that detects a moving object using motion information and edge information and outputs the detection result, it is not necessary to decode a video stream, so that high-speed, high-precision, and low-processing A moving object can be detected by a load.
[Selection] Figure 1

Description

  The present invention relates to a moving object detection method and apparatus for detecting a moving object from a video stream generated by encoding video.

  Conventionally, as this moving object detection device, there has been one as described in Patent Document 1, for example.

  This moving object detection apparatus extracts a motion vector used in the motion prediction / compensation coding method without decoding a video stream, regards the motion vector as a motion of an object in a certain region, and detects a moving object at high speed. It is to detect. FIG. 18 shows a conventional moving object detection device described in Patent Document 1. In FIG.

In FIG. 18, the coding mode, motion compensation mode, and motion vector information of the image block decoded by the variable length decoding unit 1801 and the pattern information detected by the pattern information detection unit 1802 are a moving object detection processing unit. Sent to 1803. The moving object detection processing unit 1803 uses these pieces of information to determine whether or not this image block is a moving object. This determination is performed using a motion vector, spatial similarity determination, temporal similarity determination, or the like.
Japanese Patent Laid-Open No. 10-75457

  However, in the above conventional configuration, it depends on only the motion vector that does not necessarily accurately represent the motion of the object, so it cannot be said that the accuracy is good. That is, as a method for generating a motion vector, a reference region where the compression rate of encoding is high is searched from previous and next images with respect to the region being encoded, and a reference to the searched region is used as a motion vector. Many. For this reason, the detection of the moving object using only the motion vector is not accurate.

  The present invention has been made to solve the above-described problems, and the object of the present invention is to provide a band dividing method for dividing an image into a reduced image, a horizontal component, a vertical component, and a diagonal component, and It is an object of the present invention to provide a moving object detection method and apparatus capable of detecting a moving object at high speed, with high accuracy, and with a low processing load from a video stream that has been encoded using motion prediction compensation encoding.

  A moving object detection apparatus according to the present invention includes a motion information extraction unit that extracts motion information from a video stream that has been video-encoded using hierarchical encoding and motion prediction compensation encoding in which video is divided into a plurality of layers. , An edge information extracting means for extracting edge information from the video stream, and a moving object detecting means for detecting a moving object using the motion information and the edge information and outputting the detection result.

  According to this configuration, the object contour can be detected without decoding the video stream, and the moving object can be detected from the motion information, and the moving object can be detected with high speed, high accuracy, and low processing load. be able to.

  Further, in the moving object detection device of the present invention, the edge information extraction means further includes a bit plane from the most significant bit plane to the N (N is a natural number) bit-order bit plane in the bit plane information obtained by bit-plane coding of the image. Information is extracted from the video stream as edge information.

  According to this configuration, by extracting information up to a specific bit plane, an edge having a specific intensity or more can be detected, and the contour of the object can be detected at high speed. In addition, it is possible to detect the contour of an object only with a bit plane above a specific bit position, no bit plane less than a specific bit position is necessary, and even when receiving a video stream via a communication network with a low communication speed, Highly accurate detection at a low bit rate.

  In the moving object detection device of the present invention, the video stream is further divided into a plurality of areas, and the moving object detection means has a predetermined total code length of bit plane information in the area. When the value is equal to or greater than 1, the region is determined as the contour region of the moving object.

  According to this configuration, it is possible to determine the number of edges existing in a region simply by confirming the code amount of the bit plane up to a certain threshold bit position in a region of the image, and at high speed, The contour can be detected.

  Further, in the moving object detection device of the present invention, the moving object detection means further selects the area when the total code length of the bit plane information inside the area is equal to or less than a predetermined second value. It is determined as the contour area of the moving object.

  According to this configuration, since the contour of the object is a line, if a certain region contains too many horizontal, vertical and diagonal components, for example, it is a region containing a striped pattern and a moving object Therefore, it is possible to prevent erroneous detection by determining that it is not a contour.

  In the moving object detection device of the present invention, the motion information extraction unit further extracts a motion vector from an area determined to be the contour area of the moving object, and the moving object detection unit has a magnitude of the motion vector. When the value is equal to or greater than a predetermined third value, the region is determined to be a contour region of a moving object.

  According to this configuration, it is possible to determine that an object that is not moving is not a moving object, and to improve the accuracy of detection of the moving object.

  In the moving object detection device of the present invention, the motion information extraction unit further extracts the first motion vector from the area determined to be the outline area of the moving object and selects an area located in the vicinity of the area. A second motion vector is extracted from the selected region, and the moving object detection means calculates a magnitude of a difference vector between the first motion vector and the second motion vector as a measurement value, and When the measured value is equal to or less than a predetermined fourth value, the selected area is determined to be an internal area of the moving object.

  According to this configuration, since the contour area of the moving object in the video has a different speed from the surrounding area, it is determined that the area other than the contour of the moving object is not the moving object area, and the moving object is detected. Accuracy can be improved.

  In the moving object detection device of the present invention, the motion information extraction unit further selects a plurality of regions, extracts a motion vector from each selection region, and the moving object detection unit performs the first operation for each of the selection regions. The magnitude of the difference vector between one motion vector and the motion vector of the selected area is obtained, and the sum of the magnitudes of the difference vectors for all selected areas is calculated as the measurement value.

  According to this configuration, the contour area of the moving object in the video has a different speed from the surrounding area. Therefore, it is determined that a plurality of areas other than the contour of the moving object are not moving object areas, and the moving object The accuracy of detection can be improved.

  Further, in the moving object detection device of the present invention, the moving object detection means further includes a motion vector of an area determined to be an internal area of the moving object and a motion vector of an area located in the vicinity of the area. When the magnitude of the difference vector is equal to or smaller than a predetermined fifth value, it is determined that the difference vector is an inner area of the moving object area.

  According to this configuration, it is possible to detect a region of a moving object that moves at a certain speed that is not determined as a moving object, and it is possible to improve the accuracy of detection of the moving object.

  Further, in the moving object detection device of the present invention, the moving object detection means further includes an area surrounded by the contour area of the moving object or the area determined as the inner area of the moving object as the inner area of the moving object. It is determined.

  According to this configuration, the inside determined as the contour of the moving object can be detected as the moving object region, and the accuracy of detection of the moving object can be improved.

  Further, in the moving object detection device of the present invention, the moving object detection means further includes the contour area or the internal area of the second moving object in the vicinity of the contour area or the internal area determined as the first moving object. When the number of determined areas is equal to or greater than a predetermined sixth value, the contour area or the internal area determined as the first moving object is determined again as the first moving object.

  According to this configuration, it is possible to determine that a too small area is not a moving object, and it is possible to reduce erroneous detection of moving object detection.

  The moving object detection method of the present invention is a method for detecting a moving object from a video stream, and is a hierarchical code for encoding a video divided into a plurality of layers executed by the moving object detection device for detecting the moving object. And extracting motion information from a video stream that has been encoded using motion prediction compensation encoding, extracting edge information from the video stream, and extracting the motion information and edge information. And a step of detecting a moving object using.

  According to this method, it is possible to detect the contour of an object without decoding the video stream. Furthermore, it is possible to detect a moving object from motion information, and to detect a moving object at high speed, high accuracy, and with a low processing load. can do.

  The moving object detection program of the present invention is a video encoded using hierarchical encoding and motion prediction compensation encoding, in which a computer is detected in order to detect a moving object from a video stream, and the video is divided into a plurality of layers. A step of extracting motion information from the stream, a step of extracting edge information from the video stream, and a step of detecting a moving object using the extracted motion information and edge information are executed.

  According to this program, it is possible to detect the contour of an object without decoding the video stream. Furthermore, it is possible to detect a moving object from motion information, and to detect a moving object with high speed, high accuracy, and low processing load. can do.

  The video decoding apparatus according to the present invention includes a video decoding unit that decodes a video stream encoded by hierarchical encoding and motion prediction compensation encoding that divides a video into a plurality of layers, and the video decoding unit includes: Moving object detection means for detecting a moving object from motion information and edge information extracted when decoding the video stream.

  According to this configuration, the video decoding device and the moving object detection device can share part of the processing and means, and can simultaneously perform video decoding and moving object detection at high speed, and It is possible to reduce the scale of the entire apparatus.

  In the video decoding device of the present invention, the video stream is divided into a plurality of areas, and the moving object detection means has a predetermined first code length of bit plane information in the area. When the value is equal to or greater than the value, the region is determined as the contour region of the moving object.

  According to this configuration, for example, it is possible to reduce the number of edges existing in a region only by checking the code amount of the bit plane up to a certain threshold bit position in a region having a horizontal component, a vertical component, and a diagonal component. The contour of the object can be detected at high speed.

  In the video decoding device of the present invention, the moving object detection means further moves the area when the total code length of the bit plane information in the area is equal to or less than a predetermined second value. It is determined as the contour area of the object.

  According to this configuration, since the contour of the object is a line, if a certain region contains too many horizontal, vertical and diagonal components, for example, it is a region containing a striped pattern and a moving object Therefore, it is possible to prevent erroneous detection by determining that it is not a contour.

  In the video decoding apparatus of the present invention, the video decoding means further generates a video in which the area of the moving object detected by the moving object detection means is emphasized.

  According to this method, the monitor can easily detect a moving object.

  In the video decoding apparatus according to the present invention, the video decoding means further generates a video composed of edge components, and emphasizes and displays only the area of the moving object detected by the moving object detection means.

  As a result, even when the base layer bit rate is very low due to communication speed limitations or the like and only images with extremely poor image quality can be generated, only the contour may be able to recognize details.

  Also, only moving objects are very conspicuous in the contour image, and it is easy for a monitor who watches a plurality of monitor images at the same time to detect the occurrence of an abnormality or a suspicious person. Alternatively, even in an environment where the processing capability is limited, such as when displaying a plurality of camera images, it is possible to easily display an area important for monitoring with a low processing load.

  The video encoding apparatus according to the present invention includes a video encoding unit that generates a video stream encoded using hierarchical encoding and motion prediction compensation encoding that divides a video into a plurality of layers, and the video encoding unit includes: It has moving object detection means for detecting the moving object by extracting the motion information and the edge information of the video when encoding the video. According to this configuration, the video encoding means and the moving object detection means can share a part of the processing and means, and the video encoding and the detection of the moving object can be simultaneously performed at high speed. In addition, the scale of the entire apparatus can be reduced.

  The imaging apparatus of the present invention is based on an imaging means for inputting video, a video encoding apparatus according to the present invention for encoding video input by the imaging means, and a detection result of a moving object output by the moving object detection means. The image pickup control means for controlling the image pickup function with respect to the image pickup means, and the output section for outputting the video stream and the detection result of the moving object.

  With this configuration, it is possible to detect moving objects in the process of generating a video stream that is generated for video transmission to a remote place. The video can be transmitted and the video can be monitored efficiently.

  In the imaging apparatus of the present invention, the imaging control unit controls the imaging unit so that the area of the moving object region output from the moving object detection unit is a constant ratio with respect to the total area of the input video. Is.

  With this configuration, it is possible to capture the moving object and the surrounding situation on the video, and it is possible to efficiently monitor the moving object of interest.

  The video monitoring system of the present invention decodes the imaging apparatus according to the present invention and a video stream received from the imaging apparatus, and performs image recognition of the detected moving object region using the detection result of the moving object. A video surveillance device.

  With this configuration, it is possible to detect moving objects in the process of generating a video stream that is generated for video transmission to a remote location, and to omit high-speed and low processing by omitting image recognition processing for areas other than moving objects. Since the image can be recognized with a load, it is possible to continuously detect and shoot a suspicious person as a moving object in video surveillance or the like.

  In the present invention, image recognition is not limited to detection of a moving object, but refers to automatic discrimination means using a machine image including recognition of a person / face / object and authentication of a person.

  Further, in the video decoding device of the present invention, the video stream is further encoded by being layered into a base layer and an enhancement layer, and the motion information extraction unit extracts the motion information from the video stream of the base layer, The edge information extracting means extracts the edge information from the enhancement layer video stream.

  According to this configuration, when the motion information indicates that there is no motion, it is possible to stop the processing such as extraction of the edge information and reduce the processing load. In the case where it is indicated that there is no error, the processing load such as the extraction of the motion information can be stopped to reduce the processing load, and the contour of the object can be detected at high speed.

  In the video decoding device of the present invention, the video stream is further encoded by being layered into a base layer and an enhancement layer, and the motion information extraction unit extracts the motion information from the video stream of the enhancement layer, The edge information extraction means extracts the edge information from the enhancement layer video stream.

  According to this configuration, the moving object detection process can be performed only with the enhancement layer video stream, and the contour of the object can be detected with high speed and with a small number of video streams.

  As described above, according to the present invention, video coding is performed using a band division method for dividing an image into a reduced image, a horizontal component, a vertical component, and a diagonal component, and motion prediction compensation coding. It is possible to detect the contour of an object moving at high speed, high accuracy, and low processing load without decoding the video from the video stream. At the same time, video can be decoded.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
In the first embodiment, the moving object detection method and apparatus according to the present invention is applied to a video decoding apparatus. In other words, at the same time as decoding the video stream, a moving object in the video can be detected at high speed and with high accuracy.

  First, the video stream used in this embodiment will be described. This video stream is composed of a base layer and an enhancement layer, and the base layer can be decoded independently to obtain a low-resolution video. The enhancement layer is additional information that can improve the image quality of the base layer and obtain a high-resolution image. Horizontal, vertical, and diagonal edge components (horizontal component, vertical component, and diagonal component) )including.

  Next, a method for generating this video stream will be described.

  First, the input image is band-divided to generate a reduced image, a horizontal component, a vertical component, and a diagonal component. In addition, the reduced image is encoded as a base layer that can be decoded independently by motion prediction compensation encoding. Then, the horizontal direction component, the vertical direction component, and the diagonal direction component are encoded as an extension layer for improving the image quality of the video obtained by decoding the base layer by bit plane encoding.

  Here, band division will be described. In the band division, the image is divided into four components: a reduced image, a horizontal component, a vertical component, and a diagonal component. This band division is performed by wavelet transform or using a combination of a high pass filter, a low pass filter and a down sampler. The reduced image / horizontal direction component, vertical direction component, and diagonal direction component obtained by band division can be restored to the original image by band synthesis. The horizontal direction component, the vertical direction component, and the diagonal direction component obtained by this band division are pixel value differences from neighboring pixels that can be mathematically calculated, and do not necessarily represent the contour of the object. For example, in a black and white horizontal stripe pattern, a vertical component strong at the boundary of the color appears as a horizontal line.

  FIG. 1 is a block diagram showing a configuration of a video decoding apparatus 100 according to Embodiment 1 to which the moving object detection method and apparatus of the present invention is applied.

  In FIG. 1, a video decoding apparatus 100 includes a stream input unit 101, a base layer decoding 102, an enhancement layer decoding unit 103, a band synthesis unit 104, a video output unit 105, a moving object detection unit 106, and a detection result output unit 107. Have

  Note that base layer decoding section 102, enhancement layer decoding section 103, and band synthesizing section 104 correspond to the video decoding means of the present invention, base layer decoding section 102 corresponds to the motion information extraction means, and enhancement layer The decoding unit 103 corresponds to edge information extraction means, and the moving object detection unit 106 corresponds to moving object detection means.

  Here, the video decoding means generates a video by decoding the input video stream and outputs the video. The motion information extraction unit extracts motion information from the input video stream and outputs the motion information to the moving object detection unit. The edge information extraction means extracts edge information from the input video stream and outputs it to the moving object detection means. The moving object detection means detects a moving object from the input edge information and motion information.

  Next, the operation of the video decoding apparatus 100 configured as described above will be described.

  FIG. 3 is a flowchart showing the operation of the video decoding apparatus 100 according to the first embodiment shown in FIG. The flowchart shown in FIG. 3 is executed by software by executing a control program stored in a storage device (not shown) such as a ROM or a flash memory by a CPU (not shown). It is also possible to do so.

  First, the stream input unit 101 inputs a video stream from the outside of the video decoding device 100, and outputs the base layer of the video stream to the base layer decoding unit 102 and the enhancement layer to the enhancement layer decoding unit 103 (step). S301).

  Next, the base layer decoding unit 102 extracts motion information from the base layer input from the stream input unit 101 and outputs the motion information to the moving object detection unit 106. Further, enhancement layer decoding section 103 extracts edge information from the enhancement layer input from stream input section 101 and outputs the extracted edge information to moving object detection section 106. Then, the moving object detection unit 106 detects a moving object using the motion information and edge information input from the base layer decoding unit 102 and the enhancement layer decoding unit 103, generates a moving object detection result, and outputs a detection result. Output to the unit 107 and the band synthesis unit 104 (step S302).

  Note that a video may or may not include a moving object, and may include one moving object or a plurality of moving objects.

  Hereinafter, the moving object detection process in step S302 will be described in detail.

  FIG. 4 is a flowchart illustrating an example of the procedure of the moving object detection process of FIG.

  First, in step S401, edge information extraction processing is performed. Specifically, enhancement layer decoding section 103 extracts a code including information from the enhancement layer input from stream input section 101 to a specific bit plane, generates edge information, and outputs it to moving object detection section 106 To do.

  Here, bit plane encoding will be described.

  This bit plane is a bit string in which only the same bit positions of several numerical data expressed in binary numbers are arranged. The method of coding for each bit plane is called bit plane coding. Weiping. Li, "Overview of Fine Granularity Scalability in MPEG-4 Video Standard", IEEE Transaction on Circuits and Systems for Video Technology, vol.11, pp. Excellent ability to adjust data quality as described in 301-317, Mar.2001.

  FIG. 2 is a diagram showing the concept of bit-plane coding, and the description will be given assuming that it represents a region having a horizontal component.

  In FIG. 2, one column represents one pixel of the horizontal component expressed in binary (pixel 1, pixel 2). One row represents a bit plane in a region having a horizontal component (bit plane 1 and bit plane 2), that is, a collection of only the same bits of each pixel. The higher the bit plane of the bit plane, the stronger the edge of the horizontal component can be expressed. Among the bit planes, edge information is obtained by encoding information obtained by arranging information from the most significant bit plane to a specific bit plane. For example, it includes information such as the code amount for each bit plane up to a specific bit plane for each 8 × 8 pixel or 16 × 16 pixel region. Since the horizontal direction component, the vertical direction component, and the diagonal direction component include many “0” s, bit plane encoding is performed so that the code length is shortened when there are many “0s”. Therefore, as the number of “1” s increases, the bit plane of the horizontal component, vertical component, and diagonal component regions has a longer code length.

  FIG. 5 shows the data structure of the enhancement layer according to this embodiment. The enhancement layer shown in FIG. 5 is a code for one image, and includes information on n bit planes and m areas. The enhancement layer for one image holds image header information 501 and information 502 from the bit plane 1 representing the most significant bit plane to the least significant bit plane n.

  FIG. 6 shows the data structure of the bit plane k of the enhancement layer in FIG. 5, and the bit plane k of the enhancement layer includes the header information 601 of the bit plane and the reference numeral 602 of the bit plane k of regions 1 to m.

  FIG. 7 shows the data structure of the bit plane k of the enhancement layer region j in FIG. 6. The bit plane k of the enhancement layer region j shows that the pixel component code 701 of the corresponding region and the region code have been terminated. A termination signal 702 is included.

  With the data structure as described above, bit plane information is extracted from the video stream in order from the most significant bit plane to a specific bit plane by searching for the end signals of those areas in the video stream, and the code length between the end signals of the areas is determined. Just count. Therefore, enhancement layer decoding section 103 can generate edge information at high speed.

  Next, in step S402, motion information extraction processing is performed. Specifically, the base layer decoding unit 102 extracts motion vector information from the base layer input from the stream input unit 101, generates motion information, and outputs the motion information to the moving object detection unit 106.

  This motion information is used for motion prediction compensation of the base layer, and is referred to as motion prediction compensation coding for each region or intra-frame coding, the magnitude and direction of the motion vector, and the motion vector. And information on whether the entire image is motion prediction compensation coding or intraframe coding.

  FIG. 8 shows the data structure of the base layer of the present embodiment. The base layer shown in FIG. 8 is a code for one image and includes information of m areas. That is, the base layer for one image includes image header information 801 and information 802 on regions 1 to m. FIG. 9 shows the data structure of the base layer region p in FIG. 8. The base layer region p shows the region header information 901, the motion vector 902, the pixel component code 903, and the end of the region code. A termination signal 904 is included.

  To extract a motion vector, it is only necessary to search the header information 901 and termination signal 904 of those areas from the video stream and decode only the motion vector 902 located at a fixed position from that position. As a result, the base layer decoding unit 102 can generate motion information at high speed.

  In step S403, a moving object outline detection process is performed. Specifically, the moving object detection unit 106 detects the contour area of the moving object using the motion information and edge information input from the base layer decoding unit 102 and the enhancement layer decoding unit 103, and moves the result. Store in the object detection unit 106.

  Here, a method for detecting a contour region will be described below.

  That is, the code length obtained from the bit plane of the horizontal direction component, vertical direction component and diagonal direction component for a certain region, for example, the total code length of each code amount from the most significant bit plane to the 3-bit plane is equal to or greater than the threshold A This is Condition 1. The threshold A is a reference value for determining a weak edge.

  Further, the condition 2 is that the total code length of the above region is equal to or less than the threshold value B. This threshold value B is a reference value for identifying an image that is not an edge such as a striped pattern.

  Then, it is determined whether or not the edge information included in the region represents a point, a line, or a surface, and when the total code length of the region satisfies these conditions 1 and 2, the contour of the object It is determined that the line appears. Hereinafter, a specific example will be described with reference to FIG.

  FIGS. 10A to 10C each show an example of the horizontal component in the 8 × 8 pixel region. In order to simplify the explanation, the pixel value is represented by a binary value. If “1” is included from the most significant bit plane to a specific bit plane, the square is black, and those not including “1” are white. . FIG. 10A shows the horizontal component when noise or a small point exists in the region, and FIG. 10B shows the horizontal component when a vertical line exists in the region. c) shows the horizontal component when the entire region is part of a striped pattern, for example. When each of the regions represented in FIGS. 10A to 10C is encoded, the code amount is ascending in accordance with the number of values other than 0 included in the regions in the order of FIG. 10A, FIG. 10B, It becomes FIG.10 (c). The same applies to the vertical direction component and the diagonal direction component. At this time, if the threshold A is 8 and the threshold B is 32, it is determined that the region shown in FIG. 10B where the relationship of threshold A <the total value <threshold B is satisfied includes a line appearing in the contour of the object. Can do. Note that threshold A <threshold B.

  For simpler contour extraction, it is also possible to use only the threshold value A and determine that a region where the relationship of threshold value A <the total value is satisfied includes a line that appears in the contour of the object.

  Further, whether or not a certain region determined to be a contour is a contour of a moving object is determined by whether or not the following condition 3 or condition 4 is satisfied.

  That is, the condition 3 is that the size of the motion vector of the region is less than the threshold value C, and the movement of the target moving object needs to move more than a certain amount.

  Condition 4 is that the magnitude of the difference vector between the motion vector of the region and the surrounding motion vectors is less than the threshold value D. This determines whether or not the moving object moves the same as the surroundings. The surrounding motion vector may not be one. The condition 4 in that case will be described. First, a plurality of surrounding motion vectors are extracted, and for each surrounding motion vector, the size of the difference vector from the region motion vector is obtained. Condition 4 in this case is that the sum of the magnitudes of the difference vectors is less than the threshold value D.

  In addition to the above, the following condition can be assumed for condition 4. For example, when a plurality of motion vectors are selected as the surrounding motion vectors, the sum of squares of the difference between the X direction component (horizontal direction component) of the motion vector between the region and the surrounding region, and the Y direction component (vertical method component) are also used. A value calculated by the sum of squared differences (hereinafter, variance) can be used as a reference. Condition 4 in this case is that the variance is less than the threshold value D. If the condition 4 is satisfied, the motion vector of the region is assumed to have the same direction and size as the surroundings, and is determined not to be a moving object. In addition, the calculation of the variance is not limited to this, and the product of the absolute value of the difference in magnitude of the motion vector and the absolute value of the difference in angle may be calculated as a sum of the surrounding areas. The present invention is not limited to this as long as it can be determined whether or not the motion vector of the region has a different direction and size from the surrounding motion vectors.

  When this condition 4 or 5 is satisfied, it is determined that the area is not a moving object area. Note that, as in the case where the entire image is encoded in the frame, the frame including the motion vector is waited without determining the outline in the frame not including the motion vector. This is because no motion can be detected from a frame having no motion vector.

  The moving object detection unit 106 determines that an area satisfying the condition 3 or 4 among the areas determined as the object outline from the above conditions 1 and 2 is not the outline of the moving object. This is because the contour of the moving object moves at a different speed from the surroundings.

  Next, in step S404, detection processing inside the moving object is performed. Specifically, the moving object detection unit 106 detects a region inside the moving object using the motion information input from the base layer decoding unit 102 and the stored detection result of the contour of the moving object. The detection result of the internal area is stored in the moving object detection unit 106.

  Here, a method for detecting the internal region will be described below.

  That is, the condition for determining that a certain region is inside the moving object is the case where the following condition 5 or 6 is satisfied.

  Condition 5 is that it is in the vicinity of the contour of the moving object or the region determined to be inside, and the variance of the magnitude and direction of the motion vector with the neighboring region is less than the threshold value E. The threshold value E is the contour of the moving object. This is a reference value for determining that the inside moves at the same speed.

  Condition 6 is that the moving object is surrounded by a contour or an area determined to be inside, because the moving object is surrounded by a contour.

  Next, in step S405, processing for removing erroneous detection of a moving object is performed. Specifically, the moving object detection unit 106 removes the erroneously detected area from the stored moving object outline and internal area detection result, generates a moving object detection result, and generates a detection result output unit 107. The data is output to the band synthesis unit 104.

  The condition for determining this erroneously detected area is that there are few areas around the contour or inside of the moving object, and if a too small moving object is detected, the possibility of erroneous detection is high. It is.

  The moving object detection unit 106 generates a moving object detection result from the moving object region obtained as described above. The moving object detection result is, for example, as follows.

  The first is information that describes whether or not the region is a moving object for each region. Second, one rectangle or ellipse that circumscribes one moving object is defined. Information describing the coordinates and size of each ellipse.

  If information inside the moving object is not required, the internal detection process may be omitted.

  Further, the moving object detection method is not limited to the moving object detection method using the motion vector, and other methods may be used in combination with the edge information of the present invention.

  According to the moving object detection method of the present embodiment, if the base layer includes a motion vector and the enhancement layer includes even a code up to a bit plane of a certain bit level, the transmission is at a low bit rate, and the image quality is Even if it is bad, it is possible to detect a moving object at high speed, high accuracy, and low processing load.

  Next, in step S303, the result of detecting the moving object is output. Specifically, the detection result output unit 107 outputs the coordinates of the area of the moving object input from the moving object detection unit 106 to the outside.

  Next, in step S304, base layer decoding processing is performed. Specifically, the base layer decoding unit 102 performs motion prediction compensation decoding on the base layer of the video stream input from the stream input unit 101 to generate a reduced image, and outputs the reduced image to the band synthesis unit 104.

  Next, in step S305, enhancement layer decoding processing is performed. Specifically, the enhancement layer decoding unit 103 performs bit plane decoding on the enhancement layer of the video stream input from the stream input unit 101 to generate a horizontal component, a vertical component, and a diagonal component, and a band synthesis unit To 104.

  Next, in step S306, band synthesis processing is performed. Specifically, the band synthesizing unit 104 performs band synthesis on the reduced image input from the base layer decoding unit 102 and the horizontal direction component, the vertical direction component, and the diagonal direction component input from the enhancement layer decoding unit 103, thereby decoding the decoded image. Is output to the video output unit 105. Further, the band synthesizing unit 104 may emphasize a region including the moving object in the decoded image using the moving object detection result input from the moving object detecting unit 106.

  Here, enhancement of the area of the moving object will be described. For example, the band synthesizing unit 104 performs processing such as coloring the decoded video only in the moving object region or surrounding the moving object region with a frame. In addition, an image including only the outline may be generated by performing band synthesis by setting all pixel values of the reduced image obtained by decoding the base layer to “0”, and further, the region portion of the moving object may be emphasized.

  In this way, only the moving object is very conspicuous in the image composed of contours, and it is easy for a monitor who watches a plurality of monitoring images simultaneously to detect the occurrence of an abnormality or a suspicious person. In addition, when the bit rate of the base layer is very low due to the limitation of the communication speed and only an image with extremely bad image quality can be generated, the contour alone may rather recognize the details. Alternatively, even in an environment where the processing capability is limited, such as when displaying a plurality of camera images, it is possible to make it easier to see a region important for monitoring with a lower processing load by displaying only the contour.

  Next, in step S307, video output processing is performed. Specifically, the video output unit 105 outputs the decoded video input from the band synthesis unit 104 to the outside.

  Note that it is possible to detect only a moving object without performing the decoding process. At this time, a video cannot be obtained, but the video output process (step S307) is performed from the base layer decoding process (step S304). Thus, the moving object can be detected at a higher speed and with a lower processing load.

  Next, in step S308, an end determination process is performed. When the stream input unit 101 determines whether or not there is a subsequent video stream, the video decoding apparatus 100 ends the process if it is not necessary to detect a moving object anymore or to decode the video, and so on. Return to step S301.

  In the above description, the base layer decoding process (step S304) to the video output process (step S307) are performed after the moving object detection process (steps S302 and S303). However, the present invention is not limited to this. In parallel with the enhancement layer decoding process, the moving object detection process can be performed.

  In addition, as a method for generating a video stream by another encoding method using band division, there is a method of performing bit division encoding after performing motion prediction compensation on an input image and performing bit-plane encoding. However, in this method, even if the image obtained by performing the motion prediction compensation is different from the previous and subsequent images, the horizontal component, the vertical component, and the diagonal component generated in the contour of the object cannot be obtained. In this case, only the horizontal direction component, the vertical direction component, and the diagonal direction component of the image obtained by intra-frame encoding the whole are used.

  The enhancement layer can include not only the horizontal component, the vertical component, and the diagonal component, but also information on the difference between the reduced image and the image obtained by decoding the base layer.

  As described above, according to the first embodiment, the video including the information on the horizontal direction component, the vertical direction component, and the diagonal direction component obtained by direct band division of the input image, and the motion vector generated by the motion prediction compensation By providing means for extracting edge information and motion information from a stream, a base layer using motion predictive coding and bit plane coding of horizontal, vertical and diagonal components are used. A moving object can be detected at high speed, high accuracy, and low processing load without decoding a video stream composed of an enhancement layer.

  Also, according to the first embodiment, it is possible to extract motion information from the base layer video stream and edge information from the enhancement layer video stream, and when the motion information indicates no motion, the edge Processing such as information extraction can be stopped to reduce the processing load. Also, when edge information indicates that there is no edge, processing such as motion information extraction is stopped to reduce the processing load. The contour of the object can be detected at high speed. At this time, either the motion information or the edge information may be extracted first or in parallel.

  In addition, according to Embodiment 1, since the detection of a moving object can be performed using only motion vectors and edge information of some bit planes, a low bit rate video stream such as a situation where communication speed is limited. Even in such a case, it is possible to detect a moving object at high speed and high efficiency.

  Further, in the first embodiment, the enhancement layer decoding unit 103 performs extraction of edge information necessary for detecting a moving object, and the base layer decoding unit 102 performs extraction of motion information. Since the moving object detection process can share some means and processes, it is possible to simultaneously detect the moving object and decode the video at high speed, and to increase the scale of the entire apparatus. Can be small.

  Further, according to the first embodiment, the enhancement layer decoding unit 103 searches and identifies the start signal included in the bit plane header 601 in the video stream and the termination signal 702 for each region such as 8 × 8 pixels. Edge information can be generated at high speed only by counting the code length between signals.

  Further, according to Embodiment 1, the base layer decoding unit 102 searches for an identification signal for each region such as 8 × 8 pixels in the video stream, and moves at a position determined from the identification signal. It is possible to generate motion information at high speed only by decoding a vector.

  Further, according to the first embodiment, the moving object detection unit 106 detects the contour of the moving object based on the edge information and the motion information, detects the inside of the moving object based on the motion information and the already detected result, and performs false detection. By performing the removal, it is possible to detect the moving object with high accuracy.

  Further, in the first embodiment, the band synthesis unit 104 emphasizes the area of the moving object in the decoded video, or uses a line drawing that does not band synthesize the reduced video obtained by decoding the base layer. It is possible to make it easy for the observer to detect the detection result.

(Embodiment 2)
In the second embodiment, the moving object detection method and apparatus according to the present invention are applied to a video surveillance system. The video surveillance system has an automatic tracking camera equipped with a video encoding device. In other words, the video is encoded and the video stream is generated, and at the same time, the moving object in the video is detected with high speed, high accuracy and low processing load, and the automatic tracking camera automatically tracks the moving object based on the detection result, The video can be monitored efficiently.

  The video monitoring system will be specifically described below.

  FIG. 11 is a diagram showing a configuration of a video surveillance system according to Embodiment 2 to which the moving object detection method and apparatus of the present invention is applied.

  This video monitoring system includes a video monitoring device 1100, a communication network 1110, and N automatic tracking cameras 1121 to 112N. The automatic tracking camera corresponds to the imaging device of the present invention.

  FIG. 12 is a block diagram illustrating a configuration of the automatic tracking cameras 1121 to 112N according to the second embodiment. The automatic tracking camera shown in FIG. 12 corresponds to the automatic tracking camera 1121 in the video surveillance system shown in FIG.

  In FIG. 12, the automatic tracking camera 1121 includes an imaging unit 1201, a video encoding unit 1202, and an imaging control unit 1203. The other automatic tracking cameras 1122 to 112N have the same configuration.

  The imaging unit 1201 corresponds to the imaging unit of the present invention, and the imaging control unit 1203 corresponds to the imaging control unit of the present invention.

  Here, the imaging unit 1201 performs imaging function operations such as panning, tilting, and zooming, and outputs an image captured to the video encoding unit 1202.

  The video encoding unit 1202 divides the input video into bands, and generates a video stream including information on the horizontal direction component, the vertical direction component, and the diagonal direction component, and a motion vector generated by motion prediction compensation.

  The imaging control unit 1203 inputs target information to be tracked and the result of detection of a moving object, and generates and outputs a control signal for performing pan / tilt / zoom to the imaging unit 1201.

  FIG. 13 is a block diagram showing a configuration of the video encoding unit 1202, and corresponds to a video encoding device to which the moving object detection method and apparatus of the present invention is applied.

  In FIG. 13, a video encoding unit 1202 includes a video input unit 1301, a band division unit 1302, a base layer encoding unit 1303, an enhancement layer encoding unit 1304, a stream output unit 1305, a moving object detection unit 1306, and a detection result output. Part 1307.

  Note that the band division unit 1302, the base layer encoding unit 1303, and the enhancement layer encoding unit 1304 correspond to the video encoding unit of the present invention, the base layer encoding unit 1303 corresponds to the motion information extraction unit, and the enhancement layer. The encoding unit 1304 corresponds to edge information extraction means, and the moving object detection unit 1306 corresponds to moving object detection means.

  Here, the video encoding means encodes the input video to generate and output a video stream. The band dividing unit 1302 constituting this generates a reduced image, a horizontal component, a vertical component, and a diagonal component by dividing the input image into bands, or the reduced image is subjected to motion prediction compensation encoding to independently generate a video. Encode as a decodable base layer. Also, these horizontal direction component, vertical direction component and diagonal direction component are encoded as an enhancement layer by bit plane encoding. The base layer encoding unit 1303 extracts motion information from the generated video stream and outputs the motion information to the moving object detection unit 1306. The enhancement layer encoding unit 1304 extracts edge information from the generated video stream and outputs it to the moving object detection unit 1306. The moving object detection unit 1306 detects a moving object from the input edge information and motion information. Note that the stream output unit 1305 and the detection result output unit 1307 correspond to the output unit of the present invention.

  Next, the operation of the automatic tracking camera 1121 according to this embodiment will be described. FIG. 14 is a flowchart showing the operation of the automatic tracking camera 1121 shown in FIG. The flowchart shown in FIG. 14 is executed by software by executing a control program stored in a storage device (not shown) such as a ROM or a flash memory by a CPU (not shown). It is also possible to do so.

  First, in step S1401, an imaging process is performed. Specifically, the imaging unit 1201 captures an image to be monitored and outputs an input image to the video input unit 1301 of the video encoding unit 1202. Further, the imaging unit 1201 outputs pan / tilt / zoom and installation location information to the detection result output unit 1307 of the video encoding unit 1202.

  Next, in step S1402, a video encoding process is performed. The video encoding unit 1202 encodes the input video input from the imaging unit 1202 to generate a video stream, and simultaneously detects a moving object to generate a moving object detection result. The generated video stream and the moving object detection result are output to the receiving unit 1101 of the video monitoring apparatus 1100 via the communication network 1110. The moving object detection result is output to the imaging control unit 1203.

  Next, in step S1403, an imaging control process is performed. Specifically, the imaging control unit 1203 performs panning according to a target tracking command input from the camera group control unit 1102 of the video monitoring apparatus 1100 via the communication network 1100 and a moving object detection result input from the video encoding unit. A tilt / zoom control signal is generated and output to the imaging unit 1201. The imaging unit 1201 performs pan / tilt / zoom based on a control signal input from the imaging control unit 1203.

  Here, this control signal will be described. For example, when a target tracking command generated by the video monitoring apparatus 1100 to be described later specifies coordinates or an enlargement ratio for photographing a suspicious person to be supplemented, the imaging control unit 1203 performs pan / A control signal for tilting and zooming is generated. If there is a deviation between the coordinates for photographing the suspicious person to be supplemented and the coordinates of the moving object area indicated by the moving object detection result, the imaging control unit 1203 may correct the deviation and generate a control signal. Good. The camera may be panned so that the moving object to be tracked always occupies a certain area with respect to the screen. When there is no target tracking command but there is a moving object detection result, the moving object is photographed with the center of the image. Further, the control signal may be generated so that a plurality of moving objects all fit in the video. In addition, when there is neither a target tracking command nor a moving object detection result, a control signal for causing the imaging unit 1201 to swing motion may be generated for the purpose of photographing a wide range.

  Next, in step S1404, if video monitoring is not necessary, for example, the automatic tracking camera 1121 is turned off, the process ends. Otherwise, the process returns to step S1401.

  Here, the video encoding process in step S1402 in FIG. 14 will be described in detail.

  FIG. 15 is a flowchart showing the operation of the video encoding unit 120. In the flowchart shown in FIG. 15, a control program stored in a storage device (not shown) (for example, a ROM or a flash memory) is executed by a CPU (not shown) and executed in software by executing the program. Is also possible.

  First, in step S1501, video input processing is performed. Specifically, the video input unit 1301 inputs an input image from the imaging unit 1201 of the automatic tracking camera 1121 and outputs the input image to the band dividing unit 1302.

  Next, in step S1502, band division processing is performed. Specifically, the band dividing unit 1302 performs band division on the input image input from the video input unit 1301 to generate a reduced image, a horizontal component, a vertical component, and a diagonal component, and the reduced image is converted into a base layer code. The horizontal direction component, the vertical direction component, and the diagonal direction component are output to enhancement section coding section 1304.

  Next, in step S1503, a base layer encoding process is performed. Specifically, base layer coding section 1303 generates a base layer by performing motion prediction compensation coding on the reduced image input from band dividing section 1302, and outputs the base layer to stream output section 1305. Also, motion information obtained in motion prediction compensation is output to the moving object detection unit 1306.

  Next, in step S1504, an enhancement layer encoding process is performed. Specifically, enhancement layer encoding section 1304 generates an enhancement layer by bit-plane encoding the horizontal direction component, vertical direction component, and diagonal direction component input from band division section 1302, and outputs them to stream output section 1305. To do. Also, the edge information obtained at the time of bit plane coding is output to the moving object detection unit 1306.

  Next, in step S1505, stream output processing is performed. Specifically, the stream output unit 1305 receives the base layer input from the base layer encoding unit 1303 and the enhancement layer input from the enhancement layer encoding unit 1304 via the communication network 1110, the receiving unit of the video monitoring apparatus 1100. 1101 is output.

  Next, in step S1506, a moving object detection process is performed. Specifically, the moving object detection unit 1306 detects the moving object using the motion information input from the base layer encoding unit 1303 and the edge information input from the enhancement layer encoding unit 1304, and the moving object detection result Is output to the detection result output unit 1307.

  Note that the method for detecting a moving object is the same as that in the first embodiment, and thus will not be described in detail here.

  In step S1507, detection result output processing is performed. Specifically, the detection result output unit 1307 receives the moving object detection result input from the moving object detection unit 1306 and information such as pan / tilt / zoom and installation position input from the imaging unit 1201 of the automatic tracking camera 1121. The data is output to the reception unit 1101 of the video monitoring apparatus 1100 via the communication network 1110.

  As with the video decoding apparatus described in the first embodiment, the present embodiment also includes information on the horizontal direction component, the vertical direction component, and the diagonal direction component, and a motion vector generated by motion prediction compensation. If a video stream can be generated, other band dividing methods can be used.

  Next, the configuration of video monitoring apparatus 1100 according to the present embodiment will be described below.

  In FIG. 11, the video monitoring apparatus 1100 includes a reception unit 1101, an image recognition unit 1102, and a camera group control unit 1103.

  The image recognizing unit 1102 corresponds to the image recognizing unit of the present invention, receives a video stream and a moving object detection result, performs detailed image recognition, and outputs the image recognition result to the camera group control unit 1103.

  The camera group control unit 1103 corresponds to the camera group control means of the present invention, receives the image recognition result, and generates and outputs target information to be tracked for the cameras 1121 to 112N.

  Next, the operation of the video monitoring apparatus 1100 configured as described above will be described.

  FIG. 16 is a flowchart showing the operation of the video monitoring apparatus 1100.

  First, in step S1601, reception processing is performed. Specifically, the receiving unit 1101 inputs the video stream and the moving object detection result from the automatic tracking camera 1121 via the communication network 1110 and outputs them to the image recognition unit 1102.

  Next, in step S1602, image recognition processing is performed. Specifically, the image recognition unit 1102 decodes the video stream using the video stream input from the reception unit 1101 and the moving object detection result, and detects and authenticates a person / face / object using various known image recognition methods. The result is generated and output to the camera group control unit 1103. Further, the image recognition unit 1102 can speed up the processing by not performing image recognition except for the moving object region included in the moving object detection result.

  In step S1603, camera control processing is performed. Specifically, the camera group control unit 1103 generates a target tracking command for the automatic tracking camera 1121 using the image recognition result input from the image recognition unit 1102, and performs imaging control of the automatic tracking camera 1121 via the communication network 1110. To the unit 1203. Further, when the image recognition result for the automatic tracking camera 1121 necessitates new tracking for the other automatic tracking cameras 1122 to 112N, a new target tracking command is generated and the corresponding automatic tracking camera is transmitted via the communication network 1110. The image is output to the imaging units 1203 of 1122 to 112N.

  Here, the target tracking command will be described.

  For example, when the image recognition result input from the image recognition unit 1102 indicates that there is a suspicious person in the video, the camera group control unit 1103 includes a target including coordinates, an enlargement ratio, and the like in order to photograph the suspicious person greatly. Generate a tracking instruction. If there is a suspicious person in the video but the automatic tracking camera 1121 is unable to capture the face of the suspicious person, it generates a target tracking command that causes the automatic tracking camera 1122 to image the suspicious person, A target tracking command for causing the automatic tracking camera 1121 to capture a wide range including a suspicious person is generated.

  Next, in step S1604, an end determination is performed. If it is not necessary to perform video monitoring, such as when the video monitoring apparatus 1100 is turned off, the process ends. Otherwise, the process returns to step S1601.

  The operation of the video surveillance system configured as described above will be described below.

  FIG. 17 is a sequence diagram showing the operation of the video monitoring system of the present embodiment.

  First, when the automatic tracking camera 1121 captures a monitoring target, the automatic tracking camera 1121 generates a video stream including information on the horizontal direction component, the vertical direction component, and the diagonal direction component, and a motion vector generated by motion prediction compensation. Detection results are obtained and transmitted to the video monitoring apparatus 1100 via the communication network 1110 (step S1701).

  The video monitoring apparatus 1100 decodes the received video stream, and recognizes the target object using information on the moving object detection result. Then, a target tracking command for tracking the target object is transmitted to the automatic tracking camera (step S1702).

  In response to this, the automatic tracking camera 1121 controls the imaging unit to track the object. Then, the video stream at this time is transmitted to the video monitoring apparatus 1100 (step S1703).

  Thereafter, step S1702 and step S1703 are repeated. Note that a video stream or the like from the automatic tracking camera 1121 is always transmitted to the video monitoring device 1100 regardless of whether there is a command from the video monitoring device 1100.

  As described above, the video monitoring system according to the present embodiment needs to encode the video into a video stream that is encoded and compressed in order to transmit the video from the automatic tracking camera to the video monitoring apparatus via the communication network. There is. At this time, according to the present invention, in the process of generating the video stream, the moving object detection can be performed at the same time, and the result information can be notified to the video monitoring apparatus, so that the video monitoring apparatus moves from the newly received video stream. There is no need to detect an object. Thereby, the processing of the video monitoring apparatus can be reduced.

  Further, according to the second embodiment, in an image monitoring system that receives an image captured by an automatic tracking camera at a remote location and monitors and tracks the image with the image monitoring device, the automatic tracking camera is partially Processing to encode the video stream into a video stream including information on the horizontal component, vertical component and diagonal component of the captured image, and a motion vector generated by motion prediction compensation. Since the moving object detection process can be performed, it is possible to simultaneously detect the moving object with high accuracy and to encode the video at high speed, and to reduce the scale of the entire system.

  Further, according to the second embodiment, the automatic tracking camera can control the pan / tilt / zoom imaging function in accordance with an instruction from the video monitoring apparatus obtained based on the detection result of the moving object. It is possible to efficiently monitor a moving object and eventually a suspicious person.

  In addition, according to the second embodiment, the video monitoring apparatus recognizes an image of only the moving object region based on the detection result of the moving object that is input together with the video stream, thereby reducing the load of the image recognition process. And the accuracy of image recognition is improved. In addition, this makes it possible to provide a video monitoring system that can control and monitor more automatic tracking cameras efficiently.

(Embodiment 3)
Embodiment 3 is a moving object detection method and apparatus according to the present invention.

  In the present embodiment, a method for detecting a moving object using only the video stream of the enhancement layer among the video streams composed of the base layer and the enhancement layer as in the first embodiment will be described. The enhancement layer video stream handled in this embodiment is an enhancement layer video stream such as MPEG-4 FGS (Fine Granularity Scalable coding) FGST (FGS Temporal Scalability) defined in ISO / IEC 14496-2 Amendment 2. It is assumed that motion vector information is included at the beginning of the frame.

  FIG. 19 is a block diagram showing a configuration of a moving object detection apparatus 1900 according to Embodiment 1 to which the moving object detection method and apparatus of the present invention is applied.

  19, the moving object detection apparatus 1900 includes a stream input unit 1901, a motion information extraction unit 1902, an edge information extraction unit 1903, a moving object detection unit 1904, and a detection result output unit 1905.

  In the present embodiment, unlike the first embodiment, the stream input unit 1901 inputs only an enhancement layer video stream.

  Note that the motion information extraction unit 1902 corresponds to the motion information extraction unit, the edge information extraction unit 1903 corresponds to the edge information extraction unit, and the moving object detection unit 1904 corresponds to the moving object detection unit.

  Here, the motion information extraction unit extracts the motion information from the input enhancement layer video stream and outputs the motion information to the moving object detection unit. The edge information extraction unit extracts edge information from the input enhancement layer video stream and outputs the extracted edge information to the moving object detection unit. The moving object detection means detects a moving object from the input edge information and motion information.

  Next, the operation of the moving object device 1900 configured as described above will be described.

  FIG. 20 is a flowchart showing the operation of the moving object device 1900 of the third embodiment shown in FIG. The flowchart shown in FIG. 20 is executed by software by executing a control program stored in a storage device (not shown) such as a ROM or a flash memory by a CPU (not shown). It is also possible to do so.

  First, the stream input unit 1901 inputs an enhancement layer video stream from the outside of the moving object detection apparatus 1900, and outputs it to the motion information extraction unit 1902 and the edge information extraction unit 1903 (step S2001).

  Next, the motion information extraction unit 1902 extracts motion information from the enhancement layer input from the stream input unit 1901 and outputs the motion information to the moving object detection unit 1904 (step S2002).

  Next, the edge information extraction unit 1903 extracts edge information from the enhancement layer input from the stream input unit 1902 and outputs it to the moving object detection unit 1904 (step S2003).

  Here, in FGST defined by MPEG-4 FGS, the motion vector of the entire frame area is stored at the head of the enhancement layer of one frame, and subsequently the information of the bit plane is stored. Accordingly, the stream input unit 1901 inputs up to the motion vector video stream, the motion information extraction unit 1902 generates motion information, and the bit plane video stream is input only when there is motion in the frame, and the edge information extraction unit 1903. May be output. As a result, when there is no motion in the frame, it is possible to omit the stream input process, the edge extraction process, and the moving object detection process, thereby reducing the processing load.

  Next, the moving object detection unit 1904 detects the moving object using the motion information input from the motion information extraction unit 1902 and the edge information input from the edge information extraction unit 1903, and moves as in the first embodiment. An object detection result is generated and output to the detection result output unit 1905 (steps S2004 to S2006).

  Next, the result of detecting the moving object is output. Specifically, the detection result output unit 1905 outputs the coordinates of the area of the moving object input from the moving object detection unit 1904 to the outside (step S2007).

  Next, end determination processing is performed. The stream input unit 1901 determines whether or not there is a subsequent video stream. If the moving object detection apparatus 1900 does not detect any more moving objects, the process ends. Otherwise, the process returns to step S2001 (step S2001). S2008).

  As described above, according to the third embodiment, only the enhancement layer video stream is input, the motion information extraction unit 1902 extracts the motion information, and the edge information extraction unit 1903 extracts the edge information. The contour of an object can be detected with high speed and a small video stream.

  The present invention is useful for a moving object detection apparatus that detects a moving object from a video stream generated by encoding video, and is suitable for detecting a moving object at high speed without decoding the video stream.

The figure which shows the structure of the video decoding apparatus by Embodiment 1 of this invention. Conceptual diagram of bit plane encoding in Embodiment 1 of the present invention The flowchart which shows operation | movement of the video decoding apparatus by Embodiment 1 of this invention. The flowchart which shows the operation | movement of the moving object detection process of the video decoding apparatus by Embodiment 1 of this invention. Stream structure diagram of enhancement layer in Embodiment 1 of the present invention Stream structure diagram of bit plane k of the enhancement layer in Embodiment 1 of the present invention Stream structure diagram of bit plane k of region j of the enhancement layer according to Embodiment 1 of the present invention Base layer stream structure diagram in Embodiment 1 of the present invention Stream structure diagram of base layer region j in Embodiment 1 of the present invention (A) thru | or (c) The figure showing the horizontal direction component in the 8x8 pixel area | region in Embodiment 1 of this invention. The figure which shows the structure of the video surveillance system by Embodiment 2 of this invention. The figure which shows the structure of the automatic tracking camera by Embodiment 2 of this invention. The figure which shows the structure of the video coding apparatus by Embodiment 2 of this invention. The flowchart which shows operation | movement of the automatic tracking camera by Embodiment 2 of this invention. The flowchart which shows operation | movement of the video coding apparatus by Embodiment 2 of this invention. The flowchart which shows operation | movement of the image | video monitoring apparatus by Embodiment 2 of this invention. Sequence diagram showing the operation of the video surveillance system according to the second embodiment of the present invention The figure which shows the structure of the conventional moving object detection apparatus The figure which shows the structure of the video decoding apparatus by Embodiment 3 of this invention. The flowchart which shows operation | movement of the video decoding apparatus by Embodiment 3 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Video decoding apparatus 101 Stream input part 102 Base layer decoding part 103 Enhancement layer decoding part 104 Band synthesis part 105 Video output part 106,1306 Moving object detection part 107,1307 Detection result output part 1100 Video monitoring apparatus 1101 Reception part 1102 Image recognition unit 1103 Camera group control unit 1110 Communication network 1121 to 112N Automatic tracking camera 1201 Imaging unit 1202 Video coding unit 1203 Imaging control unit 1301 Video input unit 1302 Band division unit 1303 Base layer coding unit 1304 Enhancement layer coding unit 1305 Stream output unit 1801 Variable length decoding unit 1802 Pattern information detection unit 1803 Moving object detection processing unit 1900 Moving object detection device 1901 Stream input unit 1902 Motion information extraction unit 1903 Edge information extraction 1904 moving object detection unit 1905 the detection result output unit

Claims (23)

Motion information extraction means for extracting motion information from a video stream that has been encoded using hierarchical encoding and motion predictive compensation encoding that divides and encodes a video into a plurality of layers;
Edge information extracting means for extracting edge information from the video stream;
A moving object detecting means for detecting a moving object using the movement information and the edge information and outputting the detection result;
A moving object detection apparatus having: The edge information extracting means extracts bit plane information from the most significant bit plane to N (N is a natural number) bit-order bit plane information as edge information from the bit plane information obtained by bit plane encoding the image. The moving object detection device according to claim 1. The video stream is divided into a plurality of areas,
3. The moving object detection unit according to claim 2, wherein the moving object detection unit determines the area as a contour area of the moving object when a total code length of bit plane information in the area is equal to or greater than a predetermined first value. Moving object detection device. The moving object detection means determines the area as a contour area of the moving object when the total code length of the bit plane information inside the area is equal to or less than a predetermined second value. The moving object detection device described. The motion information extraction unit extracts a motion vector from an area determined to be a contour region of the moving object, and the moving object detection unit has a case where the magnitude of the motion vector is a predetermined third value or more. The moving object detection device according to claim 3, wherein the area is determined to be a contour area of a moving object. The motion information extraction unit extracts a first motion vector from an area determined to be a contour area of the moving object, selects an area located in the vicinity of the area, and selects a second motion vector from the selected area. The moving object detecting means calculates a difference vector between the first motion vector and the second motion vector as a measured value, and the measured value is a fourth value determined in advance. The moving object detection device according to claim 3, wherein the selected area is determined to be an internal area of a moving object when: The motion information extraction means selects a plurality of regions, extracts a motion vector from each selected region,
The moving object detection means obtains the size of the difference vector between the first motion vector and the motion vector of the selected region for each of the selected regions, and calculates the size of the difference vector for all the selected regions. The moving object detection apparatus according to claim 6, wherein a total is calculated as the measurement value. The moving object detection means has a fifth difference vector in which a magnitude of a difference vector between a motion vector of an area determined to be an internal area of the moving object and a motion vector of an area located in the vicinity of the area is predetermined. The moving object detection device according to claim 6, wherein if the value is equal to or smaller than the value, the moving object detection device determines that the area is an internal area of the moving object area. 9. The moving object detection unit determines that an area surrounded by a contour area of the moving object or an area determined to be an internal area of the moving object is an internal area of the moving object. Moving object detection device. The moving object detection means has a predetermined number of regions determined to be the contour region or the internal region of the second moving object in the vicinity of the contour region or the internal region determined to be the first moving object. The moving object detection device according to claim 3, wherein, when the value is equal to or larger than a sixth value, the contour area or the internal area determined as the first moving object is re-determined as the first moving object. A method for detecting a moving object from a video stream, which is executed by a moving object detection device that detects the moving object.
A step of extracting motion information from a video stream that has been encoded using hierarchical encoding that divides and encodes a video into a plurality of layers, and motion prediction compensation encoding;
Extracting edge information from the video stream;
Detecting a moving object using the extracted motion information and edge information;
A moving object detection method comprising: To detect moving objects from the video stream,
Extracting motion information from a video stream that has been video-encoded using hierarchical encoding and motion predictive compensation encoding that divides and encodes the video into multiple layers;
Extracting edge information from the video stream;
Detecting a moving object using the extracted motion information and edge information;
Moving object detection program to execute. Video decoding means for decoding a video stream encoded by hierarchical encoding and motion prediction compensation encoding for encoding video into a plurality of layers;
Moving object detection means for detecting a moving object from motion information and edge information extracted when the video decoding means decodes the video stream;
A video decoding apparatus comprising: The video stream is divided into a plurality of areas,
The moving object detection means determines the area as a contour area of a moving object when the total code length of bit plane information inside the area is equal to or greater than a predetermined first value. Video decoding device. The moving object detection unit determines the area as a contour area of a moving object when a total code length of the bit plane information in the area is equal to or less than a predetermined second value. Video decoding device. The video decoding device according to claim 15, wherein the video decoding unit generates a video in which a region of the moving object detected by the moving object detection unit is emphasized. The video decoding means generates a video composed of edge components,
14. The video decoding apparatus according to claim 13, wherein the moving object area detected by the moving object detection means is displayed with emphasis. Video encoding means for generating a video stream encoded using hierarchical encoding and motion prediction compensation encoding for encoding video into a plurality of layers;
Moving object detecting means for detecting movement objects by extracting motion information and edge information of the video when the video encoding means encodes the video;
A video encoding device. Imaging means for inputting video;
A video encoding device according to claim 18,
An imaging control unit that controls an imaging function for the imaging unit based on a detection result of the moving object output by the moving object detection unit;
An output unit for outputting the video stream and the detection result of the moving object;
An imaging apparatus having The imaging apparatus according to claim 19, wherein the imaging control means controls the imaging means so that the area of the moving object output by the moving object detection means is a constant ratio with respect to the total area of the input video. . An imaging device according to claim 19,
A video monitoring device that decodes the video stream received from the imaging device and performs image recognition of the detected moving object region using the detection result of the moving object;
A video surveillance system. The video stream is encoded by being layered into a base layer and an enhancement layer,
The motion information extraction means extracts the motion information from the base layer video stream,
The video decoding device according to claim 1, wherein the edge information extraction unit extracts the edge information from the video stream of the enhancement layer. The video stream is encoded by being layered into a base layer and an enhancement layer,
The motion information extraction means extracts the motion information from the enhancement layer video stream,
The video decoding device according to claim 1, wherein the edge information extraction unit extracts the edge information from a video stream of an enhancement layer.

Images