WO2018148889A1 - Video analysis system and video analysis method based on video surveillance - Google Patents

Abstract

Embodiments of the present application provide a video analysis system and a video analysis method based on video surveillance. The video analysis system comprises: a video surveillance subsystem, is used for performing photographing to obtain a video surveillance image; a video analysis subsystem, used for analyzing the video surveillance image obtained by the video surveillance subsystem; and an information publishing subsystem, used for sending the video surveillance image and the analysis result. According to the embodiments, a video surveillance image obtained by the video analysis system can be automatically analyzed and the analysis result is sent to a user, thereby improving the automation degree and efficiency of video surveillance and reducing labor costs.

Description

Video analysis system and video analysis method based on video surveillance Technical field

The present application relates to the field of information technology, and in particular, to a video analysis system and a video analysis method based on video surveillance.

Background technique

In recent years, video surveillance systems have become more and more popular in China, and are widely used in public places such as transportation, banking, supermarkets, etc., and play an increasingly important role in the field of public security. Video surveillance systems typically acquire surveillance video images from surveillance cameras and send them to the user for browsing, playback, processing, and analysis.

In general, video surveillance-based analysis systems can be embedded in cameras or based on front-end embedded devices and servers. The former has a single function, and the processing speed and performance are greatly limited; the latter has strong processing power and greater flexibility.

It should be noted that the above description of the technical background is only for the purpose of facilitating a clear and complete description of the technical solutions of the present application, and is convenient for understanding by those skilled in the art. The above technical solutions are not considered to be well known to those skilled in the art simply because these aspects are set forth in the background section of this application.

Application content

The inventors of the present application have found that existing video surveillance based video analysis systems have many problems, such as:

1. High labor cost: The video obtained by the surveillance camera is almost always viewed by the human eye. Therefore, in a large-scale video surveillance system and its analysis system, it takes a lot of manpower to manage the monitoring system and maintain it all-weather;

2, low efficiency: after the event, you need to manually find evidence in a large amount of video data, sometimes it takes a long time and huge human resources to check specific objects in complex scenes;

3, low degree of automation: almost all of the event detection, data statistics, data reporting and other work based on monitoring results are done manually.

An embodiment of the present application provides a video analysis system and a video analysis method based on video surveillance, which can automatically analyze a video surveillance image acquired by a video analysis system and send the analysis result to a user, thereby improving video surveillance. The degree of automation and efficiency reduce labor costs.

According to a first aspect of the embodiments of the present application, a video monitoring system based on video surveillance is provided, including:

a video monitoring subsystem for taking a picture to obtain a video surveillance image;

a video analysis subsystem in communication with the video surveillance subsystem for analyzing the video surveillance image obtained by the video surveillance subsystem;

An information distribution subsystem, configured to communicate with the video analysis subsystem, for transmitting the video surveillance image obtained by the video monitoring subsystem and an analysis result of the video analysis subsystem,

The video analysis subsystem includes:

Motion analysis, which is used to detect the foreground of a video surveillance image;

Feature extraction unit for extracting features of the foreground;

a condition setting unit (rule definition) for setting a judgment condition corresponding to a scene to which the video monitoring is applied;

a rule comparison unit that compares the feature extracted by the feature extraction unit with the determination condition set by the determination condition setting unit to obtain the analysis result.

According to another aspect of the embodiments of the present application, a video monitoring method based on video surveillance is provided, including:

Shooting to obtain a video surveillance image;

Performing analysis on the video surveillance image to obtain an analysis result;

Transmitting the video surveillance image and the analysis result,

The analyzing the video surveillance image to obtain the analysis result includes:

Detecting the foreground of the video surveillance image;

Extracting features of the foreground;

Setting a judgment condition corresponding to a scenario applied by the video monitoring method;

The comparison is performed according to the extracted feature and the judgment condition to obtain the analysis result.

The beneficial effects of the present application are: improving the automation degree and efficiency of video monitoring, and reducing labor costs.

Specific embodiments of the present application are disclosed in detail with reference to the following description and accompanying drawings, in which <RTIgt; It should be understood that the embodiments of the present application are not limited in scope. The embodiments of the present application include many variations, modifications, and equivalents within the scope of the appended claims.

Features described and/or illustrated with respect to one embodiment may be used in one or more other embodiments in the same or similar manner, in combination with, or in place of, features in other embodiments. .

It should be emphasized that the term "comprising" or "comprises" or "comprising" or "comprising" or "comprising" or "comprising" or "comprises"

DRAWINGS

The drawings are included to provide a further understanding of the embodiments of the present application, and are intended to illustrate the embodiments of the present application Obviously, the drawings in the following description are only some of the embodiments of the present application, and those skilled in the art can obtain other drawings according to the drawings without any inventive labor. In the drawing:

1 is a schematic diagram of a video analysis system according to Embodiment 1 of the present application;

2 is a schematic diagram of a functional architecture of a video analysis subsystem according to Embodiment 1 of the present application;

3 is a schematic structural diagram of a video analysis subsystem according to Embodiment 1 of the present application;

4 is a schematic diagram of a video analysis method according to Embodiment 2 of the present application;

FIG. 5 is a schematic diagram of analyzing the video surveillance image according to Embodiment 2 of the present application.

detailed description

The foregoing and other features of the present application will be apparent from the description, The specific embodiments of the present application are specifically disclosed in the specification and the drawings, which illustrate a part of the embodiments in which the principles of the present application may be employed, it being understood that the present application is not limited to the described embodiments, but instead The application includes all modifications, variations and equivalents falling within the scope of the appended claims. Various embodiments of the present application will be described below with reference to the accompanying drawings. These embodiments are merely exemplary and are not limiting of the application.

Example 1

Embodiment 1 of the present application provides a video analysis system based on video surveillance.

1 is a schematic diagram of a video analysis system of Embodiment 1. As shown in FIG. 1, the video analysis system 100 can include a video surveillance subsystem 101, a video analysis subsystem 102, and an information distribution subsystem 103.

In this embodiment, the video monitoring subsystem 101 is configured to perform video capture to obtain a video surveillance image; the video analysis subsystem 102 is in communication with the video surveillance subsystem 101 for analyzing video surveillance images obtained by the video surveillance subsystem 101. The information distribution subsystem 103 is in communication with the video analysis subsystem 102 for transmitting the video surveillance image obtained by the video surveillance subsystem 101 and the analysis results of the video analysis subsystem 102.

As shown in FIG. 1, in the video analysis system of this embodiment, the video monitoring subsystem 101 acquires a video surveillance image and transmits the video surveillance image to the video analysis subsystem 102 in the form of a video stream. The video analysis subsystem 102 analyzes the received video surveillance image, by which, for example, event information and data information included in the video surveillance image can be obtained, and the video analysis subsystem 102 can analyze the result as an event and a data stream. The form (Event and Data Flow) is sent to the information distribution subsystem 103. The information distribution subsystem 103 can transmit the received analysis result to the user, whereby the user can obtain the analysis result based on the video surveillance image, and in particular, for the user who is in the production and operation line, can directly refer to the analysis. As a result, the corresponding processing is performed, and the users who are at the front of the production and operation can be, for example, a driver, a policeman or a forester.

In contrast, in the prior art, users often only obtain video surveillance images and very limited simple analysis results through video analysis systems, and cannot obtain these referenced analysis results directly from the video analysis system. The prior art video analysis system often only sends video surveillance images and simple analysis results to some users, and the user needs to perform further analysis to obtain the reference analysis results, so the convenience of the video analysis system in use. restricted.

In addition, in this embodiment, the video analysis subsystem 102 can also send the received video surveillance image to the information distribution subsystem 103 in the form of a video stream, and is distributed by the information distribution subsystem 103, whereby the user not only The analysis result of the video surveillance image can be obtained, and the original information of the video surveillance image can also be obtained.

The video analysis system of the embodiment can analyze the video surveillance image and transmit the analysis result, thereby improving the automation degree and efficiency of the video surveillance and reducing the labor cost.

In this embodiment, the video monitoring subsystem 101 may be an imaging device. For the structure of the imaging device, reference may be made to the prior art, which is not described in this embodiment.

In this embodiment, the analysis of the video surveillance image by the video analysis subsystem 102 may include: event detection, data statistics, and/or object searching. The event detection may be, for example, detecting a predetermined event, which may be, for example, slow traffic flow and/or personnel gathering, etc.; the data statistics may be, for example, statistics on data involved in the video surveillance image, such as a car. Data such as the flow rate and/or the speed of movement of the moving object; the object retrieval may be, for example, a retrieval of a predetermined object, such as a particular person, a particular shaped vehicle, and/or a particular license plate number, and the like.

2 is a schematic diagram of a functional architecture of the video analysis subsystem of the embodiment. As shown in FIG. 2, the video analysis subsystem 102 may include a motion analysis unit 201, a feature extraction unit 202, a judgment rule setting unit 203, and a rule comparison unit. 204.

In this embodiment, the motion analysis unit 201 may be configured to detect a foreground of the video surveillance image; the feature extraction unit 202 may be configured to extract features of the foreground; the determination condition setting unit 203 may be configured to set a determination condition, and the determination The condition may correspond to the scenario applied by the video analysis system 100; the determination unit 204 may perform comparison according to the feature extracted by the feature extraction unit 202 and the determination condition set by the determination condition setting unit 203 to obtain an analysis result.

In this embodiment, the motion analysis unit 201 may perform foreground detection on each frame image in the video surveillance image received by the video analysis subsystem 102 to detect the foreground of each frame image. Regarding the foreground detection method, reference may be made to the prior art, and this embodiment will not be described again.

In this embodiment, the feature extraction unit 202 may extract the feature of the foreground based on the foreground extracted by the motion analysis unit 201, and the feature of the foreground may include, for example, the position, motion speed, motion direction, and motion trajectory of the object in the foreground. (trajectory), size, texture, colour, and gradient. The feature extracted by the feature extraction unit 202 may be one or more of the above-listed features, or may extract features other than the enumerated features described above.

In this embodiment, the feature extraction unit 202 may further combine the extracted features to form a feature combination, where the feature combination may be a high-dimensional feature combination, and the feature combination may have the same number of types as the extracted feature. For example, the latitude of the feature combination can be 12 dimensions.

In this embodiment, the feature extraction unit 202 can select features for composing the feature combination according to the scenario applied by the video analysis system 100, thereby being able to determine the light conditions of each scene and the analysis results required by each scenario. The kind to select features for composing the feature combination, so that the video analysis system 100 of the present embodiment can be applied to different scenarios, thereby improving its scalability.

For example, when the scene applied by the video analysis system is traffic monitoring, the feature extraction unit 202 may select the motion combination, the texture, and the color gradient to form the feature combination; when the scene applied by the video analysis system is illegal parking monitoring The feature extraction unit 202 can select to combine the feature by location, motion trajectory, size, and texture.

In this embodiment, the feature extraction unit 202 can be based on a configuration file. The same scene is selected for the features that make up the feature combination, and the profile can correspond to the scene. In addition, the embodiment may also select features for composing the combination of features in other manners.

In this embodiment, the scenario applied by the video analysis system 100 may be, for example, traffic monitoring, security monitoring, forest monitoring, agricultural monitoring, or factory monitoring. In addition, the scenario applied by the video analysis system 100 of the present embodiment may not be limited thereto, and may be other scenarios than the enumerated scenarios described above.

In the present embodiment, the determination condition setting unit 203 can set the determination condition corresponding to the scene, for example, when the scene applied by the video analysis system is traffic monitoring, the judgment set by the determination condition setting unit 203 Conditions may include the location and size of the Region of Interest (RoI), the direction of the lane line, the lane function, the traffic light refreshing cycle, and the duration of the event. When the scene to which the video analysis system is applied is security monitoring, the determination condition set by the determination condition setting unit 203 may include the location of the Region of Interest (RoI) and The size, the density of the object, the speed of the object, and the frequency of the object.

In the present embodiment, the determination condition setting unit 203 can set the determination condition according to the configuration file corresponding to the scene. In addition, in this embodiment, the determination condition corresponding to the scene may be set in another manner.

In the present embodiment, the determination unit 204 can perform comparison based on the feature extracted by the feature extraction unit 202 and the determination condition set by the determination condition setting unit 203, and output the result of the analysis based on the comparison result. For example, when the comparison result is that the degree of aggregation of the object in the region of interest is above a predetermined threshold, the result of the analysis is that a person gathering event occurs.

FIG. 3 is a schematic structural diagram of a video analysis subsystem 102 of an embodiment of the present application, which can be used to implement the functions of the video analysis subsystem shown in FIG. 2.

As shown in FIG. 3, the video analysis subsystem 102 may include a backend analysis device 303 for event detection, data statistics, and video statistics. And/or object searching. In the present embodiment, the backend analysis device 303 can be implemented, for example, by a server.

As shown in FIG. 3, the video analysis subsystem 102 may further include a front end processing unit 301 and/or a front end analysis device 302.

In this embodiment, the front end processing section 301 can be used for event detection and/or data statistics on the video surveillance image. In this embodiment, the front end processing unit 301 can be embedded in the video monitoring subsystem 101, for example, The front end processing unit 301 can be embedded in the imaging device.

In this embodiment, the front end analysis device 302 can be used for event detection and/or data statistics on the video surveillance image. In this embodiment, the front end analysis device 302 can be disposed in an outdoor environment, and the front end analysis device 302 can be, for example, an industrial PC (Industrial PC), a digital processor (DSP), and/or a dedicated embedded device (specialized embedded). Device) and so on.

In the video analysis subsystem 102 of the present embodiment, the backend analysis device 303 can have the strongest data processing capability, the processing capability of the front end analysis device 302 is second, and the processing power of the front end processing unit 301 is the weakest. The processing unit 301 can perform relatively simple event detection and/or data statistics. The front-end analysis device 302 can perform relatively complex event detection and/or data statistics, and the back-end analysis device 303 can perform the most complicated event detection and/or data statistics. And can perform object retrieval.

In the present embodiment, the backend analysis device 303 can acquire the analysis result of the front end processing section 301 and/or the front end analysis device 302, whereby the backend analysis device 303 can be based on the front end processing section 301 and/or the front end analysis device 302. Analyze the results for analysis to improve efficiency. For example, in FIG. 3, the component of the video analysis subsystem 102 may further include a memory 304 capable of storing analysis results of the front end processing section 301 and/or the front end analysis device 302, and the backend analysis device 303. The analysis result can be read from the memory 304.

In this embodiment, since the backend analysis device 303 has the strongest data processing capability, the backend analysis device 303 can also have at least one of the following functions:

1. Monitor the working state of the video analysis subsystem 102. The working status includes, for example, a device connection status, a device initialization status, a control device load, and/or a storage space and a usage rate of the memory. Further, in the case where the working state is abnormal, the backend analyzing device 303 can issue an alarm signal to notify the administrator.

2. Initialize the analysis of video analysis subsystem 102. For example, the backend analysis device 303 can initialize the analysis of the video analysis subsystem 102 by setting a configuration file, which can include setting the location and size of the region of interest (ROI) for event detection. Parameters, and / or format the user's configuration data. In addition, the back-end analysis device 303 can set the initialized content according to the application scenario of the video analysis system 100. For example, in the traffic monitoring scenario, the initialized content may include: setting a lane or road area to be observed, and setting an event detection center. The required threshold, and the format of the road congestion index (jam index format) according to different needs.

3. Manage user information of the video monitoring subsystem 101. For example, the backend analysis device 303 can set different security levels for the users of the video analytics system 100 and save and track the user's usage records.

4. Manage the analysis results of the video analysis subsystem 102. For example, the backend analysis device 303 can save, update, generate an event list, generate a report to be published, and/or set a template of a report to be published, and the like. The report to be published may be, for example, a combination of text and images. For example, the report to be released may be information about a road section in which a traffic accident occurs, and the information to be released may include text, and may further include a traffic accident. Surveillance video image of the road segment.

In this embodiment, the video analysis subsystem 102 can be set independently of the video monitoring subsystem 101. Thereby, the analysis function of the video analysis subsystem 102 can be set according to the application scenario of the video analysis system 100, thereby improving the The scalability of the video analytics subsystem 102 is set.

In this embodiment, the information distribution subsystem 103 may send the video surveillance image obtained by the video monitoring subsystem 101 and the analysis result of the video analysis subsystem 102 to the user via a network, such as a local area network (LAN) or Wireless fidelity (Wi-Fi) network, etc. In addition, in this embodiment, the user can receive the video surveillance image and the analysis result via the terminal device.

In the present embodiment, the information distribution subsystem 103 can also be set independently of the video monitoring subsystem 101.

In the present embodiment, a plurality of video analysis systems 100 may be configured in a hierarchical architecture, each level of video analysis system may have different sizes and permissions, and each level of video analysis system may have the same structure. For example, the first layer video analysis system may have a size covering only one street, the second layer video analysis system may have a size covering one administrative area, the third layer video analysis system may have a size covering one city, and the second layer The video analysis system can read the data of the first layer video analysis system and can control the first layer video analysis system, and the third layer video analysis system can read the data of the first layer video analysis system and the second layer video analysis system and The first layer video analysis system and the second layer video analysis system are controlled, but the first layer video analysis system cannot read the data of the second layer video analysis system and the third layer video analysis system, and cannot be controlled.

The video analysis system of the embodiment can analyze the video surveillance image and transmit the analysis result, thereby improving the automation degree and efficiency of the video surveillance and reducing the labor cost; moreover, the video analysis system of the embodiment has a comparison Strong scalability.

Example 2

Embodiment 2 of the present application provides a video analysis method corresponding to the video analysis system of Embodiment 1.

FIG. 4 is a schematic diagram of a video analysis method according to this embodiment. As shown in FIG. 4, the method includes:

Step 401: Perform shooting to obtain a video surveillance image.

Step 402: Perform analysis on the video surveillance image to obtain an analysis result;

Step 403: Send the video surveillance image and the analysis result.

FIG. 5 is a schematic diagram of analyzing the video surveillance image according to the embodiment. As shown in FIG. 5, the analysis includes:

Step 501: Detecting a foreground of the video surveillance image;

Step 502: Extract features of the foreground;

Step 503: Set a determination condition, where the determination condition corresponds to a scenario applied by the video monitoring method;

Step 504: Perform comparison according to the extracted feature and the determination condition to obtain the analysis result.

Wherein, in step 502, the extracted features include: position, motion speed, motion direction, trajectory, size, texture, color, and color gradient of the object in the foreground.

In this embodiment, the extracted features are combined to form a feature combination; in addition, the feature types and quantities constituting the feature combination may also be set according to a scenario to which the video monitoring method is applied.

In step 503, the determining condition is set according to a configuration file corresponding to the scenario, where the scenario includes traffic monitoring, security monitoring, forest monitoring, agricultural monitoring, or factory monitoring.

For the description of the steps in the embodiment, reference may be made to the description of the components of the video analysis system in Embodiment 1, and the description thereof will not be repeated here.

According to the video monitoring-based video analysis method of the embodiment, the video surveillance image can be analyzed and the analysis result is transmitted, thereby improving the automation degree and efficiency of the video surveillance and reducing the labor cost; The video analysis method has strong scalability.

The embodiment of the present application further provides a computer readable program, wherein the program causes the video analysis system to perform the video analysis method described in Embodiment 2 when the program is executed in a video analysis system.

The embodiment of the present application further provides a storage medium storing a computer readable program, wherein the storage medium The above computer readable program is stored, and the computer readable program causes the video analysis system to perform the video analysis method described in Embodiment 2.

The video analysis system described in connection with the embodiments of the present invention may be directly embodied as hardware, a software module executed by a processor, or a combination of both. For example, one or more of the functional block diagrams shown in Figures 1, 2 and/or one or more combinations of functional block diagrams may correspond to various software modules of a computer program flow, or to individual hardware modules. These software modules may correspond to the respective steps shown in FIGS. 4 and 5, respectively. These hardware modules can be implemented, for example, by curing these software modules using a Field Programmable Gate Array (FPGA).

The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor to enable the processor to read information from, and write information to, the storage medium; or the storage medium can be an integral part of the processor. The processor and the storage medium can be located in an ASIC. The software module can be stored in the memory of the mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if a device (such as a mobile terminal) uses a larger capacity MEGA-SIM card or a large-capacity flash memory device, the software module can be stored in the MEGA-SIM card or a large-capacity flash memory device.

One or more of the functional block diagrams described with respect to Figures 1, 2 and/or one or more combinations of functional block diagrams may be implemented as a general purpose processor, digital signal processor (DSP) for performing the functions described herein. An application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof. One or more of the functional blocks described with respect to Figures 1-3 and/or one or more combinations of functional blocks may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors One or more microprocessors in conjunction with DSP communication or any other such configuration.

The present invention has been described in connection with the specific embodiments thereof, but it is to be understood that the description is intended to be illustrative and not restrictive. Various modifications and alterations of this application will be apparent to those skilled in the art in the light of the invention.

Claims (12)

A video analysis system based on video surveillance, comprising: a video monitoring subsystem for taking a picture to obtain a video surveillance image; a video analysis subsystem in communication with the video surveillance subsystem for analyzing the video surveillance image obtained by the video surveillance subsystem; An information distribution subsystem, configured to communicate with the video analysis subsystem, for transmitting the video surveillance image obtained by the video monitoring subsystem and an analysis result of the video analysis subsystem, The video analysis subsystem includes: a motion analysis unit for detecting a foreground of a video surveillance image; a feature extraction unit for extracting features of the foreground; a judgment condition setting unit configured to set a judgment condition corresponding to a scene applied by the video analysis system; a determination unit that compares the feature extracted by the feature extraction unit with the determination condition set by the determination condition setting unit to obtain the analysis result. The video analysis system according to claim 1, wherein The feature extracted by the feature extraction unit includes: a position, a motion speed, a motion direction, a motion trajectory, a size, a texture, a color, and a color gradient of the object in the foreground; The feature extraction unit also forms a feature combination based on the combination of the extracted features. The video analysis system according to claim 1, wherein The determination condition setting unit sets the determination condition according to a configuration file corresponding to the scene, wherein the scene includes traffic monitoring, security monitoring, forest monitoring, agricultural monitoring, or factory monitoring. The video analysis system according to claim 1, wherein The analysis of the video surveillance image by the video analysis subsystem includes event detection, data statistics, and/or object retrieval. The video analytics system of claim 1 wherein said information dissemination subsystem transmits events and/or statistical data detected by said video analytics subsystem. The video analytics system of claim 1 wherein said video analytics subsystem comprises: a backend analysis device for performing event detection, data statistics, and/or objects on the video surveillance image Search. The video analytics system of claim 6 wherein said video analytics subsystem further comprises: a front end processing unit embedded in the camera device of the video monitoring subsystem for performing event detection and/or data statistics on the video surveillance image; and/or The front end analysis device is disposed in an outdoor environment for performing event detection and/or data statistics on the video surveillance image. The video analysis system according to claim 7, wherein said backend analysis device is configured to acquire an analysis result of said front end processing section and/or said front end analysis device. The video analytics system of claim 7 wherein said backend analytics device is further operative to implement at least one of the following functions: Monitoring the working state of the video analysis subsystem; Initializing the analysis of the video analysis subsystem; Managing user information of the video monitoring subsystem; The analysis results of the video analysis subsystem are managed. A video analysis method based on video surveillance, comprising: Shooting to obtain a video surveillance image; Performing analysis on the video surveillance image to obtain an analysis result; Transmitting the video surveillance image and the analysis result, The analyzing the video surveillance image to obtain the analysis result includes: Detecting the foreground of the video surveillance image; Extracting features of the foreground; Setting a judgment condition corresponding to a scenario applied by the video monitoring method; The comparison is performed according to the extracted feature and the judgment condition to obtain the analysis result. The video analysis method according to claim 10, wherein The extracted features include: position, motion speed, motion direction, motion trajectory, size, texture, color, and color gradient of the object in the foreground; And, the extracted features are combined to form a feature combination. The video analysis method according to claim 10, wherein the setting the judgment condition comprises: The determining condition is set according to a configuration file corresponding to the scenario, where the scenario includes traffic monitoring, security monitoring, forest monitoring, agricultural monitoring, or factory monitoring.

Images