CN119559701A

CN119559701A - A method and system for identifying abnormal behavior in financial escort process

Info

Publication number: CN119559701A
Application number: CN202411782328.3A
Authority: CN
Inventors: 刘勇; 赵国辉
Original assignee: Ningbo Research Institute of Dalian University of Technology
Current assignee: Ningbo Research Institute of Dalian University of Technology
Priority date: 2024-12-05
Filing date: 2024-12-05
Publication date: 2025-03-04

Abstract

The present invention discloses a method and system for identifying abnormal behavior in a financial escort process, and relates to the field of data processing. The method comprises: collecting personnel feature data, establishing a multimodal biometric database; using a convolutional neural network to match the features extracted from the real-time personnel feature data with the features stored in the multimodal biometric database, and obtaining a matching result; when the matching result is a successful match, continuing to obtain the video stream of the monitoring area in real time and preprocessing; using a deep learning algorithm to perform target detection on the preprocessed video stream, and identifying the tracking target; using a target tracking algorithm to track the detected tracking target for continuous frames, and obtaining the motion trajectory of the tracking target; using a posture estimation algorithm to extract the posture information of the target, and combining the motion trajectory and spatiotemporal characteristics of the target to construct a multidimensional behavior feature vector; establishing a preset abnormal behavior rule library, and the abnormal behavior rule library contains a known abnormal behavior set; based on the output of the multidimensional behavior feature vector, combined with the preset abnormal behavior rule library, the identified behavior is analyzed in real time to identify whether the target is performing abnormal behavior. It solves the technical problem that the existing technology is based on a single personnel identification technology, which is unable to simultaneously identify the risks of the escort process and is difficult to meet the requirements of timely discovering escort risks. By verifying the multimodal biometric characteristics of personnel and tracking the motion trajectory of the target, the real-time motion trajectory is compared and analyzed with the preset trajectory, and abnormal behavior is identified, thereby improving the efficiency of real-time monitoring and reducing the technical effect of escort risks.

Description

Abnormal behavior identification method and system in financial escort process

Technical Field

The application relates to the technical field of data processing, in particular to a method and a system for identifying abnormal behaviors in a financial escort process.

Background

With the rapid development of financial transactions, financial escort work is becoming increasingly important and complex. The traditional authentication mode such as passwords, certificates and the like have potential safety hazards such as easy leakage, easy counterfeiting and the like, and the safety requirements of modern financial escort are difficult to meet.

The existing face recognition technology or iris recognition technology is adopted, however, the single person recognition technology improves the convenience and accuracy of identity verification to a certain extent, but the risk that the escort process can not be recognized at the same time after the person recognition still exists, and the financial institution can not grasp the escort dynamics in real time, so that the problems of increased safety risk, unsmooth escort flow and the like are caused.

In summary, in the prior art, because of a single personnel identification technology, risks in the escort process cannot be identified simultaneously, and technical problems of timely finding escort risk requirements are difficult to be satisfied.

Disclosure of Invention

Based on the above technical problems, it is necessary to provide a method and a system for identifying abnormal behaviors in a financial escort process, which can solve the technical problems that in the prior art, a single personnel identification technology cannot identify risks in the escort process simultaneously, and the requirements of escort risks are difficult to be found in time, and by verifying multi-modal biological characteristics of personnel and tracking motion trajectories of targets, real-time motion trajectories and preset trajectory comparison analysis are realized, and abnormal behaviors are identified, so that the efficiency of real-time monitoring is improved, and the technical effects of escort risks are reduced.

The method comprises the steps of collecting personnel characteristic data, wherein the characteristic data comprise fingerprint data, facial data and iris data, conducting characteristic extraction on the personnel characteristic data, establishing a multi-mode biological characteristic database, obtaining real-time personnel characteristic data, matching the characteristics extracted by the real-time personnel characteristic data with the characteristics stored in the multi-mode biological characteristic database through a convolutional neural network to obtain a matching result, continuously obtaining a video stream of a monitoring area in real time and conducting preprocessing when the matching result is successful, conducting target detection on the preprocessed video stream through a deep learning algorithm, identifying a tracking target, wherein the tracking target comprises personnel and vehicles, conducting continuous frame tracking on the detected tracking target through a target tracking algorithm, obtaining the motion track of the tracking target, utilizing a gesture estimation algorithm to extract gesture information of the target, combining the motion track and space-time characteristics of the target, establishing a preset abnormal behavior rule base, wherein the abnormal behavior rule base comprises a known abnormal behavior set, conducting target-state analysis on whether the abnormal behavior is being identified through the preset behavior rule based on the multi-dimensional behavior characteristic vector.

In a second aspect, a system for identifying abnormal behavior in a financial escort process is provided, the system comprises a data acquisition module, a feature extraction module, a feature matching module, a video stream acquisition module, a target recognition module, a motion track acquisition module and a multi-dimensional behavior module, wherein the data acquisition module is used for acquiring personnel feature data, the feature data comprises fingerprint data, facial data and iris data, the feature extraction module is used for carrying out feature extraction on the personnel feature data to establish a multi-mode biological feature database, the feature matching module is used for acquiring real-time personnel feature data, a convolutional neural network is used for matching the features extracted from the real-time personnel feature data with the features stored in the multi-mode biological feature database to obtain a matching result, the video stream acquisition module is used for continuously acquiring and preprocessing video streams of a monitoring area in real time when the matching result is successful, the target recognition module is used for carrying out target detection on the preprocessed video streams by using a deep learning algorithm, the tracking target comprises personnel and vehicles, the motion track acquisition module is used for carrying out continuous frame tracking on the detected tracking targets by using a target tracking algorithm, the motion track acquisition module is used for carrying out continuous frame tracking tracks on the detected tracking targets by using the target tracking algorithms, the motion track acquisition module is used for carrying out the multi-dimensional behavior feature algorithm, the motion track acquisition module is used for establishing a multi-dimensional behavior feature database is used for establishing an abnormal behavior feature database by using a motion vector, and an abnormal behavior feature database is used for establishing an abnormal behavior feature database, and has an abnormal behavior characteristic vector, and is used for setting an abnormal behavior graph is used for setting established, the abnormal behavior recognition module is used for analyzing the recognized behaviors in real time based on the output of the multidimensional behavior feature vector and combining a preset abnormal behavior rule base to recognize whether the target is in abnormal behavior.

The abnormal behavior identification method and the abnormal behavior identification system in the financial escort process solve the technical problems that in the prior art, in a single personnel identification technology, risks in the escort process cannot be identified simultaneously, and the escort risk requirement is difficult to find timely.

The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent.

Drawings

FIG. 1 is a flowchart of a method for identifying abnormal behavior in a financial escort process according to one embodiment;

FIG. 2 is a schematic flow chart of a method for identifying abnormal behaviors in a financial escort process by extracting the gesture information of a target by using a gesture estimation algorithm and combining the motion trail and space-time characteristics of the target to construct a multidimensional behavior feature vector;

FIG. 3 is a block diagram of a system for identifying abnormal behavior of a financial escort process in one embodiment.

The reference numerals illustrate a data acquisition module 11, a feature extraction module 12, a feature matching module 13, a video stream acquisition module 14, a target identification module 15, a motion trail acquisition module 16, a multi-dimensional behavior feature vector construction module 17, an abnormal behavior rule base construction module 18 and an abnormal behavior identification module 19.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

As shown in fig. 1, the present application provides a method for identifying abnormal behavior in a financial escort process, which includes:

collecting personnel characteristic data, wherein the characteristic data comprises fingerprint data, facial data and iris data;

Fingerprint data, facial data and iris data are all unique biological identification characteristics, and have high uniqueness and stability. In the financial escort process, fingerprint data, face data and iris data of escort personnel are collected for identifying and verifying personnel identity. Among other things, iris recognition techniques are used to provide a higher level of authentication for enhanced security measures to prevent unauthorized persons from accessing escort items, since iris features are difficult to forge or imitate.

Collecting personnel characteristic data, wherein the characteristic data comprises fingerprint data, facial data and iris data, and the method comprises the following steps of:

acquiring fingerprints by using professional fingerprint acquisition equipment to acquire fingerprint data;

acquiring face data of a person by using a high-resolution camera;

And (3) performing multiple scans by using an iris acquisition device to acquire iris data under different angles and illumination conditions.

The optical fingerprint acquisition equipment is used for capturing fine textures and characteristic points of fingerprints, so that the acquired fingerprint data is ensured to have high resolution and definition. When a finger is pressed on the acquisition surface in the process of acquiring fingerprint data, the reflection of lines and valleys on the surface of the finger is different, and the device captures the differences of the reflected light through the sensor to form a fingerprint image. When collecting facial data, a high-resolution camera is used for capturing facial details of a person and generating facial data, including the shape and the outline of eyes, a nose, a mouth and other parts, skin texture and other characteristics. And capturing the facial expression and the motion of the escort personnel in real time, so as to identify potential abnormal behaviors such as tension, anxiety and the like. In order to adapt to different illumination environments, the camera is provided with automatic focusing and automatic exposure functions. Under the environment with darker light, the camera can automatically increase the exposure degree, and under the environment with too strong light, the exposure degree can also be automatically reduced, so that the collected facial image has proper contrast and brightness; the iris acquisition equipment is high-precision biological identification equipment, adopts image processing and optical technology, accurately captures fine textures and characteristic points of the iris, and realizes high-precision iris identification. The iris acquisition equipment is internally provided with a special optical system, so that the incidence angle and intensity of light can be adjusted within a certain range, the light is focused on an iris area, and the texture information of the iris is accurately captured.

Extracting the characteristics of the personnel characteristic data, and establishing a multi-mode biological characteristic database;

First, feature extraction is performed from the person feature data obtained from the acquisition device, including fingerprint features, facial features, and iris features. The fingerprint feature extraction determines the pattern of the fingerprint image by analyzing the trend and the mode of the integral pattern of the fingerprint image, and comprises a bucket type, a skip type and a bow type. The fingerprint has circular or spiral fingerprint lines, dustpan-like fingerprint, and bow-shaped fingerprint lines are smooth and arc-shaped. The fingerprint image contains a number of unique minutiae such as endpoints, bifurcation points, etc. The fingerprint lines are thinned into lines with single pixel width by using an image thinning method, and the coordinate position (x, y) and the direction (expressed by angles) and the type (end points or bifurcation points) of each minutiae point are marked. Facial feature extraction locates facial key points such as corners of the eyes, nose, corners of the mouth, etc. through an active appearance model. And learning the distribution pattern of the facial key points on the training set through a facial feature point detection algorithm, and searching the matched key point positions on the input facial image. Then, the positions of the key points of the face of the person are determined, and the relative distance, angle and other relations between the key points are calculated. Including calculating the distance between the eyes, the distance from the tip of the nose to the corner of the mouth, etc. Finally, texture information of the facial skin is extracted. The iris feature extraction is to locate the collected iris image to determine the inner and outer boundaries of the iris, normalize the iris region to a fixed size, and filter the normalized iris region.

Extracting the characteristics of the personnel characteristic data, and establishing a multi-mode biological characteristic database comprises the following steps:

Extracting characteristic information such as texture, direction and frequency of the fingerprint from the collected fingerprint data by using a fingerprint characteristic extraction algorithm;

Extracting feature information such as key points, textures, shapes and the like of the face from the acquired face data by using a face feature extraction algorithm, wherein the key points of the face comprise eye corners, mouth corners and the like;

extracting characteristic information such as texture, spots, lines and the like of the iris from the acquired iris data by using an iris characteristic extraction algorithm;

And integrating the extracted fingerprint features, facial features and iris features to form multi-modal biological feature data of each person, and establishing a multi-modal biological feature database.

Feature extraction is performed by Convolutional Neural Network (CNN). Firstly, carrying out convolution operation to output a feature map, then calculating the size of the feature map generated after the convolution operation, then carrying out pooling operation on the feature map, and finally carrying out full-connection layer operation.

Convolution operation expression:

Wherein Y is an output feature map;

w is a convolution kernel;

X is an input image;

valid is a convolution type (no zero padding);

b is a bias term;

conv2 is a convolution operation.

The calculation formula of the size of the feature map generated after the convolution operation is as follows:

Where w is the size of the input image;

p is the number of layers filled;

k is the size of the convolution kernel;

s is stride;

constraint that division operations round down.

The max pooling operation expression is:

S is an output feature map;

max_ pooling represents the max pooling operation;

X is an input feature map;

k is a window size value.

The output Y expression of the full connection layer is:

Wherein W represents the weight of the full connection layer;

A represents the output of the previous layer;

b represents the bias of the fully connected layer.

Acquiring real-time personnel feature data, and matching the extracted features of the real-time personnel feature data with the features stored in the multi-mode biological feature database by using a convolutional neural network to acquire a matching result;

Firstly, performing feature matching by using a convolutional neural network, secondly, defining and training a convolutional neural network by adopting a cosine similarity verification and matching process to extract features from input data, and calculating a similarity matrix between samples by using the features. In the similarity measurement stage, similarity between the feature vectors is calculated by utilizing measurement methods such as cosine similarity and the like, so that verification and matching tasks are completed. And finally, integrating the verification results of the multiple modes to obtain a final verification conclusion. Based on the verification result, feedback (e.g., "verification success" or "verification failure") is provided to the user.

The output characteristic diagram expression is:

wherein X is an m X n input image matrix;

W is a convolution kernel matrix of k x k (k < m, k < n);

y is one Is provided.

When the matching result is that the matching is successful, continuously acquiring the video stream of the monitoring area in real time and preprocessing the video stream;

And after the matching is successful, continuously acquiring the video stream of the monitoring area in real time, and preprocessing the acquired video frames for subsequent analysis in order to improve the visibility of the video image. Firstly, the video stream image is subjected to a resizing operation, and various noises may be contained in the video stream due to the complexity of a monitoring environment, so that a Gaussian filtering denoising process is performed. Contrast enhancement processing is carried out on license plates of escort vehicles and faces of escort personnel in the financial escort process.

Performing target detection on the preprocessed video stream by using a deep learning algorithm, and identifying a tracking target, wherein the tracking target comprises personnel and vehicles;

and distinguishing escort personnel and other personnel according to the color and style characteristics of the uniform of escort personnel. By analyzing multi-frame images in the video stream, whether the person walks, runs normally or makes abnormal actions (such as suddenly squatting, waving an arm, etc.) is identified. Vehicles have significant shape features such as rectangular contours of the body, rounded wheels, etc. Meanwhile, the color and license plate information of the vehicle are also important identification features. The driving state of the vehicle, such as speed, driving direction, etc., can also be obtained by analyzing the position change of the vehicle in successive frames in the video stream. Thereby monitoring whether the escort vehicle runs along a preset route, whether abnormal acceleration or deceleration exists or not, and the like.

Target detection is carried out on the preprocessed video stream by using a deep learning algorithm, and a tracking target is identified, wherein the tracking target comprises personnel and vehicles and comprises the following components:

Extracting images from the video stream frame by frame for frame analysis;

performing enhancement processing on the extracted image and adjusting the image to a size suitable for the input of the deep learning model;

carrying out normalization processing on the image to enable the pixel value to be in a specific range;

performing feature extraction and classification by using a convolutional neural network;

Training the selected deep learning model by using the marked data set;

Applying a trained model to each frame in the video stream to perform target detection, and identifying the positions and the categories of personnel and vehicles;

a tracking algorithm is applied between successive frames to track the motion trajectory of the object in the video.

The method comprises the steps of extracting images from a video stream frame by frame, carrying out contrast enhancement and noise reduction on the extracted images, adjusting the images to a size suitable for deep learning model input, carrying out normalization on the images, normalizing pixel values to a specific range, carrying out characteristic extraction and classification by using a convolutional neural network, carrying out convolution operation firstly, carrying out pooling operation secondly, and carrying out full-connection layer operation. Training the selected deep learning model by using the marked data set, applying the trained model to each frame in the video stream to perform target detection, sliding a convolution kernel on the image, extracting features and judging whether personnel or vehicle targets exist according to the output of the full connection layer. If an object is present, the model determines the location and class of the person and vehicle based on the feature map and associated algorithms. And finally, a tracking algorithm is applied between the continuous frames to track the motion trail of the target in the video.

Tracking the detected tracking target in continuous frames by utilizing a target tracking algorithm to acquire a motion trail of the tracking target;

The regions that may contain the target are generated using an algorithm and then classified and regressed to determine the exact location and class of the target. Then, the optical flow method is used for assuming that the intensity of pixels in the image is kept unchanged in a short time, and the gray level change of the pixels between adjacent frames is analyzed to determine the movement direction and speed of the pixels so as to realize target tracking. In the target tracking process, the characteristics of the target are continuously extracted from the current frame along with the movement of the target in the video. The sliding window method is adopted when searching the most similar area in the subsequent frame. And sliding a window similar to the initial size of the target in the subsequent frames, and calculating the similarity between the region in the window and the target characteristics. When the most similar region is found (determined based on the set similarity threshold), the location of the object in the frame is considered to be found. Once the most similar area is found in the subsequent frames, the position information of the target is updated in time, and the target is accurately identified.

Tracking the detected tracking target in continuous frames by using a target tracking algorithm to acquire a motion trail of the tracking target, wherein the method comprises the following steps:

performing target detection on each frame of the video stream by using a target detection algorithm;

Initializing the detected target, wherein the initialization comprises the initial position, the size and possible appearance characteristics of the target;

Calculating the motion of pixels in the image by using an optical flow method, so as to track the motion trail of the target;

extracting features of the target from the current frame;

Searching a region most similar to the target in a subsequent frame according to the characteristics of the extracted target;

When the most similar area is found, updating the position information of the target;

and estimating the motion state of the target according to the position information of the target in the continuous frames, and obtaining the motion trail of the target.

Extracting the gesture information of the target by using a gesture estimation algorithm, and constructing a multidimensional behavior feature vector by combining the motion trail and space-time features of the target;

pose in the video frames of escort personnel and vehicles, such as the joint positions of human bodies, the angles of limbs and the like, are determined by using a pose estimation algorithm. The position coordinate points at different moments are used for representing the movement tracks of escort personnel and vehicles, and for the time, duration, time rhythm and other time characteristics of the movement of the escort personnel and the vehicles in the video, the spatial characteristics comprise whether the vehicles are in a lane, an intersection, a parking lot and other different spatial areas. And combining the gesture information, the motion trail features and the space-time features into a multidimensional behavior feature vector.

Extracting the gesture information of the target by using a gesture estimation algorithm, and constructing a multidimensional behavior feature vector by combining the motion trail and the space-time feature of the target, wherein the method comprises the following steps:

carrying out gesture estimation on a target in the video stream by using a gesture estimation algorithm to obtain key points of the target and position information of the key points;

extracting gesture information of a target by utilizing the output of a gesture estimation algorithm, wherein the gesture information comprises coordinates of key points, relative position relations among the key points, motion tracks of the key points and the like;

Acquiring a motion trail of a target by using a target tracking algorithm, wherein the motion trail is a position information sequence of the target in continuous frames;

extracting space-time characteristics of a target, wherein the space-time characteristics refer to change information of the target in time and space, and the change information comprises acceleration, speed change, gesture change and the like of the target;

And combining the extracted gesture information, motion trail and space-time characteristics to construct a multidimensional behavior feature vector.

Establishing a preset abnormal behavior rule base, wherein the abnormal behavior rule base comprises a known abnormal behavior set;

And establishing an abnormal behavior rule base, wherein the abnormal behavior rule base aims at identifying abnormal behaviors of financial escort personnel in the task execution process. By defining the rules of the behaviors, the abnormal behaviors are automatically detected by using a monitoring technology, so that the security of the field finance escort is ensured.

Establishing a preset abnormal behavior rule base, wherein the abnormal behavior rule base comprises a known abnormal behavior set and comprises the following steps:

collecting sample data containing various abnormal behaviors, and establishing a preset abnormal behavior rule base;

extracting behavior characteristic information capable of describing an abnormal behavior sample, wherein the characteristic information comprises the gesture, the motion track, the speed, the acceleration, the interaction with other targets and the like of the targets;

and constructing an abnormal behavior rule base based on the extracted characteristic information.

The method comprises the steps of establishing a preset abnormal behavior rule base, and collecting abnormal behavior sample data of escort personnel and vehicles, wherein the abnormal behaviors of the escort personnel comprise abnormal behaviors such as abnormal sitting postures, dozing, calling, eating, smoking, connecting with the ears and the like, and the abnormal behaviors of the vehicles comprise abnormal behaviors such as non-running according to a set track, overspeed or slow speed of the vehicles, collision accidents of target vehicles and other vehicles or people, abnormal vehicle components and the like. And constructing an abnormal behavior rule base, wherein behavior characteristic information in an abnormal behavior sample is extracted from the abnormal behavior rule base data, and the behavior characteristic information comprises the gesture of escort personnel, the motion track of escort vehicles, the vehicle speed, the acceleration and the interaction with other targets.

Based on the output of the multidimensional behavior feature vector, the identified behaviors are analyzed in real time by combining a preset abnormal behavior rule base, and whether the target is in abnormal behavior is identified.

Firstly, comparing and analyzing behaviors in the escort personnel and vehicle video frames output by the multidimensional behavior feature vectors with a preset abnormal behavior rule base, and identifying whether the escort personnel and the vehicle are performing abnormal behaviors.

Based on the output of the multidimensional behavior feature vector, the identified behavior is analyzed in real time by combining with a preset abnormal behavior rule base, and whether the target is in abnormal behavior is identified, which comprises the following steps:

Extracting multidimensional behavior feature vectors of targets from video streams in real time by using a gesture estimation algorithm, a target tracking algorithm and a space-time feature extraction method;

matching the extracted multidimensional behavior feature vector with a preset abnormal behavior rule base;

Judging whether the target is performing abnormal behavior in real time according to the matching result of the feature vector and the rule base;

when abnormal behavior is identified, the system immediately triggers an alarm mechanism.

And identifying the positions of the joints of the human body and the movement track information of the targets by using an attitude estimation algorithm, a target tracking algorithm and a space-time feature extraction method, wherein the movement track information comprises coordinates of joints such as the head, the shoulder, the elbow, the wrist, the hip, the knee, the ankle and the like of escort personnel and the movement track of escort vehicles. The posture of the human body, such as standing, bending down, lifting hands, moving track of the vehicle and the like, is described through the position relation of the joint points of the personnel, and the characteristics of the moving direction, the speed change and the like of the target are further analyzed. If the motion direction vector of the target is obtained by calculating the vector between adjacent points on the track, the speed information of the target is obtained by calculating the change rate of the distance between the track points along with time, and the multidimensional behavior feature vector of the target is extracted. After the multidimensional behavior feature vector of the target is extracted, the multidimensional behavior feature vector is matched with rules in an abnormal behavior rule base one by one. If the abnormal behavior is matched, the system gives an alarm, and sends alarm information to a remote monitoring center through a network.

As shown in fig. 3, an embodiment of the present application includes a system for identifying abnormal behavior of a financial escort process, the system including:

The data acquisition module 11 is used for acquiring personnel characteristic data, wherein the characteristic data comprises fingerprint data, facial data and iris data;

the feature extraction module 12 is used for carrying out feature extraction on the personnel feature data and establishing a multi-mode biological feature database;

The feature matching module 13 is used for acquiring real-time personnel feature data, and matching the features extracted from the real-time personnel feature data with the features stored in the multi-mode biological feature database by using a convolutional neural network to acquire a matching result;

The video stream obtaining module 14 is configured to continuously obtain the video stream of the monitoring area in real time and perform preprocessing when the matching result is that the matching is successful;

The target recognition module 15 is used for performing target detection on the preprocessed video stream by using a deep learning algorithm and recognizing a tracking target, wherein the tracking target comprises personnel and vehicles;

A motion track acquisition module 16, configured to track the detected tracking target in successive frames by using a target tracking algorithm, and acquire a motion track of the tracking target;

The multidimensional behavior feature vector construction module 17 is used for extracting the gesture information of the target by using a gesture estimation algorithm and constructing a multidimensional behavior feature vector by combining the motion track and the space-time feature of the target;

an abnormal behavior rule base establishing module 18, configured to establish a preset abnormal behavior rule base, where the abnormal behavior rule base includes a known abnormal behavior set;

The abnormal behavior recognition module 19 is configured to analyze the recognized behavior in real time based on the output of the multidimensional behavior feature vector in combination with a preset abnormal behavior rule base, and recognize whether the target is performing an abnormal behavior.

Further, the data acquisition module 11 further includes:

acquiring face data of a person by using a high-resolution camera;

Further, the feature extraction module 12 further includes:

Further, the object recognition module 15 further includes:

Extracting images from the video stream frame by frame for frame analysis;

Training the selected deep learning model by using the marked data set;

Further, the motion trajectory acquisition module 16 further includes:

extracting features of the target from the current frame;

Further, the multidimensional behavior feature vector constructing module 17 further includes:

Further, the abnormal behavior rule base building module 18 further includes:

Further, the abnormal behavior recognition module 19 further includes:

For specific embodiments of the abnormal behavior recognition system in the financial escort process, reference may be made to the above embodiments of the abnormal behavior recognition method in the financial escort process, which are not described herein. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.

Claims

1. A method for identifying abnormal behavior in a financial escort process, characterized in that the method comprises:

Collecting personnel characteristic data, including fingerprint data, facial data, and iris data;

Extracting features from the personnel feature data and establishing a multimodal biometric database;

Acquire real-time personnel feature data, and use a convolutional neural network to match features extracted from the real-time personnel feature data with features stored in a multimodal biometric database to obtain a matching result;

When the matching result is successful, continue to obtain the video stream of the monitored area in real time and perform preprocessing;

Using a deep learning algorithm to perform target detection on the preprocessed video stream and identify tracking targets, which include people and vehicles;

The target tracking algorithm is used to track the detected tracking target in continuous frames to obtain the motion trajectory of the tracking target;

The posture estimation algorithm is used to extract the posture information of the target, and the multi-dimensional behavior feature vector is constructed by combining the target's motion trajectory and spatiotemporal characteristics.

Establishing a preset abnormal behavior rule library, wherein the abnormal behavior rule library includes a known abnormal behavior set;

Based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library, the identified behavior is analyzed in real time to identify whether the target is performing abnormal behavior.

2. The method according to claim 1, characterized in that the personnel characteristic data is collected, and the characteristic data includes fingerprint data, facial data, and iris data, including:

Use professional fingerprint collection equipment to collect fingerprints and obtain fingerprint data;

Use a high-resolution camera to capture a person's face to obtain facial data;

Use the iris acquisition device to perform multiple scans to obtain iris data under different angles and lighting conditions.

3. The method according to claim 1, wherein extracting features from the personnel feature data and establishing a multimodal biometric feature database comprises:

Use fingerprint feature extraction algorithm to extract fingerprint texture, direction, frequency and other feature information from the collected fingerprint data;

Using a facial feature extraction algorithm, feature information such as facial key points, texture and shape are extracted from the collected facial data, wherein the facial key points include the corners of the eyes and the corners of the mouth;

Use iris feature extraction algorithm to extract iris texture, spots, lines and other feature information from the collected iris data;

The extracted fingerprint features, facial features and iris features are integrated to form each person's multimodal biometric data and establish a multimodal biometric database.

4. The method according to claim 1, characterized in that the deep learning algorithm is used to perform target detection on the preprocessed video stream to identify the tracking target, wherein the tracking target includes a person and a vehicle, including:

Extract images frame by frame from the video stream for frame-by-frame analysis;

Enhance the extracted images and resize them to a size suitable for deep learning model input;

Normalize the image so that the pixel values are within a specific range;

Use convolutional neural networks for feature extraction and classification;

Use the labeled dataset to train the selected deep learning model;

Apply the trained model to each frame in the video stream for object detection, identifying the location and category of people and vehicles;

A tracking algorithm is applied between consecutive frames to track the movement trajectory of the target in the video.

5. The method according to claim 1, characterized in that the detected tracking target is tracked in consecutive frames using a target tracking algorithm to obtain a motion trajectory of the tracking target, comprising:

Use the target detection algorithm to detect the target in each frame of the video stream;

Initializing the detected target, wherein the initialization includes the initial position, size, and possible appearance features of the target;

Use the optical flow method to calculate the movement of pixels in the image, thereby tracking the movement trajectory of the target;

Extract the features of the target from the current frame;

According to the features of the extracted target, searching for the area most similar to the target in subsequent frames;

When the most similar area is found, the location information of the target is updated;

According to the position information of the target in the continuous frames, the motion state of the target is estimated and the motion trajectory of the target is obtained.

6. The method according to claim 1, characterized in that the posture information of the target is extracted by using a posture estimation algorithm, and a multi-dimensional behavior feature vector is constructed by combining the motion trajectory and spatiotemporal characteristics of the target, comprising:

Use the posture estimation algorithm to estimate the posture of the target in the video stream and obtain the key points of the target and the position information of the key points;

The output of the posture estimation algorithm is used to extract the posture information of the target, wherein the posture information includes the coordinates of key points, the relative position relationship between key points, and the motion trajectory of key points;

Using a target tracking algorithm to obtain a motion trajectory of the target, the motion trajectory is a sequence of position information of the target in consecutive frames;

Extracting the spatiotemporal features of the target, wherein the spatiotemporal features refer to the change information of the target in time and space, and the change information includes the acceleration, speed change, posture change, etc. of the target;

The extracted posture information, motion trajectory and spatiotemporal features are combined to construct a multi-dimensional behavior feature vector.

7. The method according to claim 1, characterized in that a preset abnormal behavior rule library is established, wherein the abnormal behavior rule library contains a known abnormal behavior set, including:

Collect sample data containing various abnormal behaviors and establish a preset abnormal behavior rule library;

Extracting behavioral feature information that can describe abnormal behavior samples, the feature information includes the target's posture, motion trajectory, speed, acceleration, interaction with other targets, etc.;

Based on the extracted feature information, an abnormal behavior rule library is constructed.

8. The method according to claim 1, characterized in that, based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library, the identified behavior is analyzed in real time to identify whether the target is performing abnormal behavior, including:

Using posture estimation algorithm, target tracking algorithm and spatiotemporal feature extraction method, the multi-dimensional behavior feature vector of the target is extracted from the video stream in real time;

Matching the extracted multi-dimensional behavior feature vector with the preset abnormal behavior rule library;

Based on the matching results between the feature vector and the rule base, determine in real time whether the target is performing abnormal behavior;

When abnormal behavior is identified, the system immediately triggers the alarm mechanism.

9. A system for identifying abnormal behavior during financial escort, characterized in that the system comprises:

A data collection module, wherein the data collection module is used to collect personnel characteristic data, wherein the characteristic data includes fingerprint data, facial data, and iris data;

A feature extraction module, the feature extraction module is used to extract features from the personnel feature data and establish a multimodal biometric database;

A feature matching module, the feature matching module is used to obtain real-time personnel feature data, and use a convolutional neural network to match features extracted from the real-time personnel feature data with features stored in a multimodal biometric database to obtain a matching result;

A video stream acquisition module, wherein when the matching result is a successful match, the video stream acquisition module continues to acquire the video stream of the monitoring area in real time and performs preprocessing;

A target recognition module, which is used to perform target detection on the preprocessed video stream using a deep learning algorithm to identify tracking targets, including people and vehicles;

A motion trajectory acquisition module, wherein the motion trajectory acquisition module is used to track the detected tracking target in consecutive frames using a target tracking algorithm to acquire the motion trajectory of the tracking target;

A multi-dimensional behavior feature vector construction module, wherein the multi-dimensional behavior feature vector construction module is used to extract the target's posture information by using a posture estimation algorithm, and to construct a multi-dimensional behavior feature vector by combining the target's motion trajectory and spatiotemporal characteristics;

An abnormal behavior rule base establishment module, the abnormal behavior rule base establishment module is used to establish a preset abnormal behavior rule base, the abnormal behavior rule base contains a known abnormal behavior set;

The abnormal behavior recognition module is used to analyze the recognized behavior in real time based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library to identify whether the target is performing abnormal behavior.