WO2020252928A1

WO2020252928A1 - Method and apparatus for tracking human face in video, and computer device and storage medium

Info

Publication number: WO2020252928A1
Application number: PCT/CN2019/103541
Authority: WO
Inventors: 张磊; 宋晨; 李雪冰
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2019-06-20
Filing date: 2019-08-30
Publication date: 2020-12-24
Anticipated expiration: 2021-12-20
Also published as: CN110399795B; CN110399795A

Abstract

Provided are a method and apparatus for tracking a human face in a video, and a computer device and a storage medium. The method comprises: acquiring, by means of a front end of a system, a facial image of a video frame to be tracked; acquiring facial feature information and sending same to a back end of the system, and performing human face matching by means of the back end of the system to obtain matched second facial information; and displaying the second facial information on a display platform in the form of an image by means of the back end of the system. The matching speed for human face tracking is improved, and the operating load of a back end of a system is reduced.

Description

Face tracking method, device, computer equipment and storage medium in video

本申请要求于2019年6月20日提交中国专利局、申请号为201910537929.0，申请名称为“视频中的人脸追踪方法、装置、计算机设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on June 20, 2019, the application number is 201910537929.0, and the application title is "Face tracking method, device, computer equipment and storage medium in video", all of which The content is incorporated in this application by reference.

Technical field

本申请涉及计算机技术领域，特别涉及一种视频中的人脸追踪方法、装置、计算机设备和存储介质。This application relates to the field of computer technology, and in particular to a method, device, computer equipment and storage medium for face tracking in video.

Background technique

目前，人脸识别追踪的方式通常是通过前端拍摄获取连续不断的视频，将视频分成每一视频帧传送至后台，通过后台服务器处理每一视频帧内的人脸图像，并与存储库中的人脸进行匹配对应，导致后台的计算量过大，负载重，且匹配的时间长，前端不能及时获取对应的人脸信息。At present, the method of face recognition and tracking is usually to obtain continuous video through front-end shooting, divide the video into each video frame and transmit it to the back-end, and process the face image in each video frame through the back-end server, and compare it with the storage library. The matching of faces results in too much calculation in the background, heavy load, and long matching time, and the front end cannot obtain the corresponding face information in time.

technical problem

本申请的主要目的为提供一种视频中的人脸追踪方法、装置、计算机设备和存储介质，旨在提高追踪人脸的匹配速度。The main purpose of this application is to provide a face tracking method, device, computer equipment and storage medium in video, aiming to improve the matching speed of tracking human faces.

Technical solutions

本申请提供了一种视频中的人脸追踪方法，包括：This application provides a face tracking method in video, including:

通过系统前端获取待追踪的视频帧，其中，所述视频帧包括至少一张第一人脸，获取所有所述第一人脸的人脸图像；Obtain a video frame to be tracked through the front end of the system, where the video frame includes at least one first human face, and acquire all face images of the first human face;

将每一所述第一人脸图像输入到预设于所述系统前端的检测模型中进行检测，获取到对应的多个第一人脸特征信息，其中，所述检测模型利用已知的人脸图像，基于卷积神经网络训练得到；Each of the first face images is input into a detection model preset in the front end of the system for detection, and a plurality of corresponding first face feature information is obtained, wherein the detection model uses a known person Face image, obtained based on convolutional neural network training;

将所有所述第一人脸特征信息发送至系统后端，通过所述系统后端对每一所述第一人脸对应的所述第一人脸特征信息与预设于所述系统后端存储库中的人脸进行匹配；All the first facial feature information is sent to the system backend, and the first facial feature information corresponding to each first face is preset in the system backend through the system backend Match the faces in the repository;

若通过所述系统后端从所述存储库中匹配到与所述第一人脸吻合的第二人脸，则获取所述第二人脸的信息；If a second human face that matches the first human face is matched from the storage library through the system backend, acquiring information of the second human face;

通过所述系统后端将所述第二人脸的信息以图像的形式加载于显示平台上显示。The information of the second face is loaded on a display platform in the form of an image through the back end of the system for display.

本申请还提供了一种视频中的人脸追踪装置，包括：This application also provides a face tracking device in video, including:

第一获取模块，用于通过系统前端获取待追踪的视频帧，其中，所述视频帧包括至少一张第一人脸，获取所有所述第一人脸的人脸图像；The first acquisition module is configured to acquire a video frame to be tracked through the front end of the system, wherein the video frame includes at least one first human face, and acquires all face images of the first human face;

第二获取模块，用于将每一所述第一人脸图像输入到预设于所述系统前端的检测模型中进行检测，获取到对应的多个第一人脸特征信息，其中，所述检测模型利用已知的人脸图像，基于卷积神经网络训练得到；The second acquisition module is configured to input each of the first face images into a detection model preset in the front end of the system for detection, and acquire a plurality of corresponding first face feature information, where the The detection model is obtained by using known face images based on convolutional neural network training;

第一匹配模块，用于将所有所述第一人脸特征信息发送至系统后端，通过所述系统后端对每一所述第一人脸对应的所述第一人脸特征信息与预设于所述系统后端存储库中的人脸进行匹配；The first matching module is configured to send all the first facial feature information to the back end of the system, and compare the first facial feature information corresponding to each of the first faces with the pre- Matching faces in the back-end storage library of the system;

第三获取模块，用于若通过所述系统后端从所述存储库中匹配到与所述第一人脸吻合的第二人脸，则获取所述第二人脸的信息；The third acquisition module is configured to acquire information about the second human face if a second human face that matches the first human face is matched from the storage library through the system backend;

显示模块，用于通过所述系统后端将所述第二人脸的信息以图像的形式加载于显示平台上显示。The display module is used to load the information of the second face in the form of images on the display platform through the back end of the system for display.

本申请还提供一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述处理器执行所述计算机可读指令时实现上述任一项所述方法的步骤。The present application also provides a computer device, including a memory and a processor, the memory stores computer-readable instructions, and the processor implements the steps of any one of the above methods when the computer-readable instructions are executed by the processor.

本申请还提供一种计算机非易失性可读存储介质，其上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现上述任一项所述的方法的步骤。The present application also provides a computer non-volatile readable storage medium, on which computer readable instructions are stored, and when the computer readable instructions are executed by a processor, the steps of any of the above methods are implemented.

Beneficial effect

本申请中提供的视频中的人脸追踪方法、装置、计算机设备和存储介质，通过系统前端获取到第一人脸对应的人脸特征信息，并通过系统后端进行匹配该人脸特征信息，以获取到对应的第二人脸，将该第二人脸的信息以图像形式显示于显示平台上，利用系统的前端与后端同时进行操作，通过系统前端去分析获取人脸的特征信息，提高追踪人脸的匹配速度，且无需将获取人脸的特征信息归属于系统后端操作，降低系统后端运行负载。The face tracking method, device, computer equipment, and storage medium in the video provided in this application obtain the face feature information corresponding to the first face through the front end of the system, and match the face feature information through the back end of the system. In order to obtain the corresponding second face, the information of the second face is displayed on the display platform in image form, the front end and the back end of the system are used to operate at the same time, and the characteristic information of the face is obtained through the front end of the system. Improve the matching speed of tracking human faces, and there is no need to attribute the acquisition of facial feature information to the back-end operation of the system, and reduce the back-end operation load of the system.

Description of the drawings

图1是本申请一实施例中视频中的人脸追踪方法步骤示意图；FIG. 1 is a schematic diagram of the steps of a face tracking method in a video in an embodiment of the present application;

图2是本申请一实施例中视频中的人脸追踪装置结构框图；2 is a structural block diagram of a face tracking device in a video in an embodiment of the present application;

图3是本申请一实施例的计算机设备的结构示意框图。FIG. 3 is a schematic block diagram of the structure of a computer device according to an embodiment of the present application.

本申请的最佳实施方式The best implementation of this application

应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

参照图1，为本申请一实施例中提供了一种视频中的人脸追踪方法，包括以下步骤：1, a face tracking method in video is provided in an embodiment of this application, which includes the following steps:

步骤S1，通过系统前端获取追踪的视频帧，其中，所述视频帧包括至少一张第一人脸，获取所有所述第一人脸的人脸图像；Step S1: Obtain a tracked video frame through the front end of the system, where the video frame includes at least one first face, and all face images of the first face are obtained;

步骤S2，将每一所述第一人脸图像输入到预设于所述系统前端的检测模型中进行检测，获取到对应的多个第一人脸特征信息，其中，所述检测模型利用已知的人脸图像，基于卷积神经网络训练得到；Step S2: Input each of the first face images into a detection model preset in the front end of the system for detection, and obtain a plurality of corresponding first face feature information, wherein the detection model uses the Known face image, obtained based on convolutional neural network training;

步骤S3，将所有所述第一人脸特征信息发送至系统后端，通过所述系统后端对每一所述第一人脸对应的所述第一人脸特征信息与预设于所述系统后端存储库中的人脸进行匹配；Step S3: Send all the first facial feature information to the back end of the system. Through the back end of the system, the first facial feature information corresponding to each first face is preset to the Match the face in the back-end repository of the system;

步骤S4，若通过所述系统后端从所述存储库中匹配到与所述第一人脸吻合的第二人脸，则获取所述第二人脸的信息；Step S4, if a second human face that matches the first human face is matched from the storage library through the system backend, obtain information of the second human face;

步骤S5，通过所述系统后端将所述第二人脸的信息以图像的形式加载于显示平台上显示。In step S5, the information of the second face is loaded on a display platform in the form of an image through the back end of the system for display.

以上步骤中，将实时采集的视频流根据时间的顺序分成一张一张的视频帧，获取到存在人脸的视频帧，其中，视频帧的图片格式可以设置为JPG、PNG和BMP图片格式，通过系统前端获取视频帧内所有的第一人脸的人脸图像，将每一第一人脸图像输入到检测模型中进行检测，其中，该检测模型是利用已知的人脸图像，基于卷积神经网络训练得到，获取每一人脸图像不同部位的区别特征，以便可以根据区别特征得到对应的人脸特征信息；将每一第一人脸的所有第一人脸特征信息发送至系统后端，通过系统后端对每一第一人脸对应的第一人脸特征信息与预设于系统后端存储库中的人脸进行匹配，其中，系统后预设有存储多张人脸的存储库，若从存储库中匹配到与所述第一人脸对应的第一人脸特征信息吻合的第二人脸，则获取第二人脸的信息，在一具体实施例中，所述第二人脸的信息包括有姓名、性别、身份证、来往记录、业务等等；将获取到的第二人脸的信息发送至显示平台上，并以图像的形式显示。In the above steps, the video stream collected in real time is divided into video frames one by one according to the order of time, and the video frames with human faces are obtained. Among them, the picture format of the video frame can be set to JPG, PNG and BMP picture formats. Obtain the face images of all the first faces in the video frame through the front end of the system, and input each first face image into the detection model for detection. The detection model uses known face images based on the volume Obtained by the product neural network training to obtain the distinguishing characteristics of different parts of each face image, so that the corresponding face characteristic information can be obtained according to the distinguishing characteristics; all the first face characteristic information of each first face is sent to the back end of the system , Through the back end of the system, the first face feature information corresponding to each first face is matched with the face preset in the back-end storage library of the system, and the system is preset to store multiple faces in the back If a second face that matches the first face feature information corresponding to the first face is matched from the storage library, the information of the second face is obtained. In a specific embodiment, the first face The information of the two faces includes name, gender, ID card, transaction records, business, etc.; the obtained information of the second face is sent to the display platform and displayed in the form of an image.

卷积神经网络，是图像识别领域的核心算法之一，并在大量学习数据时有稳定的表现。对于一般的大规模图像分类问题，卷积神经网络可用于构建阶层分类器(hierarchical classifier)，也可以在精细分类识别(fine-grained recognition)中用于提取图像的判别特征以供其它分类器进行学习。对于后者，特征提取可以人为地将图像的不同部分分别输入卷积神经网络，也可以由卷积神经网络通过非监督学习自行提取。Convolutional neural network is one of the core algorithms in the field of image recognition, and has a stable performance when learning a large amount of data. For general large-scale image classification problems, convolutional neural networks can be used to construct hierarchical classifiers, or in fine-grained recognition (fine-grained recognition) to extract discriminative features of images for other classifiers to perform Learn. For the latter, the feature extraction can artificially input different parts of the image into the convolutional neural network, or it can be extracted by the convolutional neural network through unsupervised learning.

在一具体实施例中，若利用系统后端并未匹配到与所述第一人脸吻合的第二人脸信息，则判断该第一人脸为陌生脸孔，将该第一人脸剪辑成相片，并放置于预设格式的电子表中，发送至显示平台显示，以便查看显示平台的后台工作人员可以尽快联系对应的呈陌生状态的第一人脸。In a specific embodiment, if the back end of the system does not match the second face information that matches the first face, it is determined that the first face is a strange face, and the first face is edited Create a photo, place it in an electronic watch in a preset format, and send it to the display platform for display, so that the background staff who view the display platform can contact the corresponding first face in an unfamiliar state as soon as possible.

在本实施例中，对获取到的每一视频帧的人脸，采用UUID进行识别标识，其中上述UUID是通用唯一识别码(Universally Unique Identifier)的缩写，以让分布式系统中的所有元素，都能有唯一的辨识资讯，而不需要透过中央控制端来做辨识资讯的指定。如此，每个人都可以建立不与其它人冲突的UUID，具体的，当前视频帧通过神经网络识别进行剪脸，识别视频帧内有5个人的人脸特征，5个人的配置标识可为1、2、3、4、5，其配置的标识相互独立，不重复，不会识别该人脸的特征后，显示其他的人的信息。若标识为1的人脸脱离监控范围，重新进入时，则会标识为新的标识信息，如标识为6。In this embodiment, a UUID is used to identify the face of each acquired video frame, where the above UUID is the abbreviation of Universally Unique Identifier to allow all elements in the distributed system to All have unique identification information, and there is no need to specify identification information through the central control terminal. In this way, everyone can establish a UUID that does not conflict with other people. Specifically, the current video frame is recognized by neural network to cut the face, and the facial features of 5 people in the video frame are recognized. The configuration identifier of 5 people can be 1, 2, 3, 4, and 5, the configured logos are independent of each other, do not repeat, and will not display the information of other people after identifying the characteristics of the face. If the face identified as 1 leaves the monitoring range and re-enters, it will be identified as new identification information, such as the identification number 6.

在一实施例中，所述若通过所述系统后端从所述存储库中匹配到与所述第一人脸特征信息吻合的第二人脸，则获取所述第二人脸的信息的步骤S4之后，还包括：In an embodiment, if a second face that matches the first face feature information is matched from the storage library through the system backend, the information of the second face is acquired After step S4, it also includes:

步骤S41，通过所述系统后端将匹配到的所有所述第二人脸存储于预设的临时库中，其中，所述临时库为定时删除存储于内的人脸数据的存储库；Step S41, storing all the matched second faces in a preset temporary library through the back end of the system, where the temporary library is a storage library for periodically deleting the stored face data;

步骤S42，若在待追踪的视频帧中获取到新的第一人脸的人脸特征信息时，与所述临时库内的人脸进行匹配；Step S42, if the face feature information of the new first face is acquired in the video frame to be tracked, it is matched with the face in the temporary library;

步骤S43，若与所述临时库内的人脸匹配失败，则将所述新的第一人脸的人脸特征信息与所述存储库中的人脸进行匹配；Step S43: if the matching with the face in the temporary library fails, then the face feature information of the new first face is matched with the face in the storage library;

步骤S44，若与所述存储库中的人脸匹配成功，则将所述匹配成功的所述新的第一人脸的信息存储于所述临时库；若与所述存储库中的人脸匹配失败，则以图像的形式发送至所述显示平台上显示。Step S44, if the matching with the face in the storage library is successful, store the information of the new first face that is successfully matched in the temporary library; if it matches the face in the storage library If the matching fails, it is sent to the display platform for display in the form of an image.

以上步骤中，上述临时库为预设于系统后端的临时存储的临时库，其可根据需要进行代码改写，以更改临时库定时删除存储于内的人脸数据，如时间设置在半小时或者十分钟，临时库内的人脸数据若超过指定的时间，则自动删除，在其他实施例中，还可以通过用最新接收到的第二人脸数据替换最早的接收到的第二人脸数据。In the above steps, the above-mentioned temporary library is a temporary library preset in the back end of the system. The code can be rewritten as needed to change the temporary library and periodically delete the face data stored in it. For example, the time is set at half an hour or ten. Minutes, if the face data in the temporary library exceeds the specified time, it will be automatically deleted. In other embodiments, it is also possible to replace the earliest received second face data with the latest received second face data.

系统后端将匹配成功的所有第二人脸数据存储于临时库中，当在待追踪的视频帧中获取到新的第一人脸的人脸特征信息时，先将该新的第一人脸特征与临时库内的人脸进行匹配，其中，追踪的视频帧分为一帧一帧，依顺序对所有的视频帧进行获取其中的人脸，也即是对每一帧视频进行获取新的第一人脸，若新的第一人脸与临时库内的人脸匹配失败，则将新的第一人脸的人脸特征与存储库中的人脸进行匹配，若与所述存储库中的人脸匹配成功，则将所述匹配成功的所述新的第一人脸的信息存储于所述临时库；若与所述存储库中的人脸匹配失败，则以图像的形式直接发送至所述显示平台上显示。The back end of the system stores all the successfully matched second face data in a temporary library. When the face feature information of the new first face is obtained in the video frame to be tracked, the new first person The facial features are matched with the faces in the temporary library, where the tracked video frames are divided into one frame by one, and all the video frames are acquired in sequence, that is, each frame of video is acquired. If the new first face fails to match the face in the temporary library, then the face feature of the new first face is matched with the face in the storage library, if it matches the stored If the face in the library is successfully matched, the information of the new first face that is successfully matched is stored in the temporary library; if the face in the storage library fails to be matched, it is in the form of an image Send directly to the display platform for display.

在一实施例中，所述若在待追踪的视频帧中获取到新的第一人脸的人脸特征信息时，与所述临时库内的人脸进行匹配的步骤S42之后，包括：In an embodiment, if the face feature information of the new first face is acquired in the video frame to be tracked, after step S42 of matching with the face in the temporary library, the method includes:

步骤S402，若与所述临时库内的人脸匹配成功，则将与所述新的第一人脸特征信息匹配的人脸的信息以图像的信息加载于所述显示平台上显示，并更新所述临时库内所匹配的人脸的最新匹配时间。Step S402: If the matching with the face in the temporary library is successful, the information of the face matching the new first facial feature information is loaded on the display platform as image information for display, and updated The latest matching time of the matched face in the temporary library.

以上步骤中，若新的第一人脸的人脸特征信息与临时库内的人脸匹配成功，则将与新的第一人脸特征信息匹配的人脸的信息以图像的信息加载于显示平台上显示，无需再次进入存储库中进行人脸匹配，并更新临时库内所匹配的人脸的最新匹配时间，以便延长在临时库中匹配成功的人脸的时间，减少系统后端将获取的下一视频帧人脸的人脸特征信息与存储库进行匹配，节省时间。In the above steps, if the facial feature information of the new first face is successfully matched with the face in the temporary library, the information of the face that matches the new first facial feature information is loaded into the display as the image information. It is displayed on the platform that there is no need to enter the repository again for face matching, and update the latest matching time of the matched faces in the temporary library, so as to extend the time for successfully matching faces in the temporary library and reduce the system backend acquisition The face feature information of the face in the next video frame is matched with the storage library, saving time.

在一实施例中，所述通过所述系统后端将所述第二人脸的信息以图像的形式加载于显示平台上显示的步骤S5，包括：In an embodiment, the step S5 of loading the information of the second human face in the form of an image on a display platform through the back end of the system for display includes:

步骤S51，判断所述第二人脸的信息是否是有重点标记，其中，所述重点标记包括预先标记为优先显示的人脸或者重点关注的人脸；Step S51: It is judged whether the information of the second face is a key mark, wherein the key mark includes a face that is pre-marked for priority display or a face that is of focus;

步骤S52，若是，则将设有所述重点标记的所述第二人脸的信息以图像的形式加载于所述显示平台的中心区域。Step S52: If yes, load the information of the second face with the key mark in the form of an image in the central area of the display platform.

以上步骤中，上述的重点标记包括针对某一人脸进行的优先显示，重点关注等的标记，系统针对重点标记的人脸，其对应会做出对应的突出标记，判断第二人脸的信息是否携带有重点标记，若判断携带有重点标记，则将第二人脸的信息以图像的形式加载于显示平台的中心区域进行显示。如在一具体实施例中，一般客户显示界面为蓝色，而重点客户显示界面为红色，将具有重点标记的第二人脸的信息以图像的形式加载于显示平台的中心区域进行显示，以便后台工作人员可以及时了解重点标记的人脸(客户)信息。In the above steps, the above-mentioned key marks include priority display, key attention, etc. marks for a certain face. The system will make corresponding highlight marks for the key marked faces to determine whether the information of the second face is Carrying a key mark, if it is determined that the key mark is carried, the information of the second face is loaded in the center area of the display platform in the form of an image for display. For example, in a specific embodiment, the general customer display interface is blue, and the key customer display interface is red, and the information of the second face with the key mark is loaded in the form of an image in the central area of the display platform for display, so that The back-end staff can learn about the key-marked face (customer) information in time.

在一实施例中，所述通过所述系统后端对每一所述第一人脸对应的所述第一人脸特征信息与预设于所述系统后端存储库中的人脸进行匹配的步骤S3，包括：In an embodiment, the first face feature information corresponding to each of the first faces is matched with the face preset in the system backend storage library through the system backend The step S3 includes:

步骤S31，将任一所述第一人脸对应的所有所述第一人脸特征信息按指定顺序进行排序生成第一矩阵，其中，所述第一矩阵为一横排或者一竖排的矩阵；Step S31: Sort all the first face feature information corresponding to any one of the first faces in a specified order to generate a first matrix, where the first matrix is a horizontal row or a vertical row matrix ；

步骤S32，将所述第一矩阵内的每一所述第一人脸特征信息的个数复制成与预设的第二矩阵的人脸数量相同的个数，并以所述第二矩阵的排序顺序形成第三矩阵，其中，所述第二矩阵为所述存储库内多个人脸中的每一人脸对应的人脸特征信息按照所述第一矩阵的所述第一人脸特征信息的顺序进行排序生成；Step S32: Copy the number of each of the first facial feature information in the first matrix into the same number as the number of the preset second matrix, and use the number of the second matrix The sorting order forms a third matrix, where the second matrix is the number of face feature information corresponding to each of the multiple faces in the storage library according to the first face feature information of the first matrix Sort and generate in order;

步骤S33，将所述第三矩阵与所述第二矩阵进行相减得到第四矩阵；Step S33, subtracting the third matrix and the second matrix to obtain a fourth matrix;

步骤S34，将所述第四矩阵中的每一数值进行绝对值运算，得到第五矩阵；Step S34, performing an absolute value operation on each value in the fourth matrix to obtain a fifth matrix;

步骤S35，将所述第五矩阵的每一人脸所对应的所有数值的绝对值进行相加，得到每一人脸的匹配总值；Step S35, adding the absolute values of all the numerical values corresponding to each face of the fifth matrix to obtain the total matching value of each face;

步骤S36，对比所有所述匹配总值，获取所述匹配总值中的最小值；Step S36, comparing all the total matching values to obtain the minimum value among the total matching values;

步骤S37，判断所述匹配总值中的最小值是否小于人脸阈值；Step S37, judging whether the minimum value in the total matching value is less than a face threshold;

步骤S38，若是，则得到所述匹配总值中的最小值对应的人脸，并从所述存储库中查找与所述匹配总值中的最小值对应的人脸。Step S38, if yes, obtain the face corresponding to the minimum value in the total matching value, and search for the face corresponding to the minimum value in the total matching value from the storage library.

以上步骤中，将第一人脸对应的所有第一人脸特征信息按指定顺序进行排序生成第一矩阵，若第一人脸特征信息有M个，每一个对应有对应的人脸特征信息值，则形成一组一行M列的第一矩阵或者一列M行的第一矩阵，存储库中预设存储有N张人脸，每一人脸均对应有M个特征信息，则那么总共就有N*M个特征信息，以一个待测人脸包括512个特征信息为例，当有N个人脸时，就有N个人脸特征信息，N个人脸特征信息就可以生成一个N行512列的矩阵，即第二矩阵，多个人脸特征信息就生成一个多阶矩阵。其中N个人脸特征信息生成第二矩阵的方法，可参照上述形成第一矩阵的方法，运用c++可以通过OpenCV生成第二矩阵，其中OpenCV是一个基于BSD许可(开源)发行的跨平台计算机视觉库，python(计算机程序设计语言)可以通过NumPy库来生成第二子矩阵，NumPy系统是python的一种开源的数值计算扩展，可用来存储和处理大型矩阵也可运用其他算法生成，在此不做赘述。In the above steps, sort all the first face feature information corresponding to the first face in a specified order to generate the first matrix. If there are M first face feature information, each one has a corresponding face feature information value , The first matrix with one row and M columns or the first matrix with one column and M rows is formed. There are N faces stored in the storage library by default, and each face corresponds to M feature information, so there are N *M feature information, taking a face to be tested including 512 feature information as an example, when there are N faces, there are N face feature information, and N face feature information can generate a matrix with N rows and 512 columns , The second matrix, multiple facial feature information generates a multi-level matrix. The method for generating the second matrix with N facial feature information can refer to the method of forming the first matrix above. The second matrix can be generated by OpenCV using C++, where OpenCV is a cross-platform computer vision library based on BSD license (open source) , Python (computer programming language) can generate the second sub-matrix through the NumPy library. The NumPy system is an open source numerical calculation extension of python. It can be used to store and process large matrices and can also be generated by other algorithms. Repeat.

在一实施例中，以一个第一人脸有512个标号数组为例，第一矩阵为一个1行512列的矩阵，第二矩阵为一个N行512列的矩阵，将所述第一矩阵生成 N行512列的矩阵，其中，每一行均为待测人脸的各元素按照指定顺序排列的人脸特征信息，也即将第一矩阵第1行512列复制成N行512列的矩阵，形成第三矩阵。In an embodiment, taking a first face with 512 label arrays as an example, the first matrix is a matrix with 1 row and 512 columns, and the second matrix is a matrix with N rows and 512 columns. The first matrix is Generate a matrix of N rows and 512 columns, where each row is the face feature information of the elements of the face to be tested arranged in a specified order, that is, copy the first row and 512 columns of the first matrix into a matrix of N rows and 512 columns, Form the third matrix.

将第三矩阵与第二矩阵进行相减，得到新的第四矩阵，其中，第四矩阵为每一人脸之间的人脸特征差异匹配值，第四矩阵内的每个数值代表待测人脸特征信息与该数值所对应的人脸库中的人脸的特征信息计算得到的结果，数值的大小代表待测人脸与第二矩阵中的其中一列所对应的人脸的匹配程度。The third matrix and the second matrix are subtracted to obtain a new fourth matrix, where the fourth matrix is the matching value of the face feature difference between each face, and each value in the fourth matrix represents the person to be tested The face feature information is calculated from the feature information of the face in the face database corresponding to the value, and the magnitude of the value represents the degree of matching between the face to be tested and the face corresponding to one of the columns in the second matrix.

上述步骤中，将第四矩阵中的每一人脸特征差异匹配值进行绝对值运算，得到一个矩阵内的每一数值均大于或等于零的第五矩阵，将第五矩阵中的每一人脸各元素对应的人脸特征差异匹配值的绝对值进行相加，得到每一人脸的匹配总值，该匹配总值为人脸库中的每一人脸与待测人脸之间的差异总值。在一具体实施例中，所述匹配总值包括N个数值，是待测人脸特征信息与人脸库中的N个人脸特征信息分别计算得到的数值，选取匹配总值中N个数值中的最小值，判断所述最小值是否大于预设的人脸阈值，如果小于阈值，则得到所述匹配总值中的最小值对应的人脸，并从所述存储库中查找与所述匹配总值中的最小值对应的人脸，在其他实施例中，如果有多个数值均小于所述阈值，则可能由于阈值设定的过低，或者由于待测人脸的图像不清晰，导致待测人脸的特征信息不准确；可以输入一个较大的阈值，然后再进行运算匹配，或者对待测人脸的图像进行处理，得到更加清晰的图像；如果得到的小于阈值的数值的个数较少时，缩小了筛选的范围，可以直接通过人眼识别；如果大于阈值，则匹配失败。In the above steps, the absolute value of each face feature difference matching value in the fourth matrix is calculated to obtain a fifth matrix in which each value in the matrix is greater than or equal to zero, and each element in the fifth matrix The absolute values of the corresponding face feature difference matching values are added to obtain the total matching value of each face, and the total matching value is the total difference between each face in the face database and the face to be tested. In a specific embodiment, the total matching value includes N values, which are values calculated from the facial feature information to be tested and the N facial feature information in the face database, respectively, and the N values in the total matching value are selected Determine whether the minimum value is greater than the preset face threshold value, and if it is less than the threshold value, obtain the face corresponding to the minimum value in the total matching value, and search the storage library for the face corresponding to the minimum value. The face corresponding to the minimum value in the total value. In other embodiments, if there are multiple values that are all less than the threshold, it may be that the threshold is set too low, or the image of the face to be tested is not clear, resulting in The feature information of the face to be tested is inaccurate; you can enter a larger threshold and then perform arithmetic matching, or process the image of the face to be tested to obtain a clearer image; if the number of values less than the threshold is obtained When it is less, the screening range is reduced and can be directly recognized by human eyes; if it is greater than the threshold, the matching fails.

在一实施例中，所述得到所述匹配总值中的最小值对应的人脸，并从所述存储库中查找与所述匹配总值中的最小值对应的人脸的步骤S38，包括：In an embodiment, the step S38 of obtaining the face corresponding to the minimum value in the total matching value and searching the storage library for the face corresponding to the minimum value in the total matching value includes :

步骤S381，判断所述匹配总值中的最小值所对应的所述第五矩阵的每一人脸特征信息的绝对值是否小于所对应的人脸特征预设阈值；Step S381, judging whether the absolute value of each face feature information of the fifth matrix corresponding to the minimum value in the total matching value is smaller than the corresponding preset threshold of the face feature;

步骤S382，若是，则将所述匹配总值中的最小值对应的人脸作为最匹配的人脸。Step S382, if yes, the face corresponding to the smallest value in the total matching value is taken as the most matching face.

以上步骤中，获取匹配总值中的最小值所对应的第五矩阵的每一人脸特征差异匹配值，并判断每一人脸特征差异匹配值是否小于所对应的人脸特征预设阈值，若是，则将所述匹配总值中的最小值对应的所述预设第二矩阵的人脸特征信息吻合的人脸作为最匹配的人脸，若否，则排除该匹配总值中最小值所对应的人脸作为匹配人脸。如在一具体实施例中，若右眼睛大小的人脸特征差异匹配值为0.09，而在设置的人脸特征预设阈值中，右眼的预设阈值为0.07，即使匹配总值小于人脸阈值，但由于右眼的特征差异匹配值不符合匹配要求，该匹配总值对应的人脸依然不能作为对应的匹配人脸。In the above steps, each face feature difference matching value of the fifth matrix corresponding to the minimum value in the total matching value is obtained, and it is judged whether each face feature difference matching value is less than the corresponding face feature preset threshold, if so, Then, the face that matches the facial feature information of the preset second matrix corresponding to the minimum value in the total matching value is taken as the most matched face, and if not, the face corresponding to the minimum value in the total matching value is excluded As the matching face. For example, in a specific embodiment, if the matching value of the face feature difference of the right eye size is 0.09, and in the preset threshold of the set face features, the preset threshold of the right eye is 0.07, even if the total matching value is less than the face Threshold, but because the matching value of the feature difference of the right eye does not meet the matching requirements, the face corresponding to the total matching value still cannot be used as the corresponding matching face.

在一实施例中，所述将任一所述第一人脸对应的所有所述第一人脸特征信息按指定顺序进行排序生成第一矩阵的步骤S31之后，包括In an embodiment, after the step S31 of sorting all the first face feature information corresponding to any one of the first faces in a specified order to generate a first matrix, the method includes

步骤S311，识别所述第一人脸的性别，其中，所述性别包括男性和女性；Step S311: Identify the gender of the first face, where the gender includes male and female;

步骤S312，根据所述第一人脸的性别，从所述存储库中查找与所述第一人脸性别一致的人脸做成所述第二矩阵。Step S312: According to the gender of the first face, search for a face consistent with the gender of the first face from the storage library to form the second matrix.

以上步骤中，获取到所述第一人脸后，识别所述第一人脸的性别，其中，所述性别包括男性和女性，根据所述待测人脸的性别，从存储库中的人脸进行初步筛选，如在一具体实施例中，识别到第一人脸为男性时，将存储库中的男性人脸提取出来，然后根据获取到的男性人脸获取到对应的人脸特征信息并生成所述第二矩阵，如此便可以缩小范围，并且节省了后续运算的时间。In the above steps, after the first face is obtained, the gender of the first face is recognized, where the gender includes male and female, and according to the gender of the face to be tested, the person in the storage Face preliminary screening. For example, in a specific embodiment, when the first face is recognized as a male, the male face in the storage library is extracted, and then the corresponding facial feature information is obtained according to the acquired male face And generate the second matrix, so that the range can be reduced and the time for subsequent operations can be saved.

在一实施例中，所述识别所述第一人脸的性别的步骤，包括：In an embodiment, the step of recognizing the gender of the first face includes:

步骤S3111，根据获取到的所述第一人脸的图像，获取到所述第一人脸图像的梯度特征；Step S3111: Acquire the gradient feature of the first face image according to the acquired image of the first face;

步骤S3112，根据所述第一人脸图像的梯度特征，得到每一所述梯度特征对应的浮点数值，以通过所述浮点数值获取到所述第一人脸的性别。Step S3112: Obtain a floating point value corresponding to each gradient feature according to the gradient feature of the first face image, so as to obtain the gender of the first face through the floating point value.

在本实施例中，识别人脸图像的性别的方法是：预先获取大量的人脸图像的梯度特征，将提取的人脸图像梯度特征输入到SVM(支持向量机)中训练，通过建立控制台project以及配置OpenCv环境对人脸图像进行训练，得到对应的梯度特征，并以浮点数容器的形式呈现，当获取到待测人脸的图像时，进而可以得到每一元素对应的浮点数值，且获取到对应的人脸性别。In this embodiment, the method for identifying the gender of a face image is: obtaining a large number of gradient features of face images in advance, and inputting the extracted gradient features of face images into SVM (support vector machine) for training, and establishing a console project and configure the OpenCv environment to train the face image, obtain the corresponding gradient feature, and present it in the form of a floating-point number container. When the image of the face to be tested is obtained, the floating-point value corresponding to each element can be obtained. And the corresponding face gender is obtained.

在其他实施例中，还可以将存储库中的所有人脸预先根据性别做成两个大矩阵，当识别到待测人脸的性别后，可以直接提取对应的性别的矩阵进行运算，待后续人脸库中增加新的人脸，可以根据其性别对应放置到对应的性别矩阵中。In other embodiments, all faces in the storage library can be made into two large matrices based on gender in advance. After the gender of the face to be tested is recognized, the matrix of the corresponding gender can be directly extracted for calculation, and the subsequent A new face is added to the face database and can be placed in the corresponding gender matrix according to its gender.

在其他实施例中，还可以通过识别第一人脸所在的年龄层次，根据第一人脸的年龄层次，到所述存储库中查找与所述第一人脸年龄层次一致的人脸做成预设的第二矩阵，以此缩小运算的范围，节省运算时间，使其匹配人脸的速度更快。In other embodiments, by identifying the age level of the first face, according to the age level of the first face, search for a face consistent with the age level of the first face in the storage library to make The preset second matrix reduces the scope of calculation, saves calculation time, and makes it faster to match human faces.

综上所述，为本申请实施例中提供的视频中的人脸追踪方法，通过系统前端获取到第一人脸对应的人脸特征信息，并通过系统后端进行匹配该人脸特征信息，以获取到对应的第二人脸，将该第二人脸的信息以图像形式显示于显示平台上，利用系统的前端与后端同时进行操作，通过系统前端去分析获取人脸的特征信息，提高追踪人脸的匹配速度，且无需将获取人脸的特征信息归属于系统后端操作，降低系统后端运行负载。To sum up, the face tracking method in the video provided in this embodiment of the application obtains the face feature information corresponding to the first face through the front end of the system, and matches the face feature information through the back end of the system. In order to obtain the corresponding second face, the information of the second face is displayed on the display platform in image form, the front end and the back end of the system are used to operate at the same time, and the characteristic information of the face is obtained through the analysis of the front end of the system. Improve the matching speed of tracking human faces, and there is no need to attribute the acquisition of facial feature information to the back-end operation of the system, and reduce the back-end operation load of the system.

参照图3，本申请一实施例中还提供了一种视频中的人脸追踪装置，包括：Referring to Fig. 3, an embodiment of the present application also provides a face tracking device in video, including:

第一获取模块10，用于通过系统前端获取待追踪的视频帧，其中，所述视频帧包括至少一张第一人脸，并获取所有所述第一人脸的人脸图像；The first acquiring module 10 is configured to acquire a video frame to be tracked through the front end of the system, wherein the video frame includes at least one first face, and acquires all face images of the first face;

第二获取模块20，用于将每一所述第一人脸图像输入到预设于所述系统前端的检测模型中进行检测，获取到对应的多个第一人脸特征信息，其中，所述检测模型利用已知的人脸图像，基于卷积神经网络训练得到；The second acquiring module 20 is configured to input each of the first face images into a detection model preset in the front end of the system for detection, and acquire corresponding multiple pieces of first face feature information. The detection model is obtained by using known face images based on convolutional neural network training;

第一匹配模块30，用于将所有所述第一人脸特征信息发送至系统后端，通过所述系统后端对每一所述第一人脸对应的所述第一人脸特征信息与预设于所述系统后端存储库中的人脸进行匹配；The first matching module 30 is configured to send all the first face feature information to the back end of the system, and compare the first face feature information corresponding to each first face with the back end of the system. Match the face preset in the back-end storage library of the system;

第三获取模块40，用于若通过所述系统后端从所述存储库中匹配到与所述第一人脸吻合的第二人脸，则获取所述第二人脸的信息；The third acquiring module 40 is configured to acquire information about the second face if a second face that matches the first face is matched from the storage library through the system backend;

显示模块50，用于通过所述系统后端将所述第二人脸的信息以图像的形式加载于显示平台上显示。The display module 50 is configured to load the information of the second human face in the form of an image on a display platform through the back end of the system for display.

本实施例中，上述第一获取模块、第二获取模块、第一匹配模块、第三获取模块与显示模块的功能和作用的实现过程具体详见上述视频中的人脸追踪方法中对应步骤S1-S5的实现过程，在此不再赘述。In this embodiment, the implementation process of the functions and roles of the first acquisition module, the second acquisition module, the first matching module, the third acquisition module, and the display module are detailed in the corresponding step S1 in the face tracking method in the video. The implementation process of -S5 will not be repeated here.

在一实施例中，所述视频中的人脸追踪装置还包括：In an embodiment, the face tracking device in the video further includes:

第一存储模块，用于通过所述系统后端将匹配到的所有所述第二人脸存储于预设的临时库中，其中，所述临时库为定时删除存储于内的人脸数据的存储库；The first storage module is configured to store all the matched second human faces in a preset temporary library through the system backend, where the temporary library is for periodically deleting the face data stored therein Repository

第二匹配模块，用于若在待追踪的视频帧中获取到新的第一人脸的人脸特征信息时，与所述临时库内的人脸进行匹配；The second matching module is used for matching with the face in the temporary library if the face feature information of the new first face is acquired in the video frame to be tracked;

第三匹配模块，用于若与所述临时库内的人脸匹配失败，则将所述新的第一人脸的人脸特征信息与所述存储库中的人脸进行匹配；The third matching module is configured to match the face feature information of the new first human face with the human face in the storage database if the matching with the face in the temporary library fails;

第二存储模块，用于若与所述存储库中的人脸匹配成功，则将所述匹配成功的所述新的第一人脸的信息存储于所述临时库；若与所述存储库中的人脸匹配失败，则以图像的形式发送至所述显示平台上显示。The second storage module is configured to store the information of the new first face that is successfully matched in the temporary database if the matching with the face in the storage library is successful; If the face matching fails, it will be sent to the display platform in the form of an image for display.

本实施例中，上述第一存储模块、第二匹配模块、第三匹配模块与第二存储模块的功能和作用的实现过程具体详见上述视频中的人脸追踪方法中对应步骤S41-S44的实现过程，在此不再赘述。In this embodiment, the realization process of the functions and roles of the first storage module, the second matching module, the third matching module, and the second storage module are detailed in the steps S41-S44 in the face tracking method in the above video. The realization process will not be repeated here.

在一实施例中，所述视频中的人脸追踪装置包括：In an embodiment, the face tracking device in the video includes:

加载模块，用于若与所述临时库内的人脸匹配成功，则将与所述新的第一人脸特征信息匹配的人脸的信息以图像的信息加载于所述显示平台上显示，并更新所述临时库内所匹配的人脸的最新匹配时间。The loading module is used to load and display the information of the human face matching the new first facial feature information as image information on the display platform if the matching with the human face in the temporary library is successful, And update the latest matching time of the matched face in the temporary database.

本实施例中，上述加载模块的功能和作用的实现过程具体详见上述视频中的人脸追踪方法中对应步骤S402的实现过程，在此不再赘述。In this embodiment, the implementation process of the function and role of the above loading module is detailed in the implementation process of step S402 in the face tracking method in the above video, which will not be repeated here.

在一实施例中，显示模块50包括：In an embodiment, the display module 50 includes:

第一判断单元，用于判断所述第二人脸的信息是否是有重点标记，其中，所述重点标记包括预先标记为优先显示的人脸或者重点关注的人脸；The first determining unit is configured to determine whether the information of the second face is a key mark, wherein the key mark includes a face that is pre-marked for priority display or a face of focus;

第一执行单元，用于若是，则将设有所述重点标记的所述第二人脸的信息以图像的形式加载于所述显示平台的中心区域。The first execution unit is configured to, if yes, load the information of the second face with the key mark on the central area of the display platform in the form of an image.

本实施例中，上述第一判断单元与第一执行单元的功能和作用的实现过程具体详见上述视频中的人脸追踪方法中对应步骤S51-S52的实现过程，在此不再赘述。In this embodiment, the implementation process of the functions and roles of the first judgment unit and the first execution unit is detailed in the implementation process of the corresponding steps S51-S52 in the face tracking method in the video, which will not be repeated here.

在一实施例中，第一匹配模块30包括：In an embodiment, the first matching module 30 includes:

第一生成单元，用于将任一所述第一人脸对应的所有所述第一人脸特征信息按指定顺序进行排序生成第一矩阵，其中，所述第一矩阵为一横排或者一竖排的矩阵；The first generating unit is configured to sort all the first face feature information corresponding to any one of the first faces in a specified order to generate a first matrix, where the first matrix is a horizontal row or a Vertical matrix

第二生成单元，用于将所述第一矩阵内的每一所述第一人脸特征信息的个数复制成与预设的第二矩阵的人脸数量相同的个数，并以所述第二矩阵的排序顺序形成第三矩阵，其中，所述第二矩阵为所述存储库内多个人脸中的每一人脸对应的人脸特征信息按照所述第一矩阵的所述第一人脸特征信息的顺序进行排序生成；The second generating unit is configured to copy the number of each of the first face feature information in the first matrix into the same number as the preset number of faces in the second matrix, and use the The sorting order of the second matrix forms a third matrix, where the second matrix is the face feature information corresponding to each of the multiple faces in the storage library according to the first person in the first matrix The order of facial feature information is sorted and generated;

第一获取单元，用于将所述第三矩阵与所述第二矩阵进行相减得到第四矩阵；A first obtaining unit, configured to subtract the third matrix and the second matrix to obtain a fourth matrix;

第二获取单元，用于将所述第四矩阵中的每一数值进行绝对值运算，得到第五矩阵；The second obtaining unit is configured to perform an absolute value operation on each value in the fourth matrix to obtain a fifth matrix;

第三获取单元，用于将所述第五矩阵的每一人脸所对应的所有数值的绝对值进行相加，得到每一人脸的匹配总值；The third acquiring unit is configured to add the absolute values of all the numerical values corresponding to each face of the fifth matrix to obtain the total matching value of each face;

第四获取单元，用于对比所有所述匹配总值，获取所述匹配总值中的最小值；The fourth obtaining unit is configured to compare all the total matching values and obtain the minimum value among the total matching values;

第二判断单元，用于判断所述匹配总值中的最小值是否小于人脸阈值；The second judging unit is used to judge whether the minimum value in the total matching value is less than the face threshold;

第二执行单元，用于若是，则得到所述匹配总值中的最小值对应的人脸，并从所述存储库中查找与所述匹配总值中的最小值对应的人脸。The second execution unit is configured to, if yes, obtain the face corresponding to the minimum value in the total matching value, and search the storage library for the face corresponding to the minimum value in the total matching value.

本实施例中，上述第一生成单元、第二生成单元、第一获取单元、第二获取单元、第三获取单元、第四获取单元、第二判断单元与第二执行单元的功能和作用的实现过程具体详见上述视频中的人脸追踪方法中对应步骤S31-S38的实现过程，在此不再赘述。In this embodiment, the functions and effects of the above-mentioned first generation unit, second generation unit, first acquisition unit, second acquisition unit, third acquisition unit, fourth acquisition unit, second judgment unit, and second execution unit For details of the implementation process, please refer to the implementation process corresponding to steps S31-S38 in the face tracking method in the above video, and will not be repeated here.

在一实施例中，第二执行单元包括：In an embodiment, the second execution unit includes:

判断子单元，用于判断所述匹配总值中的最小值所对应的所述第五矩阵的每一人脸特征信息的绝对值是否小于所对应的人脸特征预设阈值；A judging subunit for judging whether the absolute value of each face feature information of the fifth matrix corresponding to the minimum value in the total matching value is less than the corresponding preset threshold of the face feature;

执行子单元，用于若是，则将所述匹配总值中的最小值对应的人脸作为最匹配的人脸。The execution subunit is configured to, if yes, use the face corresponding to the smallest value in the total matching value as the most matched face.

本实施例中，上述判断子单元与执行子单元的功能和作用的实现过程具体详见上述视频中的人脸追踪方法中对应步骤S381-S382的实现过程，在此不再赘述。In this embodiment, the implementation process of the functions and roles of the judgment subunit and the execution subunit is detailed in the implementation process of the corresponding steps S381-S382 in the face tracking method in the video, which will not be repeated here.

识别单元，用于识别所述第一人脸的性别，其中，所述性别包括男性和女性；A recognition unit, configured to recognize the gender of the first face, where the gender includes male and female;

查找单元，用于根据所述第一人脸的性别，从所述存储库中查找与所述第一人脸性别一致的人脸做成所述第二矩阵。The searching unit is configured to search the storage database for the faces with the same gender as the first face according to the gender of the first face to form the second matrix.

本实施例中，上述识别单元与查找单元的功能和作用的实现过程具体详见上述视频中的人脸追踪方法中对应步骤S311-S312的实现过程，在此不再赘述。In this embodiment, the implementation process of the functions and roles of the recognition unit and the search unit is detailed in the implementation process of the corresponding steps S311-S312 in the face tracking method in the video, which will not be repeated here.

在一实施例中，识别单元包括：In an embodiment, the identification unit includes:

第一获取子单元，用于根据获取到的所述第一人脸的图像，获取到所述第一人脸图像的梯度特征；The first acquiring subunit is configured to acquire the gradient feature of the first face image according to the acquired image of the first face;

第二获取子单元，用于根据所述第一人脸图像的梯度特征，得到每一所述梯度特征对应的浮点数值，以通过所述浮点数值获取到所述第一人脸的性别。The second obtaining subunit is used to obtain the floating point value corresponding to each gradient feature according to the gradient feature of the first face image, so as to obtain the gender of the first face through the floating point value .

本实施例中，上述第一获取子单元与第二获取子单元的功能和作用的实现过程具体详见上述视频中的人脸追踪方法中对应步骤S3111-S3112的实现过程，在此不再赘述。In this embodiment, the implementation process of the functions and roles of the first acquisition subunit and the second acquisition subunit are detailed in the implementation process of the corresponding steps S3111-S3112 in the face tracking method in the video above, and will not be repeated here. .

综上所述，为本申请实施例中提供的视频中的人脸追踪装置，通过系统前端获取到第一人脸对应的人脸特征信息，并通过系统后端进行匹配该人脸特征信息，以获取到对应的第二人脸，将该第二人脸的信息以图像形式显示于显示平台上，利用系统的前端与后端同时进行操作，通过系统前端去分析获取人脸的特征信息，提高追踪人脸的匹配速度，且无需将获取人脸的特征信息归属于系统后端操作，降低系统后端运行负载。In summary, the face tracking device in the video provided in this embodiment of the application obtains the face feature information corresponding to the first face through the front end of the system, and matches the face feature information through the back end of the system. In order to obtain the corresponding second face, the information of the second face is displayed on the display platform in image form, the front end and the back end of the system are used to operate at the same time, and the characteristic information of the face is obtained through the analysis of the front end of the system. Improve the matching speed of tracking human faces, and there is no need to attribute the acquisition of facial feature information to the back-end operation of the system, and reduce the back-end operation load of the system.

参照图3，本申请实施例中还提供一种计算机设备，该计算机设备可以是服务器，其内部结构可以如图3所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设计的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性可读存储介质、内存储器。该非易失性可读存储介质存储有操作系统、计算机可读指令和数据库。该内存储器为非易失性可读存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储第二人脸等数据。该计算机可读指令的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令在执行时，执行如上述个方法的实施例的流程。本领域技术人员可以理解，图3中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定。3, an embodiment of the present application also provides a computer device. The computer device may be a server, and its internal structure may be as shown in FIG. 3. The computer equipment includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the computer designed processor is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile readable storage medium and an internal memory. The non-volatile readable storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile readable storage medium. The database of the computer device is used to store data such as the second face. The network interface of the computer-readable instructions is used to communicate with an external terminal through a network connection. When the computer-readable instruction is executed, it executes the process of the above-mentioned method embodiment. Those skilled in the art can understand that the structure shown in FIG. 3 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.

本申请一实施例还提供一种计算机非易失性可读存储介质，其上存储有计算机可读指令，该计算机可读指令在执行时，可包括如上述各方法的实施例的流程。An embodiment of the present application further provides a computer non-volatile readable storage medium on which computer readable instructions are stored. When the computer readable instructions are executed, they may include the processes of the above-mentioned method embodiments.

以上所述仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其它相关的技术领域，均同理包括在本申请的专利保护范围内。The above are only the preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the specification and drawings of this application, or directly or indirectly applied to other related The technical field is equally included in the scope of patent protection of this application.

Claims

A face tracking method in video, which is characterized in that it comprises the following steps:

Acquiring a video frame to be tracked through the front end of the system, where the video frame includes at least one first human face, and acquiring all face images of the first human face;

Each of the first face images is input into a detection model preset in the front end of the system for detection, and a plurality of corresponding first face feature information is obtained, wherein the detection model uses a known person Face image, obtained based on convolutional neural network training;

All the first facial feature information is sent to the system backend, and the first facial feature information corresponding to each first face is preset in the system backend through the system backend Match the faces in the repository;

If a second human face that matches the first human face is matched from the storage library through the system backend, acquiring information of the second human face;

The information of the second face is loaded on a display platform in the form of an image through the back end of the system for display.

The face tracking method in video according to claim 1, wherein if the second face that matches the first face is matched from the storage library through the system backend, After the step of obtaining the information of the second face, it further includes:

Store all the matched second faces in a preset temporary library through the system backend, where the temporary library is a storage library that periodically deletes the face data stored therein;

If the face feature information of the new first face is acquired in the video frame to be tracked, matching with the face in the temporary library;

If the matching with the face in the temporary library fails, matching the face feature information of the new first face with the face in the storage library;

If the matching with the face in the storage library is successful, store the information of the new first face that is successfully matched in the temporary library; if the matching with the face in the storage library fails, Then it is sent to the display platform for display in the form of an image.

The face tracking method in a video according to claim 2, wherein if the face feature information of the new first face is acquired in the video frame to be tracked, the information in the temporary library After the steps of matching face, include:

If the matching with the face in the temporary library is successful, the information of the face matching the new first facial feature information is loaded on the display platform as image information for display, and the temporary database is updated. The latest matching time of the matched face in the library.

The method for tracking a face in a video according to claim 1, wherein the step of loading the information of the second face in the form of an image on a display platform through the back end of the system for display includes :

Judging whether the information of the second face is provided with a key mark, wherein the key mark includes a face that is pre-marked for priority display or a face that is of focus;

If so, the information of the second face with the key mark is loaded in the central area of the display platform in the form of an image.

The method for tracking a face in a video according to claim 1, wherein the first face feature information corresponding to each of the first faces is preset to The step of matching faces in the back-end storage library of the system includes:

Sort all the first face feature information corresponding to any one of the first human faces in a specified order to generate a first matrix, where the first matrix is a horizontal row or a vertical row;

Copy the number of each of the first facial feature information in the first matrix into the same number as the preset number of faces in the second matrix, and form it in the order of the second matrix The third matrix, wherein the second matrix is that the facial feature information corresponding to each of the multiple faces in the storage library is sorted according to the order of the first facial feature information of the first matrix generate;

Subtracting the third matrix and the second matrix to obtain a fourth matrix;

Performing an absolute value operation on each value in the fourth matrix to obtain a fifth matrix;

Adding the absolute values of all the numerical values corresponding to each face of the fifth matrix to obtain the total matching value of each face;

Comparing all the total matching values to obtain the smallest value among the total matching values;

Judging whether the minimum value in the total matching value is less than a face threshold;

If it is, the face corresponding to the minimum value in the total matching value is obtained, and the face corresponding to the minimum value in the total matching value is searched from the storage library.

The method for tracking a face in a video according to claim 5, wherein said obtaining the face corresponding to the smallest value in the total matching value, and searching the storage library for the face corresponding to the total matching value The steps of the face corresponding to the minimum value in include:

Judging whether the absolute value of each facial feature information of the fifth matrix corresponding to the minimum value in the total matching value is less than the corresponding preset threshold of facial features;

If so, the face corresponding to the smallest value in the total matching value is taken as the most matching face.

The face tracking method in a video according to claim 5, wherein the first face feature information corresponding to any one of the first faces is sorted in a specified order to generate a first matrix After the steps, include:

Identifying the gender of the first face, where the gender includes male and female;

According to the gender of the first human face, the second matrix is formed by searching for human faces with the same gender as the first human face from the storage database.

The face tracking method in a video according to claim 5, wherein the step of identifying the gender of the first face comprises:

Acquiring the gradient feature of the first human face image according to the acquired image of the first human face;

According to the gradient feature of the first face image, a floating point value corresponding to each gradient feature is obtained, so as to obtain the gender of the first face through the floating point value.

A face tracking device in video, which is characterized in that it comprises:

The first acquisition module is configured to acquire a video frame to be tracked through the front end of the system, wherein the video frame includes at least one first human face, and acquires all face images of the first human face;

The second acquisition module is configured to input each of the first face images into a detection model preset in the front end of the system for detection, and acquire a plurality of corresponding first face feature information, where the The detection model is obtained by using known face images based on convolutional neural network training;

The first matching module is configured to send all the first facial feature information to the back end of the system, and compare the first facial feature information corresponding to each of the first faces with the pre- Matching faces in the back-end storage library of the system;

The third acquisition module is configured to acquire information about the second human face if a second human face that matches the first human face is matched from the storage library through the system backend;

The display module is used to load the information of the second face in the form of images on the display platform through the back end of the system for display.

The face tracking device in a video according to claim 9, further comprising:

The first storage module is configured to store all the matched second human faces in a preset temporary library through the system backend, where the temporary library is for periodically deleting the face data stored therein Repository

The second matching module is used for matching with the face in the temporary library if the face feature information of the new first face is acquired in the video frame to be tracked;

The third matching module is configured to match the face feature information of the new first human face with the human face in the storage database if the matching with the face in the temporary library fails;

The second storage module is configured to store the information of the new first face that is successfully matched in the temporary database if the matching with the face in the storage library is successful; If the face matching fails, it will be sent to the display platform in the form of an image for display.

The face tracking device in the video according to claim 10, further comprising:

The loading module is used to load and display the information of the human face matching the new first facial feature information as image information on the display platform if the matching with the human face in the temporary library is successful, And update the latest matching time of the matched face in the temporary database.

The face tracking device in a video according to claim 9, wherein the display module comprises:

The first determining unit is configured to determine whether the information of the second face is provided with a key mark, wherein the key mark includes a face that is pre-marked for priority display or a face of focus;

The first execution unit is configured to, if yes, load the information of the second face with the key mark on the central area of the display platform in the form of an image.

The face tracking device in a video according to claim 9, wherein the first matching module comprises:

The first generating unit is configured to sort all the first face feature information corresponding to any one of the first faces in a specified order to generate a first matrix, where the first matrix is a horizontal row or a Vertical matrix

The second generating unit is configured to copy the number of each of the first face feature information in the first matrix into the same number as the preset number of faces in the second matrix, and use the The sorting order of the second matrix forms a third matrix, where the second matrix is the face feature information corresponding to each of the multiple faces in the storage library according to the first person in the first matrix The order of facial feature information is sorted and generated;

A first obtaining unit, configured to subtract the third matrix and the second matrix to obtain a fourth matrix;

The second obtaining unit is configured to perform an absolute value operation on each value in the fourth matrix to obtain a fifth matrix;

The third acquiring unit is configured to add the absolute values of all the numerical values corresponding to each face of the fifth matrix to obtain the total matching value of each face;

The fourth obtaining unit is configured to compare all the total matching values and obtain the minimum value among the total matching values;

The second judging unit is used to judge whether the minimum value in the total matching value is less than the face threshold;

The second execution unit is configured to, if yes, obtain the face corresponding to the minimum value in the total matching value, and search the storage library for the face corresponding to the minimum value in the total matching value.

The face tracking device in a video according to claim 13, wherein the second execution unit comprises:

A judging subunit for judging whether the absolute value of each face feature information of the fifth matrix corresponding to the minimum value in the total matching value is less than the corresponding preset threshold of the face feature;

The execution subunit is configured to, if yes, use the face corresponding to the smallest value in the total matching value as the most matched face.

The face tracking device in a video according to claim 13, wherein the first matching module comprises:

A recognition unit, configured to recognize the gender of the first face, where the gender includes male and female;

The searching unit is configured to search the storage database for the faces with the same gender as the first face according to the gender of the first face to form the second matrix.

The face tracking device in a video according to claim 15, wherein the recognition unit comprises:

The first acquiring subunit is configured to acquire the gradient feature of the first face image according to the acquired image of the first face;

The second obtaining subunit is used to obtain the floating point value corresponding to each gradient feature according to the gradient feature of the first face image, so as to obtain the gender of the first face through the floating point value .

A computer device includes a memory and a processor, and computer-readable instructions are stored in the memory, wherein the processor implements a face tracking method in a video when the processor executes the computer-readable instructions. The face tracking methods include:

The computer device according to claim 17, wherein if the second face that matches the first face is matched from the storage library through the system backend, the first face is acquired After the steps of the two face information, it also includes:

A computer non-volatile readable storage medium having computer readable instructions stored thereon, wherein the computer readable instructions are executed by a processor to implement a face tracking method in a video, and the person in the video Face tracking methods include:

The computer non-volatile readable storage medium according to claim 19, wherein the second person matching the first person’s face is matched from the storage library through the system backend Face, after the step of obtaining the information of the second human face, the method further includes:

Storing all the matched second faces in a preset temporary library through the system backend, where the temporary library is a storage library for periodically deleting the stored face data;

If the matching with the face in the storage library is successful, store the information of the new first face that was successfully matched in the temporary library; if the matching with the face in the storage library fails, Then it is sent to the display platform for display in the form of an image.