WO2023125028A1 - System for improving positioning precision of pan-tilt camera and control method therefor - Google Patents

Abstract

The invention relates to the technical field of computer vision in a remote monitoring system. The invention relates to a control method for improving the positioning precision of a pan-tilt camera. The control method comprises: firstly, initialization is performed; a panoramic image of an application scenario is displayed on a display screen of a computing platform, a user selects a region of interest in the panoramic image of the application scenario, an image of the region of interest is stored as a target image, and the user selects one target image from a plurality of target images as a target to be rotated. The beneficial effects of the invention are that: control is performed on a software layer, so there is no need to modify the hardware elements of a traditional camera, the target can be accurately positioned through software operation even if the rotating axis of a camera is inaccurate, and there is no need to replace camera devices. Thus, hardware deployment costs can be greatly reduced, and the service life of camera hardware is prolonged. The present invention allows the user to balance the speed and precision of panorama composition and matching calculations based on the actual situation. The present invention is highly flexible.

Description

A system for improving the positioning accuracy of a pan-tilt camera and its control method

This application claims the priority of the Chinese patent application submitted to the China Patent Office on December 27, 2021, with the application number 202111608415.3, and the title of the invention is "A System and Control Method for Improving the Positioning Accuracy of PTZ Cameras", the entire content of which Incorporated in this application by reference.

technical field

The invention belongs to the technical field of computer vision in a remote monitoring system, and in particular relates to a system for improving the positioning accuracy of a pan-tilt camera and a control method thereof.

Background technique

At present, the use of urban security and monitoring equipment has penetrated into all walks of life. The Chinese government continues to increase human, material and financial investment in monitoring security, and strives to build a "harmonious society, safe city" slogan. However, the traditional surveillance camera is a bullet camera, and its monitoring position is fixed, resulting in a limited monitoring range. To solve this problem, PTZ cameras have appeared. Surveillance personnel can adjust the rotation of the PTZ camera by operating the keyboard, and most PTZ cameras provide a preset point function to achieve security defense. The preset point function is to save the current mechanical position of the motor for the PTZ camera, and the user can turn to this position again at any position. The preset point is stored on the SD card inside the camera by recording the pitch angle, deflection angle of the pan/tilt, focal length of the lens, camera aperture, exposure, white balance and other camera parameters, and is given a number for user indexing. The user can let the camera restore its own camera parameters to the parameter values recorded in the preset point according to the preset number of the specified number, so as to realize the picture of the predetermined area. The preset point function greatly facilitates the monitoring personnel to check.

However, the preset point function needs to be manually configured. The method of using the mechanical position as the steering standard determines that the preset point function cannot completely cover the on-site environment. When the monitoring personnel want to view the area other than the preset point It is necessary to manually control the rotation of the pan-tilt, which hinders the monitoring personnel from grasping the situation on the spot in real time. In addition, due to the mechanical aging caused by the long-term operation of the motor, the preset point function often no longer can accurately rotate to the area of interest/focus of the user after long-term use. In addition, due to the aging of the camera shaft, the traditional mechanical rotation method is no longer accurate when the camera is used for a period of time to rotate to the position of the user's attention. Frequent replacement of the camera is required, which increases the hardware cost.

Contents of the invention

The purpose of the invention is to overcome the deficiencies in the prior art, and provide a system and a control method thereof for improving the positioning accuracy of the pan-tilt camera.

For reaching above-mentioned purpose, the present invention provides following technical scheme:

A control method for improving the positioning accuracy of a pan-tilt camera, comprising the steps of:

Step 101, first perform initialization: both the computing platform and the PTZ camera are powered on, and after waiting for the self-test of the computing platform and the PTZ camera to complete, the computing platform calls the network video stream sequence of the PTZ camera frame by frame based on the ffmpeg library; then calculates The platform controls the PTZ camera to collect image information of all application scenarios, and constructs a panorama of the application scenarios based on the image stitching method;

Step 102, the display screen of the computing platform displays a panorama of the application scene, the user selects an area of interest in the panorama of the application scene, and saves the image of the area of interest as a target image, and the user selects an area of interest in a plurality of target images Select a target image as the target to be turned;

Step 103, rough search: the calculation platform receives the target image provided by the user, the FLANN matching algorithm obtains the coordinates of the target image and the current image, and calculates the coordinate difference between the target image and the current image; if the coordinate difference does not meet the threshold condition, then according to the coordinate The difference determines the rotation direction and rotation distance of the PTZ camera, and controls the PTZ camera to rotate according to the obtained rotation direction and rotation distance; if the coordinate difference satisfies the threshold condition, the rotation is not performed; repeat step 103 until the obtained target image The coordinate difference with the current image satisfies the threshold condition;

Step 104. Precise search: the computing platform selects two images that are close to each other and the matching point of the target image, and extracts feature points; use the perspective transformation method to splice the two images in the same coordinate system to obtain an accurate horizontal The coordinate difference Δx _p and the precise ordinate difference Δy _p ; if the abscissa difference and the ordinate difference meet the threshold condition, the computing platform sends a control command through the serial port of the pan-tilt camera, and the pan-tilt camera The distance and direction continue to rotate, and the rotation time is a fixed time window length T _s ; if the difference in abscissa and ordinate does not meet the threshold condition, repeat step 104 until the distance between the two images is less than the set threshold R _s , the image matching is ended; the precise positioning and output of the latitude and longitude of the location of the target to be identified are used for the control circuit to rotate the camera to the target area selected by the user.

Preferably, the PTZ camera is a camera adapted to a network video protocol such as RTSP (Real Time Streaming Protocol) protocol; the computing platform communicates with the PTZ camera in a wired or wireless manner.

As preferably, the computing platform controls the pan-tilt camera to collect the image information of all application scenarios in a specific manner as follows:

The computing platform sends control rotation commands through the serial port of the PTZ camera, and the camera turns left or right for a fixed time window length T _q and then stops; in different scenarios, the user adjusts the balance factor σ to balance the speed of panorama image generation and The success rate, the balance formula is:

A=1-σ,T _q =σT (1)

Among them, A represents the splicing success rate, T represents a fixed time period, and T _q is the fixed time window length; when σ=1, the image acquisition speed is the fastest and the success rate is the lowest; when σ=0, the image acquisition The slowest speed, the highest success rate;

After a rotation stops, the computing platform extracts the real-time video frame and saves the video frame as a picture; the camera continues to extract the video frame and save it as a picture until all the working scene information is saved to the computing platform by the camera.

Preferably, the image stitching method is based on the Stitching method in OpenCV, and the computing platform uses the saved scene information images to stitch to form a panorama of the application scene.

Preferably, step 103 specifically includes the following steps:

Step 103-1. After receiving the target image provided by the user, the computing platform matches the target image with the panoramic image based on the FLANN matching algorithm, and the FLANN matching algorithm returns the coordinates (X, Y) of the target point matching all the features; comparison The coordinates of two adjacent target points, if the distance between the coordinates of two adjacent target points exceeds the length or width of the original image, then redefine the target point as a mismatch point;

x轴最小与y轴最大点

x轴最大与y轴最小点

x轴最大与y轴最大点

若匹配点所在区域的宽度或长度大于原图大小，则舍弃x轴或y轴最边上的两个匹配点，重新构造四个顶点，直到图片尺寸小于等于云台摄像机获取的图片尺寸；对于x轴，若max(X)-min(X)大于原图的宽度，则在匹配的目标点坐标(X,Y)中丢弃x轴坐标中含有max(X),min(X)的所有坐标；对于y轴，若max(Y)-min(Y)大于原图的长度，则在匹配的目标点坐标(X,Y)中丢弃y轴坐标中含有max(Y),min(Y)的所有坐标； Step 103-2. Traverse all matching points and find out the four vertices in the area where the matching points are located: the minimum point on the x-axis and the minimum point on the y-axis

x-axis minimum and y-axis maximum points

x-axis maximum and y-axis minimum point

x-axis maximum and y-axis maximum point

If the width or length of the area where the matching point is located is greater than the size of the original image, discard the two matching points on the extreme side of the x-axis or y-axis, and reconstruct four vertices until the size of the image is smaller than or equal to the size of the image obtained by the PTZ camera; for x-axis, if max(X)-min(X) is greater than the width of the original image, then discard all coordinates containing max(X), min(X) in the x-axis coordinates in the matched target point coordinates (X,Y) ;For the y-axis, if max(Y)-min(Y) is greater than the length of the original image, then discard the coordinates of the y-axis that contain max(Y), min(Y) in the matched target point coordinates (X,Y) all coordinates;

Finally, calculate the midpoint of the matching image based on the four vertices

与目标图像匹配中点

Step 103-3. The computing platform reads the current video frame, compares the current frame and the target image with the panorama of the application scene, and obtains the matching midpoint of the current image

Match the midpoint with the target image

以及在y轴方向的距离

来决定摄像机的转动方向；计算平台通过云台摄像机的串口进行发送控制命令，使云台摄像机以速度θ _c持续转动，摄像机每次的转动时间为一个固定时间窗长T _c；转动一次完成后，计算平台再次读取当前视频帧，重新计算匹配图像的中点以及与目标图像的之间的坐标差值，若坐标差值大于等于预设的精度阈值R _c，则重复步骤103-1至步骤103-4，直到两匹配中点的距离小于设定阈值R _c。 Step 103-4, according to the distance between the current image matching midpoint and the target image matching midpoint in the x-axis direction

and the distance in the y-axis direction

to determine the rotation direction of the camera; the computing platform sends control commands through the serial port of the pan-tilt camera, so that the pan-tilt camera continues to rotate at a speed θ _c , and the rotation time of the camera is a fixed time window length T _c each time; after one rotation is completed , the computing platform reads the current video frame again, recalculates the coordinate difference between the midpoint of the matching image and the target image, and if the coordinate difference is greater than or equal to the preset accuracy threshold R _c , repeat steps 103-1 to Step 103-4, until the distance between the two matching midpoints is less than the set threshold R _c .

As a preference, in the rough search and the fine search proposed in step 103 and step 104, the rotation time window is customized:

_Tc = αT(4)

T _s =βT(5)

Among them, α, β∈[0,1] are used to assist users to adjust the threshold value flexibly; when α=1, β=0, it reaches the maximum value, each rotation takes a long time, the number of calculations is small, and the matching speed is fast; When α=0, β=0, T _c =T _s =T _min , where T _min is the set minimum rotation time window.

As a preference, in step 103 and step 104, the computing platform controls the pan-tilt camera to rotate according to the obtained rotation direction and rotation distance, and the rotation direction has the following 8 situations:

When Δx>R, -R≤Δy≤R, the pan-tilt camera rotates to the right at the speed θ _c ;

When Δx<R, -R≤Δy≤R, the pan-tilt camera rotates to the left at the speed θ _c ;

When -R≤Δx≤R, Δy>R, the pan-tilt camera rotates upward at the speed θ _c ;

When -R≤Δx≤R, Δy<R, the pan-tilt camera rotates downward at the speed θ _c ;

When Δx>R, Δy>R, the pan-tilt camera rotates to the upper right at the speed θ _c ;

When Δx>R, Δy≤-R, the PTZ camera rotates downward and rightward at the speed θ _c ;

When Δx≤-R, Δy>R, the pan-tilt camera rotates to the upper left at the speed θ _c ;

When Δx≤-R, Δy≤-R, the pan-tilt camera rotates to the lower left at the speed θ _c ;

Among them, R>0, R is the preset accuracy threshold R _c or R _s ; in rough search, when the difference between the camera coordinates and the target coordinates is less than or equal to R _c , stop the rough search; in exact match, when When the difference between the coordinates of the camera and the coordinates of the target is less than or equal to R _s , stop the precise matching.

A system for improving the positioning accuracy of a pan-tilt camera, comprising:

PTZ camera for rotating camera;

The battery module and the power module are used to supply power to the computing platform and the PTZ camera;

The computing platform is used to communicate with the PTZ camera and control the rotation of the PTZ camera, including:

An information receiving device for calling the network video stream sequence of the PTZ camera frame by frame and receiving the target image provided by the user;

A control device for controlling the pan-tilt camera to collect image information of all application scenarios and controlling the pan-tilt camera to rotate;

A display device for displaying application scenarios and allowing users to select target images;

A data processing device for extracting image features and stitching images, and obtaining the coordinate difference between the target image and the current image;

A data storage device for saving real-time extracted video frames as pictures.

The beneficial effects of the present invention are:

Aiming at the problem that the preset point function in the prior art needs to be manually configured and the aging of the camera shaft requires frequent replacement of the camera, the present invention proposes a system and a control method for improving the positioning accuracy of the pan-tilt camera. Construct a panoramic image of the scene you care about, and grasp the information of the entire scene more clearly; and based on this panoramic image, compare it with the target image provided by the user to the matching algorithm and the current camera monitoring image, the algorithm controls the camera to accurately and clearly view the scene from any position Turn to the target image position provided by the user to complete the monitoring of the region of interest. It solves the inconvenient and inaccurate problems of the traditional preset function, and makes the monitoring system more scientific.

In addition, the method of the present invention is controlled from the software level without modifying the hardware part of the traditional camera. Even if the rotation axis of the camera is inaccurate, the target can be precisely positioned through software operation, and there is no need to replace the camera equipment, so the hardware deployment can be greatly saved Cost, but also prolong the service life of the hardware camera. The invention allows users to balance the speed and precision of panorama construction and matching algorithms according to actual conditions, and has strong flexibility.

Instructions attached

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some of the present invention. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1 is a schematic diagram of a system architecture for improving the positioning accuracy of a pan-tilt camera applied in the present invention;

Fig. 2 is a work flow diagram of a system control method for improving the positioning accuracy of a pan-tilt camera provided by the present invention;

3 is a schematic diagram of the first stage of matching target image coordinates provided by the present invention;

4 is a schematic diagram of the second stage of matching target image coordinates provided by the present invention;

Fig. 5 is the result figure of testing different target images in the campus outdoor environment in the embodiment of the present invention;

FIG. 6 is a result diagram of testing different target images in an indoor campus environment in an embodiment of the present invention;

FIG. 7 is a diagram showing test results of different target images in an approximate factory indoor environment in an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Embodiment one

As shown in Figure 1, the present embodiment provides a system for improving the positioning accuracy of a pan-tilt camera, including: a pan-tilt camera, a computing platform, a power supply module and a battery; the pan-tilt camera is used for rotating camera; a battery module and a power supply module, It is used to supply power to the computing platform and the PTZ camera; the computing platform is used to communicate with the PTZ camera and control the rotation of the PTZ camera. The image information receiving device is used to control the pan-tilt camera to collect image information of all application scenes and the control device to control the pan-tilt camera to rotate. It is used to display the application scene and the display device for the user to select the target image. It is used to perform image characteristics A data processing device for extracting and image splicing, and obtaining the coordinate difference between the target image and the current image, and a data storage device for saving the video frames extracted in real time as pictures.

Embodiment two

On the basis of Embodiment 1, Embodiment 2 of the present application provides a work flow chart of the system control method for improving the positioning accuracy of the pan-tilt camera in Embodiment 1, as shown in FIG. 2 . Based on a commonly used and normally operating computing platform, and using the FLANN algorithm and the image perspective transformation method combined with the high-precision matching method of the PTZ camera; the PTZ camera continuously interacts with the environment, and transmits the network video stream to the computing platform, and the computing platform is displayed on the screen. On the display, the user selects the area of interest, and the image of the area is provided to the matching algorithm, and then the matching algorithm controls the camera to precisely turn to the area of interest of the user.

The control method of the present invention includes step 101 to step 104 . Please refer to the detailed introduction below for the specific process.

Step 101, image stitching to construct a panorama.

Specifically, the computing platform sends a control rotation command through the serial port of the pan-tilt camera, so that the camera rotates to the left (or to the right, depending on the specific working scene) for a fixed time window length T _q and then stops. T represents a fixed time period, and σ is a balance factor. In different scenarios, users can adjust σ to balance the speed and success rate of panorama image generation. The balance formula can be expressed as:

A=1-σ,T _q =σT (1)

Among them, A represents the splicing success rate. When σ=1, it means that the image acquisition speed is the fastest, but the relative success rate is also the lowest. When σ=0, it means that the image acquisition speed is the slowest, but the relative success rate is also the highest. Users can adjust the balance factor according to the target scene and specific needs.

After a rotation stops, the computing platform extracts the real-time video frame, and saves the picture on the computing platform. The camera continues this action until all required work scene information has been saved by the camera to the computing platform. Then, based on the Stitching method in OpenCV, the computing platform uses the saved scene information images to stitch together to form a panoramic picture.

Step 102, the user selects an area of interest.

Specifically, the computing platform provides the panoramic image to the user, and the user can set any area in the working scene as the area of interest in advance, and save the image of the area of interest as the target image, and then the user can Select a target image as the target to turn to.

Step 103 , rough search: the FLANN algorithm obtains the coordinates of the target and the current image, and controls the rotation of the camera according to the coordinate difference. The coordinate difference is calculated again, and if the threshold condition is not met, step 103 is repeated until the threshold condition is met.

x轴最小与y轴最大点

x轴最大与y轴最小点

x轴最大与y轴最大点

若宽度或长度大于原图大小，则舍弃x轴(或y轴)最边上两个匹配点，重新构造四个顶点，直到图片大小小于等于摄像机图片大小。最后根据四个顶点计算出匹配图像的中点

Specifically, after receiving the target image of the user, the computing platform matches the target image with the panoramic image based on the FLANN matching algorithm. FLANN returns the coordinates (X,Y) of the target point where all features match. First of all, considering that there will be many similar equipment or items in the industrial scene, FLANN can easily match two feature points that are far apart but have similar characteristics. At this time, compare two adjacent points, if the distance exceeds the length (or width) of the original image, then redefine the point as a mismatch point. Then, traverse all matching points to find four vertices, namely the minimum x-axis and y-axis minimum points

x-axis minimum and y-axis maximum points

x-axis maximum and y-axis minimum point

x-axis maximum and y-axis maximum point

If the width or length is greater than the size of the original image, discard the two matching points on the edge of the x-axis (or y-axis), and reconstruct four vertices until the image size is smaller than or equal to the camera image size. Finally, calculate the midpoint of the matching image based on the four vertices

与目标图像匹配中点

根据两点x轴的距离

以及y轴的距离

来决定摄像机的转动方向。 Further, in the search phase, a rough search is performed first. The computing platform reads the current video frame, compares the current frame and the target image with the panoramic image, and the current image matches the midpoint

Match the midpoint with the target image

According to the distance between two points on the x-axis

and the distance on the y-axis

To determine the direction of rotation of the camera.

The computing platform sends control commands through the serial port of the PTZ camera, so that the camera continues to rotate at the speed θ _c , and the rotation direction has the following 8 situations:

When Δx>R, -R≤Δy≤R, the pan-tilt camera rotates to the right at the speed θ _c ;

When Δx<R, -R≤Δy≤R, the pan-tilt camera rotates to the left at the speed θ _c ;

When -R≤Δx≤R, Δy>R, the pan-tilt camera rotates upward at the speed θ _c ;

When -R≤Δx≤R, Δy<R, the pan-tilt camera rotates downward at the speed θ _c ;

When Δx>R, Δy>R, the pan-tilt camera rotates to the upper right at the speed θ _c ;

When Δx>R, Δy≤-R, the PTZ camera rotates downward and rightward at the speed θ _c ;

When Δx≤-R, Δy>R, the pan-tilt camera rotates to the upper left at the speed θ _c ;

When Δx≤-R, Δy≤-R, the pan-tilt camera rotates to the lower left at the speed θ _c ;

Wherein, R>0 is one of the preset accuracy thresholds R _c and R _s .

At this stage, the rotation time of the camera each time is a fixed time window length T _c . After one rotation is completed, the computing platform reads the current video frame again, and recalculates the coordinate difference between the midpoint of the matching image and the target image. If the preset accuracy threshold is not met, the operation of this rough stage is repeated. Until the distance between the two matching midpoints is less than the set threshold R _c .

Step 104, precise search: extract feature points, perform image perspective transformation to obtain precise coordinate difference, and control camera rotation according to coordinate difference. The coordinate difference is calculated again, and if the threshold condition is not met, step 104 is repeated until the threshold condition is met.

Specifically, the current video frame is close to but not completely matched with the target image, and FLANN cannot complete the pixel-level precise matching function under such circumstances. The computing platform extracts the feature points of the two images. Since the images are close to each other and the matching points are close, the two images can be spliced in the same coordinate system by using the perspective change, so as to obtain the precise distance Δx _p , Δy _p . The computing platform sends control commands through the serial port of the PTZ camera, so that the camera continues to rotate according to the calculated precise distance and direction, and the rotation time is a fixed time window length T _s ; repeat this step until the distance between the two images is less than the set The matching method ends at the threshold R _s .

In the two-stage method of coarse search and fine search proposed in 103 and 104, the rotation time windows T _c and T _s of the two stages can be customized to achieve a balance between speed and accuracy.

In the rough search and fine search proposed in step 103 and step 104, the rotation time window is customized:

_Tc = αT (4)

T _s =βT (5)

Among them, α, β∈[0,1], the parameters can assist users to adjust the threshold flexibly. When α=1, β=0, the system pursues speed in the rough matching stage, and T _c reaches the maximum value, which means that each rotation takes a long time, the number of calculations is small, and the matching speed is fast, but problems such as oversteering may occur. The system pursues the accuracy rate in the precise matching stage, and T _s reaches the minimum value, which means that each rotation time is short, the number of calculations is large, and the matching speed is slow, but the matching accuracy rate is high and the failure rate is low. Among them, it needs special attention that when α=0, β=0, T _c and T _c will not directly drop to 0, at this time T _c =T _c =T _min , where T _min is the set minimum rotation time window.

与当前图像的中点

并且 进一步计算出中点之间的差值。如果该差值的横纵坐标大于阈值R _c，则需要对摄像机进行调整，方法是当计算平台正常时，通过串口向摄像机发送旋转指令，使其向目标区域旋转，默认系统工作在高精度设置，即α＝0，β＝0；当差值小于阈值R _c时，计算平台通过串口向摄像机发送停止指令；若差值开始便没有超出阈值R _c，则摄像机静止。 As shown in Figure 3, the FLANN algorithm is used to match the target image with the current image in the first stage, and the midpoint of the target image is calculated

Midpoint with the current image

And further calculate the difference between the midpoints. If the horizontal and vertical coordinates of the difference are greater than the threshold R _c , you need to adjust the camera. The method is to send a rotation command to the camera through the serial port when the computing platform is normal, so that it rotates to the target area. The default system works at high-precision settings , ie α=0, β=0; when the difference is less than the threshold R _c , the computing platform sends a stop command to the camera through the serial port; if the difference does not exceed the threshold R _c from the beginning, the camera stops.

As shown in Figure 4, after the first stage of matching is completed, the current image is close to the target image but not exactly matched. In the second stage, the matching feature points obtained by FLANN are used to construct image perspective transformation to obtain two images The precise distance between Δx _p , Δy _p . The method of rotation is the same as the first stage. Finally, the camera is turned to the target image, and the error does not exceed the threshold R _s .

As shown in Fig. 5, Fig. 6 and Fig. 7, one hundred tests were carried out in the indoor environment of the campus environment in normal weather, the outdoor environment of the campus and the similar indoor environment of the factory, and the target image was changed after every ten tests. The test success rates were respectively It is 99%, 100% and 100%, which fully demonstrates that the method proposed by the present invention can perform accurate matching under various situations and different target images, and can greatly improve the working efficiency of monitoring personnel.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other.

In this paper, specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; meanwhile, for those of ordinary skill in the art, according to the present invention Thoughts, there will be changes in specific implementation methods and application ranges. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims (8)

A control method for improving the positioning accuracy of a pan-tilt camera, characterized in that it comprises the steps: Step 101, first perform initialization: both the computing platform and the PTZ camera are powered on, and after waiting for the self-test of the computing platform and the PTZ camera to complete, the computing platform calls the network video stream sequence of the PTZ camera frame by frame based on the ffmpeg library; then calculates The platform controls the PTZ camera to collect image information of all application scenarios, and constructs a panorama of the application scenarios based on the image stitching method; Step 102, the display screen of the computing platform displays a panorama of the application scene, the user selects an area of interest in the panorama of the application scene, and saves the image of the area of interest as a target image, and the user selects an area of interest in a plurality of target images Select a target image as the target to be turned; Step 103, rough search: the calculation platform receives the target image provided by the user, the FLANN matching algorithm obtains the coordinates of the target image and the current image, and calculates the coordinate difference between the target image and the current image; if the coordinate difference does not meet the threshold condition, then according to the coordinate The difference determines the rotation direction and rotation distance of the PTZ camera, and controls the PTZ camera to rotate according to the obtained rotation direction and rotation distance; if the coordinate difference satisfies the threshold condition, the rotation is not performed; repeat step 103 until the obtained target image The coordinate difference with the current image satisfies the threshold condition; Step 104. Precise search: the computing platform selects two images that are close to each other and the matching point of the target image, and extracts feature points; use the perspective transformation method to splice the two images in the same coordinate system to obtain an accurate horizontal The coordinate difference Δx _p and the precise ordinate difference Δy _p ; if the abscissa difference and the ordinate difference meet the threshold condition, the computing platform sends a control command through the serial port of the pan-tilt camera, and the pan-tilt camera The distance and direction continue to rotate, and the rotation time is a fixed time window length T _s ; if the difference in abscissa and ordinate does not meet the threshold condition, repeat step 104 until the distance between the two images is less than the set threshold R _s , the image matching is ended; the precise positioning and output of the latitude and longitude of the location of the target to be identified are used for the control circuit to rotate the camera to the target area selected by the user. The control method for improving the positioning accuracy of the pan-tilt camera according to claim 1, wherein the pan-tilt camera is a camera adapted to a network video protocol; the computing platform and the pan-tilt camera are wired or wirelessly communication. According to the control method for improving the positioning accuracy of the pan-tilt camera according to claim 1, it is characterized in that, the specific manner in which the computing platform controls the pan-tilt camera to collect the image information of all application scenarios is: The computing platform sends control rotation commands through the serial port of the PTZ camera, and the camera turns left or right for a fixed time window length T _q and then stops; in different scenarios, the user adjusts the balance factor σ to balance the speed of panorama image generation and The success rate, the balance formula is: A=1-σ,T _q =σT (1) Among them, A represents the splicing success rate, T represents a fixed time period, and T _q is the fixed time window length; when σ=1, the image acquisition speed is the fastest and the success rate is the lowest; when σ=0, the image acquisition The slowest speed, the highest success rate; After a rotation stops, the computing platform extracts the real-time video frame and saves the video frame as a picture; the camera continues to extract the video frame and save it as a picture until all the working scene information is saved to the computing platform by the camera. According to the control method for improving the positioning accuracy of the pan-tilt camera according to claim 3, it is characterized in that the image stitching method is based on the Stitching method in OpenCV, and the computing platform uses the stored scene information images to stitch to form a panorama of the application scene . According to the control method of improving the positioning accuracy of the pan-tilt camera according to claim 4, it is characterized in that step 103 specifically comprises the following steps: Step 103-1. After receiving the target image provided by the user, the computing platform matches the target image with the panoramic image based on the FLANN matching algorithm, and the FLANN matching algorithm returns the coordinates (X, Y) of the target point matching all the features; comparison The coordinates of two adjacent target points, if the distance between the coordinates of two adjacent target points exceeds the length or width of the original image, then redefine the target point as a mismatch point; Step 103-2. Traverse all matching points and find out the four vertices in the area where the matching points are located: the minimum point on the x-axis and the minimum point on the y-axis

x-axis minimum and y-axis maximum points

x-axis maximum and y-axis minimum point

x-axis maximum and y-axis maximum point

If the width or length of the area where the matching point is located is greater than the size of the original image, discard the two matching points on the extreme side of the x-axis or y-axis, and reconstruct four vertices until the size of the image is smaller than or equal to the size of the image obtained by the PTZ camera; for x-axis, if max(X)-min(X) is greater than the width of the original image, then discard all coordinates containing max(X), min(X) in the x-axis coordinates in the matched target point coordinates (X,Y) ;For the y-axis, if max(Y)-min(Y) is greater than the length of the original image, then discard the coordinates of the y-axis that contain max(Y), min(Y) in the matched target point coordinates (X,Y) all coordinates; Finally, calculate the midpoint of the matching image based on the four vertices

Step 103-3. The computing platform reads the current video frame, compares the current frame and the target image with the panorama of the application scene, and obtains the matching midpoint of the current image

Match the midpoint with the target image

Step 103-4, according to the distance between the current image matching midpoint and the target image matching midpoint in the x-axis direction

and the distance in the y-axis direction

to determine the rotation direction of the camera; the computing platform sends control commands through the serial port of the pan-tilt camera, so that the pan-tilt camera continues to rotate at a speed θ _c , and the rotation time of the camera is a fixed time window length T _c each time; after one rotation is completed , the computing platform reads the current video frame again, recalculates the coordinate difference between the midpoint of the matching image and the target image, and if the coordinate difference is greater than or equal to the preset accuracy threshold R _c , repeat steps 103-1 to Step 103-4, until the distance between the two matching midpoints is less than the set threshold R _c . According to the control method for improving the positioning accuracy of the pan-tilt camera according to claim 5, it is characterized in that, in the rough search and the fine search proposed in step 103 and step 104, the custom setting rotation time window: _Tc = αT (4) T _s =βT (5) Among them, α, β∈[0,1] are used to assist users to adjust the threshold value flexibly; when α=1, β=0, it reaches the maximum value, each rotation takes a long time, the number of calculations is small, and the matching speed is fast; When α=0, β=0, T _c =T _s =T _min , where T _min is the set minimum rotation time window. According to the control method for improving the positioning accuracy of the pan-tilt camera according to claim 5, it is characterized in that, in step 103 and step 104, the computing platform controls the pan-tilt camera to rotate according to the obtained rotation direction and the rotation distance, and the rotation direction has the following 8 situations: When Δx>R, -R≤Δy≤R, the pan-tilt camera rotates to the right at the speed θ _c ; When Δx<R, -R≤Δy≤R, the pan-tilt camera rotates to the left at the speed θ _c ; When -R≤Δx≤R, Δy>R, the pan-tilt camera rotates upward at the speed θ _c ; When -R≤Δx≤R, Δy<R, the pan-tilt camera rotates downward at the speed θ _c ; When Δx>R, Δy>R, the pan-tilt camera rotates to the upper right at the speed θ _c ; When Δx>R, Δy≤-R, the PTZ camera rotates downward and rightward at the speed θ _c ; When Δx≤-R, Δy>R, the pan-tilt camera rotates to the upper left at the speed θ _c ; When Δx≤-R, Δy≤-R, the pan-tilt camera rotates to the lower left at the speed θ _c ; Among them, R>0, R is the preset accuracy threshold R _c or R _s ; in rough search, when the difference between the camera coordinates and the target coordinates is less than or equal to R _c , stop the rough search; in exact match, when When the difference between the coordinates of the camera and the coordinates of the target is less than or equal to R _s , stop the precise matching. A system for improving the positioning accuracy of the pan-tilt camera according to the control method according to claim 1, characterized in that it comprises: PTZ camera for rotating camera; The battery module and the power module are used to supply power to the computing platform and the PTZ camera; The computing platform is used to communicate with the PTZ camera and control the rotation of the PTZ camera, including: An information receiving device for calling the network video stream sequence of the PTZ camera frame by frame and receiving the target image provided by the user; A control device for controlling the pan-tilt camera to collect image information of all application scenarios and controlling the pan-tilt camera to rotate; A display device for displaying application scenarios and allowing users to select target images; A data processing device for extracting image features and stitching images, and obtaining the coordinate difference between the target image and the current image; A data storage device for saving real-time extracted video frames as pictures.

Images