CN116977677A - Image feature point matching screening method, device, equipment and medium based on clustering - Google Patents

Abstract

The present invention relates to the technical field of image processing, and in particular to a clustering-based image feature point matching and screening method, device, equipment and medium. The method includes extracting a first set of feature points in a first image and a third set of feature points in a second image. Two feature point sets, perform feature point matching on the first feature point set and the second feature point set, and obtain N sets of feature point matching pairs. Randomly select from the N sets of feature point matching pairs to obtain M temporary matching pairs. According to the temporary The matching pairs are calculated to obtain the temporary rotation matrix and the temporary translation vector. M temporary rotation matrices and M temporary translation vectors are clustered respectively to obtain the reference rotation matrix and reference translation vector. The projection error of each set of feature point matching pairs is calculated, which will satisfy The feature point matching pairs with preset conditions are determined as the target point matching pairs, and the feature point matching pairs are filtered through the clustering results of the transformation parameters, thereby improving the accuracy and precision of feature point matching.

Description

Image feature point matching and screening methods, devices, equipment and media based on clustering

Technical field

The present invention relates to the technical field of image processing, and in particular to a clustering-based image feature point matching and screening method, device, equipment and medium.

Background technique

At present, image matching technology has been widely used in various tasks such as image splicing, three-dimensional reconstruction, positioning and navigation, image retrieval, etc. to identify and align the content or structure with the same or similar attributes in two images on a pixel basis. Matching technology is based on image feature point extraction technology. Existing image feature point extraction technology usually uses operator-based algorithms such as Scale-invariant feature transform (SIFT) algorithm, ORB (OrientedFast and Rotated Brief) algorithm. extraction method, or use an extraction method based on a deep learning model.

However, under the requirement of high-precision image matching, it is necessary to obtain accurate feature point matching relationships between two images, and none of the above methods can ensure that the feature point matching is accurate enough, that is, the accuracy of image feature point matching is poor. Therefore, , how to improve the accuracy of image feature point matching, and then improve the accuracy of image matching, has become an urgent problem to be solved.

Contents of the invention

In view of this, embodiments of the present invention provide a clustering-based image feature point matching and screening method, device, equipment and medium to solve the problem of poor accuracy in image feature point matching.

In a first aspect, embodiments of the present invention provide a clustering-based image feature point matching and screening method. The image feature point matching and screening method includes:

Obtain the first image and the second image to be matched, extract feature points in the first image to obtain a first feature point set, extract feature points in the second image to obtain a second feature point set;

Feature point matching is performed on the first feature point set and the second feature point set to obtain N sets of feature point matching pairs, where the feature point matching pairs include one feature point in the first feature point set and all A feature point in the second feature point set, N is an integer greater than five;

M groups of temporary matching pairs are randomly selected from the N groups of feature point matching pairs, and based on each group of temporary matching pairs, the temporary rotation between the first image and the second image in the corresponding temporary matching pair is calculated. Matrix and temporary translation vector, get M temporary rotation matrices and M temporary translation vectors, M is an integer less than or equal to N;

The M temporary rotation matrices are clustered to obtain the first cluster center point, the first cluster center point is determined as the reference rotation matrix, and the M temporary translation vectors are clustered to obtain the second clustering center point. Class center point, determine the second cluster center point as the reference translation vector;

According to the reference rotation matrix and the reference translation vector, the projection error of each set of feature point matching pairs is calculated, and the feature point matching pairs corresponding to the projection errors within the preset range are determined as target point matching pairs. The matching pair is used to calculate a target rotation matrix and a target translation vector between the first image and the second image to indicate image matching between the first image and the second image.

In a second aspect, embodiments of the present invention provide an image feature point matching and screening device based on clustering. The image feature point matching and screening device includes:

A feature point extraction module is used to obtain the first image and the second image to be matched, extract the feature points in the first image to obtain the first feature point set, extract the feature points in the second image to obtain the third Two feature point sets;

A feature point matching module, configured to perform feature point matching on the first feature point set and the second feature point set to obtain N sets of feature point matching pairs, where the feature point matching pairs include the first feature point set One feature point in and one feature point in the second feature point set, N is an integer greater than five;

A matching pair selection module, configured to randomly select M groups of temporary matching pairs from the N groups of feature point matching pairs, and calculate the first image and the first image in the corresponding temporary matching pair according to each group of temporary matching pairs. The temporary rotation matrix and temporary translation vector between the two images are obtained, and M temporary rotation matrices and M temporary translation vectors are obtained. M is an integer less than or equal to N;

A clustering processing module is used to cluster the M temporary rotation matrices to obtain a first cluster center point, determine the first cluster center point as a reference rotation matrix, and perform processing on the M temporary translation vectors. Cluster, obtain the second cluster center point, and determine the second cluster center point as the reference translation vector;

A matching pair determination module, configured to calculate the projection error of each set of feature point matching pairs according to the reference rotation matrix and the reference translation vector, and determine the feature point matching pair corresponding to the projection error within the preset range as the target point Matching pair, the target point matching pair is used to calculate the target rotation matrix and target translation vector between the first image and the second image to instruct the first image and the second image to perform image processing. match.

In a third aspect, embodiments of the present invention provide a computer device. The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor executes the The computer program implements the image feature point matching and filtering method described in the first aspect.

In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, the image feature point matching as described in the first aspect is implemented. Screening methods.

Compared with the prior art, the beneficial effects of the embodiments of the present invention are:

Obtain the first image and the second image to be matched, extract the feature points in the first image to obtain the first feature point set, extract the feature points in the second image to obtain the second feature point set, and compare the first feature point set Perform feature point matching with the second set of feature points to obtain N sets of feature point matching pairs. The initial selection number is zero. M sets of temporary matching pairs are randomly selected from the N sets of feature point matching pairs. According to each set of temporary matching pairs, , calculate the temporary rotation matrix and temporary translation vector between the first image and the second image in the corresponding temporary matching pair, obtain M temporary rotation matrices and M temporary translation vectors, and combine the M temporary rotation matrices Perform clustering to obtain the first cluster center point, determine the first cluster center point as the reference rotation matrix, cluster the M temporary translation vectors, obtain the second cluster center point, and determine the second cluster center point as Reference translation vector, according to the reference rotation matrix and reference translation vector, calculate the projection error of each set of feature point matching pairs, and determine the feature point matching pair corresponding to the projection error within the preset range as the target point matching pair, which is obtained through clustering The reference rotation matrix and reference translation matrix filter the feature point matching pairs, thereby retaining reliable target point matching pairs, improving the precision and accuracy of feature point matching, and thus improving the accuracy of image matching.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings in the following description are only illustrative of the present invention. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic diagram of an application environment of a clustering-based image feature point matching and screening method provided in Embodiment 1 of the present invention;

Figure 2 is a schematic flow chart of a clustering-based image feature point matching and screening method provided in Embodiment 1 of the present invention;

Figure 3 is a schematic structural diagram of a clustering-based image feature point matching and filtering device provided in Embodiment 2 of the present invention;

Figure 4 is a schematic structural diagram of a computer device provided in Embodiment 3 of the present invention.

Detailed ways

In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the present invention in unnecessary detail.

It will be understood that, when used in the description of the present invention and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and/or components but does not exclude one or more other The presence or addition of features, integers, steps, operations, elements, components and/or collections thereof.

It will also be understood that the term "and/or" as used in the specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in the description of the present invention and the appended claims, the term "if" may be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. ". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, to mean "once determined" or "in response to a determination" or "once the [described condition or event] is detected ]" or "in response to detection of [the described condition or event]".

In addition, in the description of the present specification and appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description and shall not be understood as indicating or implying relative importance.

Reference in the specification of the invention to "one embodiment" or "some embodiments" or the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the invention. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments", etc. appearing in different places in this specification are not necessarily References are made to the same embodiment, but rather to "one or more but not all embodiments" unless specifically stated otherwise. The terms "including", "includes", "having" and their variations are all used to mean "including but not limited to" unless otherwise specifically emphasized in the description of the present invention.

It should be understood that the sequence number of each step in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

In order to illustrate the technical solution of the present invention, specific examples will be described below.

The clustering-based image feature point matching and filtering method provided in Embodiment 1 of the present invention can be applied in the application environment as shown in Figure 1, where the client communicates with the server. Among them, clients include but are not limited to handheld computers, desktop computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, cloud terminal devices, personal digital assistants (personal digital assistants, PDAs), etc. Computer equipment. The server can be an independent server, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and content delivery networks (Content Delivery Network). , CDN), as well as cloud servers for basic cloud computing services such as big data and artificial intelligence platforms.

The client can connect to at least one image collection terminal to obtain the collected images to be matched from the image collection terminal. The image collection terminal includes but is not limited to cameras, video cameras, video recorders, handheld photography equipment, mobile devices with photography functions, etc. . Since this embodiment involves image matching technology, when a single image collection terminal is used, the posture of the image collection terminal can be moved, that is, the first image and the second image are collected by a single image collection terminal in different postures. For example, mobile devices with photography functions used by humans, cameras deployed on movable platforms such as moving tracks, etc., when multiple image collection terminals are used, the images collected by the two image collection terminals are used as the third image collection terminal. one image and a second image. The client can be used in various application scenarios such as 3D modeling, intelligent navigation, and pedestrian recognition to collect images to be matched through the connected image acquisition terminal. Taking the intelligent navigation scenario as an example, the image acquisition terminal can refer to the vehicle-mounted Using visual sensors, the image acquisition terminal can be used to collect road image information and transmit it to the client. The client transmits the collected road image information to be matched to the server in the vehicle based on high computing power chips for image matching processing. , thereby realizing specific application functions such as road scene modeling, lane line recognition, and pedestrian recognition.

Refer to Figure 2, which is a schematic flow chart of a clustering-based image feature point matching and screening method provided in Embodiment 1 of the present invention. The above image feature point matching and screening method can be applied to the server in Figure 1, and the computer corresponding to the server The device is connected to the client to obtain the first image and the second image to be matched. The computer device corresponding to the server can perform computing tasks such as projection matrix operations and matrix decomposition. As shown in Figure 2, the image feature points are matched and filtered. Methods can include the following steps:

Step S201: Obtain the first image and the second image to be matched, extract feature points in the first image to obtain a first set of feature points, and extract feature points in the second image to obtain a second set of feature points.

Among them, the first image and the second image may refer to the two images to be matched. Due to the characteristics of the image matching task, there should be the same image information between the first image and the second image. Taking scene modeling as an example, The first image and the second image may be images of the scene area to be modeled collected at different orientations, and there are reference objects in the scene area to be modeled that simultaneously exist in the first image and the second image, that is, the first image There is partial overlap with the image information of the second image.

Feature points can refer to points in the image that have distinctive characteristics and can effectively reflect the essential characteristics of the image, or that can identify the target object in the image. Feature points can usually include corner points, edge points, edge end points, extreme points, etc., usually In this case, the feature point has characteristics that are different from other points in the image. For example, the grayscale change around the feature point is more drastic than that of other points, the corresponding curvature change of the feature point is more drastic than that of other points, etc.

The first feature point set includes at least five feature points extracted from the first image, and the second feature point set includes at least five feature points extracted from the second image.

Specifically, since when the internal parameters of the image acquisition device are known, based on the epipolar constraint principle, at least five pairs of matching points are needed to restore the transformation information between the image acquisition devices, so the first feature point set and the second feature point set are limited Each set should contain at least five feature points respectively. The transformation information between the image acquisition devices can refer to the pose difference caused by the movement of a single image capture device, or it can refer to the deployment pose between two image capture devices. difference.

The internal parameters of the image acquisition device may include pixel focal length and optical center coordinate information. The origin of the pixel coordinate system is usually not consistent with the zero point of the image acquisition device. Therefore, the offset from the optical center to the zero point of the image acquisition device is recorded as c _x , _cy , that is, The optical center is offset on the horizontal axis and the vertical axis on the pixel coordinate system, and c _x and c _y are used as the optical center coordinate information.

Correspondingly, the relationship between a point (x, y) in the image coordinate system and a pixel point (u, v) in the pixel coordinate system can be expressed as, Among them, a point (x, y) in the image coordinate system is converted from a point (X, Y, Z) in the camera coordinate system of the image acquisition device. The conversion method is: k and l are the dimensions of each pixel in the horizontal and vertical directions respectively, let Taking f _x and f _y as the pixel focal length, the internal parameters of the image acquisition device can be represented by the internal parameter matrix K, which can be expressed as/>

It should be noted that if the number of feature points contained in the extracted first feature point set or the second feature point set is less than five, you need to reselect the first image and the second image for image matching, or re-select the first image and the second image for image matching. The feature point extraction method is selected until the condition that the first feature point set and the second feature point set each contain at least five feature points is met.

Optionally, extract feature points in the first image to obtain a first set of feature points, and extract feature points in the second image to obtain a second set of feature points, including:

Use the feature point extraction operator to search within the first image to obtain Q first feature points and their corresponding first feature descriptors in the first image, where Q is an integer greater than N;

Use the feature point extraction operator to search within the second image to obtain S second feature points and their corresponding second feature descriptors in the second image, where S is an integer greater than N;

The Q first feature points in the first image form a first feature point set of the first image, and the S second feature points in the second image form a second feature point set of the second image.

Among them, the feature point extraction operator can be used to extract feature points in the image. In this embodiment, the feature point extraction operator can use the Harris operator. The implementer can also use SIFT, Speeded-Up Robust Features , SURF), Features from accelerated segment test (FAST), ORB and other feature extraction methods for feature point extraction. There is no restriction on the selection method of the feature point extraction operator. The implementer can choose any feature point extraction method. Operators used to extract feature points in images are all within the scope of the present invention.

The first feature point may refer to the feature point extracted in the first image, the first feature descriptor may refer to the description information corresponding to the first feature point and its neighboring pixels, and the second feature point may refer to the feature point in the second image. The extracted feature point and the second feature descriptor may refer to the description information corresponding to the second feature point and its neighboring pixels.

Specifically, since N groups of feature point matching pairs are subsequently extracted, each feature point set should contain at least N feature points, and the number of feature points extracted from different feature point sets may not be consistent, so there is no need to limit Q and S is the same, just ensure that both Q and S are greater than N.

In this embodiment, feature points and feature descriptors in the image are extracted, thereby extracting the accuracy of subsequent feature point matching, while ensuring that subsequent matching-based feature point pairs can normally perform the screening process, and improving the overall image feature point matching and screening process. s efficiency.

The above steps of obtaining the first image and the second image to be matched, extracting the feature points in the first image to obtain the first feature point set, extracting the feature points in the second image to obtain the second feature point set are the subsequent features. Point matching provides a foundation so that the feature point matching and screening process can be executed normally, thereby improving the efficiency of the feature point matching and screening process.

Step S202: Perform feature point matching on the first feature point set and the second feature point set to obtain N sets of feature point matching pairs.

Wherein, the feature point matching pair includes one feature point in the first feature point set and one feature point in the second feature point set, and N is an integer greater than five.

Optionally, feature point matching is performed on the first feature point set and the second feature point set to obtain N sets of feature point matching pairs, including:

For any first feature point in the first feature point set, similarity is calculated between the first feature descriptor of the first feature point and the second feature descriptor of each second feature point in the second feature point set. Calculate to obtain the similarity value corresponding to each second feature point in the second feature point set, and determine that the second feature point corresponding to the maximum value among all similarity values and the first feature point form a set of feature point matching pairs;

Traverse all the first feature points in the first feature point set to obtain N sets of matching pairs of feature points.

The similarity value may represent the degree of difference between the first feature descriptor of the targeted first feature point and the second feature descriptor corresponding to the second feature point.

Specifically, the feature descriptors can be represented by vectors, and the similarity between the feature descriptors is easy to calculate. The calculation method can use distance measurement methods such as Euclidean distance, Hamming distance, and cosine similarity.

Taking cosine similarity as an example, the value range of cosine similarity is [-1,1]. The closer the calculated cosine similarity is to -1, the greater the difference between the feature descriptors used for calculation. That is, the less similar the corresponding feature points are, the closer the calculated cosine similarity is to 1, which means the smaller the difference between the feature descriptors used for calculation, that is, the more similar the corresponding feature points are.

The above-mentioned method of determining that the second feature point corresponding to the maximum value among all similarity values and the first feature point form a set of feature point matching pairs may refer to a brute force matching method. When the implementer actually performs feature point matching, the method may also be The cross matching method, K-Nearest Neighbor (KNN) matching method, Random SampleConsensus (RANSAC) matching method, etc. are used.

Among them, the cross-matching method may refer to two matchings. The first matching is based on a first feature point to obtain a match between the first feature point and a second feature point. The second matching is based on the second feature point. If If the second feature point and the first feature point are still matched, it is determined that the first feature point and the second feature point are correctly matched.

The KNN matching method may refer to selecting several second feature points that are most similar to the targeted first feature point. If the difference between these second feature points is large, the most similar second feature point is considered to be the first Correct matching of feature points.

The RANSAC matching method can refer to randomly selecting several feature point matching pairs for projection matrix calculation, and using the calculated projection matrix to calculate the projection error of each pair of feature point matching pairs in the image, looking for the projection matrix that minimizes the global projection error, and determining The feature point matching pairs corresponding to the projection matrix with the smallest global projection error are correct matching pairs. However, RANSAC usually needs to be repeated multiple times to ensure that the filtered matching pairs have a high accuracy, and it cannot guarantee that the filtered matching pairs will be correct. The matching pair is exactly right.

It should be noted that the selection method of the feature point matching method is not limited here. Any feature point matching method selected by the implementer to perform image feature point matching processing is within the protection scope of the present invention.

In this embodiment, the feature points are characterized by feature descriptors, and then the similarity between the feature points is calculated. The feature points are matched through the feature point matching algorithm, thereby obtaining the initial feature point matching pair, which is convenient for subsequent feature matching. It provides a basis for the screening of point matching pairs, which is helpful to improve the efficiency of the feature point matching and screening process.

The above-mentioned steps of performing feature point matching on the first feature point set and the second feature point set to obtain N sets of feature point matching pairs are performed. Feature point matching is performed through the feature point matching algorithm to obtain initial feature point matching pairs, which is convenient for subsequent pairs. It provides a basis for screening feature point matching pairs, which is beneficial to improving the efficiency of the feature point matching and screening process.

Step S203, randomly select M groups of temporary matching pairs from N groups of feature point matching pairs, and calculate the temporary rotation matrix and temporary rotation matrix between the first image and the second image in the corresponding temporary matching pair according to each group of temporary matching pairs. Translate the vector to get M temporary rotation matrices and M temporary translation vectors.

Among them, the temporary matching pair can refer to the matching pair of feature points used to calculate the rotation matrix and the translation vector. The temporary rotation matrix and the temporary translation vector can refer to the camera external parameter information calculated by the temporary matching pair. M is less than or equal to N. is an integer, and there is a corresponding relationship between a temporary rotation matrix calculated from a set of temporary matching pairs and a temporary translation vector.

Specifically, in this embodiment, the initial number of selections is zero, and random selection is made from N groups of feature point matching pairs to obtain temporary matching pairs corresponding to the number of selections. Random selection may refer to random sampling with equal probability. Based on the temporary matching pairs The temporary rotation matrix and temporary translation vector between the first image and the second image are calculated, that is, the temporary rotation matrix and the temporary translation vector corresponding to the number of selections, and the number of selections is increased by one to facilitate subsequent iterative execution processes.

In this embodiment, the value of M can be set to 50, that is, 50 temporary rotation matrices and 50 temporary translation vectors are obtained. It should be noted that in order to avoid randomly selecting the same feature point matching point as the temporary Matching pairs, therefore, can be implemented by random sampling without replacement when selecting randomly multiple times.

It should be noted that the process of randomly selecting M sets of temporary matching pairs from N sets of feature point matching pairs is not limited to the above implementation process. The implementer can also choose other random selection methods such as probability sampling and random undersampling. The implementer chooses Any random sampling method for selecting temporary matching pairs is within the scope of the present invention.

Optionally, according to each group of temporary matching pairs, calculate the temporary rotation matrix and temporary translation vector between the first image and the second image in the corresponding temporary matching pair, including:

For any temporary matching pair, calculate the essential matrix between the first image and the second image within the temporary matching pair according to the epipolar constraint and the temporary matching pair;

Perform matrix decomposition on the essential matrix to obtain a temporary rotation matrix and a temporary translation vector between the first image and the second image.

Among them, the epipolar constraint can be used to determine the depth information of a point in the world coordinate system, that is, Z in the above example (X, Y, Z). Assume that the two points included in the temporary matching pair are p and p′, Then the epipolar constraint can be expressed as p′ ^T Fp=0, where F can refer to the basic matrix, and the relationship between the essential matrix E and the basic matrix F can be expressed as F=K ^-T EK ^-1 , where K can be Refers to the internal parameter matrix of the camera. The relationship between the essential matrix E, the rotation matrix R and the translation vector t can be expressed as E=t^R. Matrix decomposition can refer to singular value decomposition.

In this embodiment, the temporary rotation matrix and temporary translation vector of each temporary matching pair are calculated through epipolar constraints. Compared with the conventional method, the calculation amount is smaller and the calculation is simpler, thereby improving the efficiency of the overall image matching feature point screening process. efficiency.

The above-mentioned M groups of temporary matching pairs are randomly selected from N groups of feature point matching pairs. According to each group of temporary matching pairs, the temporary rotation matrix and temporary translation vector between the first image and the second image in the corresponding temporary matching pair are calculated. , obtain M temporary rotation matrices and M temporary translation vector steps, obtain temporary matching pairs through iterative execution, and calculate based on this to obtain the temporary rotation matrix and temporary translation vector corresponding to the temporary matching pairs, which provides the basis for subsequent clustering processing. Improved the accuracy of overall image feature point matching and screening.

Step S204, cluster the M temporary rotation matrices to obtain the first cluster center point, determine the first cluster center point as the reference rotation matrix, cluster the M temporary translation vectors, and obtain the second cluster center point. , determine the second cluster center point as the reference translation vector.

Wherein, the first clustering center point may refer to the center point of the clustering result of M temporary rotation matrices, and the second clustering center point may refer to the center of the clustering result of M temporary translation vectors. Points, reference rotation matrices and reference translation vectors may be used as reference external parameter information for subsequent screening of matching pairs of feature points.

Optionally, cluster the M temporary rotation matrices to obtain the first cluster center point, and determine the first cluster center point as the reference rotation matrix, including:

Convert each temporary rotation matrix into a temporary rotation vector, cluster all temporary rotation vectors according to the Euclidean distance between temporary rotation vectors, obtain the first cluster center point, and determine the first cluster center point as the reference rotation vector ;

Convert the reference rotation vector to the reference rotation matrix.

Among them, each temporary rotation matrix can be converted into a temporary rotation vector according to the conversion relationship between Lie groups and Lie algebras, and the reference rotation vector can be converted into a reference rotation matrix according to the conversion relationship between Lie groups and Lie algebras.

The conversion relationship between Lie groups and Lie algebras can guide the conversion of temporary rotation matrices into corresponding elements of Lie algebras, and Lie algebras can be represented by three-dimensional vectors, that is, the matrix representation is converted into a three-dimensional vector representation, so that it can be performed in a three-dimensional Euclidean space Clustering processing.

Euclidean distance can represent the difference information between temporary rotation vectors, and clustering can use the DBSCAN clustering algorithm.

Specifically, the DBSCAN clustering algorithm needs to define the sample point radius and the minimum number of points. For all points to be clustered, the local range of each point is determined based on the sample point radius. If the local range of any point contains the number of other points is greater than or equal to the minimum number of points, then this point is taken as the core point, and a temporary cluster is formed with this core point and other points included in its local range. For any temporary cluster, if the temporary cluster also contains other core points, then The temporary clusters corresponding to each core point are merged into the same cluster, and all temporary clusters are traversed to obtain the final clustering result, that is, multiple merged clusters are used as multiple cluster sets. It should be noted that the DBSCAN clustering algorithm is Density-based clustering method does not need to set the number of cluster sets in advance.

It should be noted that the implementation of distance measurement and clustering algorithms is not limited to the above process. The implementer can also choose other distance measurement methods such as cosine distance, and other clustering methods such as K-means. The implementer can choose any distance Obtaining the reference rotation vector using a measurement method or a clustering method is within the scope of the present invention.

In this embodiment, the temporary rotation matrix is converted into a three-dimensional vector for clustering processing, which makes the clustering process fast and efficient, improves the feasibility of cluster analysis, and provides a reference rotation matrix determined based on the clustering results for subsequent use. Feature point matching is the basis for filtering.

Optionally, cluster the M temporary translation vectors to obtain the second cluster center point, and determine the second cluster center point as the reference translation vector, including:

Map each temporary translation vector to a temporary translation point on the unit sphere, cluster all temporary translation points according to the spherical distance between the temporary translation points, and obtain the second cluster center point;

Determine the second cluster center point as the reference translation vector.

Among them, the unit sphere may refer to a sphere with a radius of 1, the temporary translation point may refer to the mapping point of the temporary translation vector on the unit sphere, and the spherical distance may represent the distance of the temporary translation point on the sphere.

Specifically, although the translation vector is already a three-dimensional vector, it is obtained through essential matrix decomposition and lacks scale information. Normally, the modulus length of the translation vector is always 1, so it can be regarded as a point on the unit sphere. The clustering method also uses the DBSCAN clustering algorithm. It should be noted that when determining the second cluster center point, it needs to be normalized to ensure that its module length is still 1, that is, it is still on the unit sphere.

In this embodiment, the temporary translation vector is converted to the unit sphere for clustering processing, so that the clustering process is more in line with the data characteristics of the temporary translation vector, improves the accuracy of cluster analysis, and provides subsequent clustering results based on The determined reference translation vector serves as the basis for feature point matching and screening, improving the accuracy of the overall image feature point matching and screening process.

As above, the M temporary rotation matrices are clustered to obtain the first cluster center point. The first cluster center point is determined as the reference rotation matrix. The M temporary translation vectors are clustered to obtain the second cluster center point. Determine The second clustering center point is the reference translation vector. Based on the correct feature point matching pair, the calculated rotation matrix and the translation vector are the same a priori information. The reference rotation matrix and the reference translation vector are calculated through clustering, which are the subsequent features. Point matching provides a basis for screening and improves the efficiency and accuracy of feature point matching for screening.

Step S205: Calculate the projection error of each set of feature point matching pairs based on the reference rotation matrix and the reference translation vector, and determine the feature point matching pair corresponding to the projection error within the preset range as the target point matching pair.

Among them, the projection error can refer to the pixel position deviation when the feature point matching pair is projected according to the reference rotation matrix and the reference translation vector. The preset range can be used to filter the feature point matching pairs according to the projection error, and the target point matching pair is used to calculate A target rotation matrix and a target translation vector between the first image and the second image to indicate image matching between the first image and the second image.

Optionally, the feature point matching pairs corresponding to the projection error within the preset range are determined as the target point matching pairs, including:

Calculate the mean and variance of all projection errors. According to the mean and variance, multiply the variance and the preset coefficient to obtain the multiplication result. Use the difference between the mean and the multiplication result as the first boundary point, and use the mean and the multiplication result as the first boundary point. The sum of is used as the second boundary point;

A preset range is formed by the first boundary point and the second boundary point, and a feature point matching pair corresponding to the projection error within the preset range is determined as a target point matching pair.

Among them, the mean can represent the average of all projection errors, the variance can represent the fluctuation of all projection errors, the preset coefficient can be used to adjust the preset range, the preset coefficient is an adjustable parameter, and the first boundary point can refer to the preset range. Assuming the left boundary point of the range, the second boundary point may refer to the right boundary point of the preset range.

Specifically, based on the prior that the projection error conforms to the normal distribution, the variance and mean of all projection errors are calculated, and the matching point near the mean of the projection error is regarded as the correct matching point, while the vicinity of the mean needs to be determined by a preset range. Sure.

In this embodiment, the projection error is screened based on the prior of normal distribution, which can quickly and accurately screen feature point matching pairs. Compared with the commonly used RANSAC screening method, etc., it effectively improves the image feature point matching and screening process. s efficiency.

The above steps of calculating the projection error of each set of feature point matching pairs based on the reference rotation matrix and the reference translation vector, and determining the feature point matching pair corresponding to the projection error within the preset range as the target point matching pair can accurately calculate the projection error according to the Determine reliable target point matching pairs and improve the precision and accuracy of matching feature point screening.

In this embodiment, the reference rotation matrix and the reference translation matrix obtained by clustering are used to filter feature point matching pairs, thereby retaining reliable target point matching pairs, improving the precision and accuracy of feature point matching, and thus improving image matching. accuracy.

Corresponding to the clustering-based image feature point matching and screening method in the above embodiment, Figure 3 shows a structural block diagram of the clustering-based image feature point matching and screening device provided in Embodiment 2 of the present invention. The above image feature point matching and screening method The device is applied to the server, and the computer equipment corresponding to the server is connected to the client to obtain the first image and the second image to be matched. The computer equipment corresponding to the server can perform computing tasks such as projection matrix operations, matrix decomposition, etc., in order to For convenience of explanation, only parts related to the embodiments of the present invention are shown.

Referring to Figure 3, the image feature point matching and filtering device includes:

The feature point extraction module 31 is used to obtain the first image and the second image to be matched, extract the feature points in the first image to obtain the first feature point set, and extract the feature points in the second image to obtain the second feature point. gather;

The feature point matching module 32 is used to perform feature point matching on the first feature point set and the second feature point set to obtain N sets of feature point matching pairs, where the feature point matching pairs include one feature point in the first feature point set and the second feature point set. One feature point in the set of two feature points, N is an integer greater than five;

The matching pair selection module 33 is used to randomly select M groups of temporary matching pairs from N groups of feature point matching pairs, and calculate the distance between the first image and the second image in the corresponding temporary matching pair according to each group of temporary matching pairs. Temporary rotation matrices and temporary translation vectors are used to obtain M temporary rotation matrices and M temporary translation vectors, where M is an integer less than or equal to N;

The clustering processing module 34 is used to cluster the M temporary rotation matrices to obtain the first cluster center point, determine the first cluster center point as the reference rotation matrix, cluster the M temporary translation vectors, and obtain the first cluster center point. Two cluster center points, determine the second cluster center point as the reference translation vector;

The matching pair determination module 35 is used to calculate the projection error of each set of feature point matching pairs based on the reference rotation matrix and the reference translation vector, and determine the feature point matching pair corresponding to the projection error within the preset range as the target point matching pair, The target point matching pair is used to calculate the target rotation matrix and the target translation vector between the first image and the second image to indicate image matching between the first image and the second image.

Optionally, the above feature point extraction module 31 includes:

The first feature point extraction unit is used to search within the first image using a feature point extraction operator to obtain Q first feature points in the first image and their corresponding first feature descriptors, where Q is an integer greater than N. ;

The second feature point extraction unit is used to search within the second image using a feature point extraction operator to obtain S second feature points in the second image and their corresponding second feature descriptors, where S is an integer greater than N. ;

A set forming unit configured to form a first feature point set of the first image from the Q first feature points in the first image, and to form a second feature point set of the second image from the S second feature points in the second image. gather.

Optionally, the above feature point matching module 32 includes:

A similarity calculation unit, configured to compare the first feature descriptor of the first feature point with the first feature descriptor of each second feature point in the second feature point set for any first feature point in the first feature point set. Calculate the similarity between the two feature descriptors to obtain the similarity value corresponding to each second feature point in the second feature point set, and determine that the second feature point corresponding to the maximum value among all similarity values and the first feature point form a Group feature point matching pairs;

The feature point traversal unit is used to traverse all the first feature points in the first feature point set to obtain N sets of feature point matching pairs.

Optionally, the above matching pair selection module 33 includes:

An essential matrix calculation unit, used for calculating, for any temporary matching pair, the essential matrix between the first image and the second image within the temporary matching pair according to the epipolar constraint and the temporary matching pair;

The matrix decomposition unit is used to perform matrix decomposition on the essential matrix to obtain a temporary rotation matrix and a temporary translation vector between the first image and the second image.

Optionally, the above-mentioned clustering processing module 34 includes:

The matrix conversion unit is used to convert each temporary rotation matrix into a temporary rotation vector, cluster all temporary rotation vectors according to the Euclidean distance between the temporary rotation vectors, obtain the first cluster center point, and determine the first cluster The center point is the reference rotation vector;

Vector conversion unit, used to convert the reference rotation vector into the reference rotation matrix.

Optionally, the above-mentioned clustering processing module 34 includes:

The vector mapping unit is used to map each temporary translation vector to a temporary translation point on the unit sphere, cluster all temporary translation points according to the spherical distance between the temporary translation points, and obtain the second cluster center point;

The vector determination unit is used to determine the second cluster center point as the reference translation vector.

Optionally, the above matching pair determination module 35 includes:

The boundary point calculation unit is used to calculate the mean and variance of all projection errors. According to the mean and variance, the variance and the preset coefficient are multiplied to obtain the multiplication result. The difference between the mean and the multiplication result is used as the first boundary point. , taking the sum of the mean and the multiplication result as the second boundary point;

A range determination unit is configured to form a preset range from the first boundary point and the second boundary point, and determine a matching pair of feature points corresponding to the projection error within the preset range as a matching pair of target points.

It should be noted that the information interaction and execution process between the above-mentioned modules and units are based on the same concept as the method embodiments of the present invention. For details of their specific functions and technical effects, please refer to the method embodiments section. No further details will be given.

FIG. 4 is a schematic structural diagram of a computer device provided in Embodiment 3 of the present invention. As shown in Figure 4, the computer device of this embodiment includes: at least one processor (only one is shown in Figure 4), a memory, and a computer program stored in the memory and executable on at least one processor. The processor executes The computer program implements the steps in any of the above embodiments of the clustering-based image feature point matching and screening method.

The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that FIG. 4 is only an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or may combine certain components, or use different components. , for example, it may also include a network interface, a display screen, an input device, etc.

The so-called processor can be a CPU, which can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field- Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

The memory includes readable storage media, internal memory, etc., wherein the internal memory can be the memory of the computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device. In other embodiments, it may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMC), etc. equipped on the computer device. Secure Digital (SD) card, Flash Card, etc. Further, the memory may also include both internal storage units of the computer device and external storage devices. The memory is used to store operating systems, application programs, boot loaders (Boot Loaders), data, and other programs, such as program codes of computer programs. The memory can also be used to temporarily store data that has been output or is to be output.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional units is used as an example. In practical applications, the implementer can allocate the above functions to different functional units according to needs. Complete, that is, divide the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware. , can also be implemented in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of distinguishing each other and are not used to limit the scope of the present invention. For the specific working processes of the units and modules in the above device, reference can be made to the corresponding processes in the foregoing method embodiments, which will not be described again here. Integrated units may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as independent products. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can be processed after being processed. When the processor is executed, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. Computer-readable media may at least include: any entity or device capable of carrying computer program code, recording media, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrical carrier wave Signals, telecommunications signals, and software distribution media. For example, U disk, mobile hard disk, magnetic disk or CD, etc. In some jurisdictions, subject to legislation and patent practice, computer-readable media may not be electrical carrier signals and telecommunications signals.

The present invention can implement all or part of the processes in the above method embodiments, and can also be completed through a computer program product. When the computer program product is run on a computer device, the computer device can implement the steps in the above method embodiments when executed. step.

In the above embodiments, each embodiment is described with its own emphasis. For parts that are not detailed or documented in a certain embodiment, please refer to the relevant descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered to be beyond the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/computer equipment and methods can be implemented in other ways. For example, the apparatus/computer equipment embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components. can be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separate. A component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions of the foregoing embodiments. Modifications are made to the recorded technical solutions, or equivalent substitutions are made to some of the technical features; however, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present invention, and should all be included in the present invention. within the scope of protection.

Claims (10)

1. A clustering-based image feature point matching and screening method, characterized in that the image feature point matching and screening method includes: Obtain the first image and the second image to be matched, extract feature points in the first image to obtain a first feature point set, extract feature points in the second image to obtain a second feature point set; Feature point matching is performed on the first feature point set and the second feature point set to obtain N sets of feature point matching pairs, where the feature point matching pairs include one feature point in the first feature point set and all A feature point in the second feature point set, N is an integer greater than five; M groups of temporary matching pairs are randomly selected from the N groups of feature point matching pairs, and based on each group of temporary matching pairs, the temporary rotation between the first image and the second image in the corresponding temporary matching pair is calculated. Matrix and temporary translation vector, get M temporary rotation matrices and M temporary translation vectors, M is an integer less than or equal to N; The M temporary rotation matrices are clustered to obtain the first cluster center point, the first cluster center point is determined as the reference rotation matrix, and the M temporary translation vectors are clustered to obtain the second clustering center point. Class center point, determine the second cluster center point as the reference translation vector; According to the reference rotation matrix and the reference translation vector, the projection error of each set of feature point matching pairs is calculated, and the feature point matching pairs corresponding to the projection errors within the preset range are determined as target point matching pairs. The point matching pair is used to calculate a target rotation matrix and a target translation vector between the first image and the second image to indicate image matching between the first image and the second image. 2. The image feature point matching and screening method according to claim 1, characterized in that: extracting feature points in the first image to obtain a first feature point set, and extracting feature points in the second image , get the second feature point set, including: Use a feature point extraction operator to search within the first image to obtain Q first feature points and their corresponding first feature descriptors in the first image, where Q is an integer greater than N; Use the feature point extraction operator to search within the second image to obtain S second feature points and their corresponding second feature descriptors in the second image, where S is an integer greater than N; The first feature point set of the first image is formed by Q first feature points in the first image, and the second feature point set of the second image is formed by S second feature points in the second image. Collection of feature points. 3. The image feature point matching and screening method according to claim 2, wherein the feature point matching is performed on the first feature point set and the second feature point set to obtain N groups of feature point matching pairs. ,include: For any first feature point in the first feature point set, combine the first feature descriptor of the first feature point and the second feature point of each second feature point in the second feature point set. The feature descriptor performs similarity calculation to obtain the similarity value corresponding to each second feature point in the second feature point set, and determines the second feature point corresponding to the maximum value among all similarity values and the first feature points form a set of feature point matching pairs; Traverse all first feature points in the first feature point set to obtain the N sets of matching pairs of feature points. 4. The image feature point matching and screening method according to claim 1, characterized in that, according to each group of temporary matching pairs, the distance between the first image and the second image in the corresponding temporary matching pair is calculated. The temporary rotation matrix and temporary translation vector include: For any temporary matching pair, calculate the essential matrix between the first image and the second image within the temporary matching pair according to the epipolar constraint and the temporary matching pair; Perform matrix decomposition on the essential matrix to obtain a temporary rotation matrix and a temporary translation vector between the first image and the second image. 5. The image feature point matching and screening method according to claim 1, wherein the M temporary rotation matrices are clustered to obtain a first cluster center point, and the first cluster center is determined. Points are reference rotation matrices, including: Convert each temporary rotation matrix into a temporary rotation vector, cluster all temporary rotation vectors according to the Euclidean distance between the temporary rotation vectors, obtain the first cluster center point, and determine the first cluster center point is the reference rotation vector; Convert the reference rotation vector into the reference rotation matrix. 6. The image feature point matching and screening method according to claim 1, characterized in that the M temporary translation vectors are clustered to obtain a second cluster center point, and the second cluster center is determined. The point is the reference translation vector, including: Map each temporary translation vector to a temporary translation point on the unit sphere, cluster all temporary translation points according to the spherical distance between the temporary translation points, and obtain the second cluster center point; The second cluster center point is determined as the reference translation vector. 7. The image feature point matching and screening method according to any one of claims 1 to 6, wherein the feature point matching pair corresponding to the projection error within the preset range is determined as the target point matching pair, including : The mean and variance of all projection errors are calculated. According to the mean and the variance, the variance is multiplied by a preset coefficient to obtain a multiplication result. The difference between the mean and the multiplication result is used as the first A boundary point, taking the sum of the mean value and the multiplication result as the second boundary point; The preset range is formed by the first boundary point and the second boundary point, and a feature point matching pair corresponding to a projection error within the preset range is determined as the target point matching pair. 8. An image feature point matching and screening device based on clustering, characterized in that the image feature point matching and screening device includes: A feature point extraction module is used to obtain the first image and the second image to be matched, extract the feature points in the first image to obtain the first feature point set, extract the feature points in the second image to obtain the third Two feature point sets; A feature point matching module, configured to perform feature point matching on the first feature point set and the second feature point set to obtain N sets of feature point matching pairs, where the feature point matching pairs include the first feature point set One feature point in and one feature point in the second feature point set, N is an integer greater than five; A matching pair selection module, configured to randomly select M groups of temporary matching pairs from the N groups of feature point matching pairs, and calculate the first image and the first image in the corresponding temporary matching pair according to each group of temporary matching pairs. The temporary rotation matrix and temporary translation vector between the two images are obtained, and M temporary rotation matrices and M temporary translation vectors are obtained. M is an integer less than or equal to N; A clustering processing module is used to cluster the M temporary rotation matrices to obtain a first cluster center point, determine the first cluster center point as a reference rotation matrix, and perform processing on the M temporary translation vectors. Cluster, obtain the second cluster center point, and determine the second cluster center point as the reference translation vector; A matching pair determination module, configured to calculate the projection error of each set of feature point matching pairs according to the reference rotation matrix and the reference translation vector, and determine the feature point matching pair corresponding to the projection error within the preset range as the target point Matching pair, the target point matching pair is used to calculate the target rotation matrix and target translation vector between the first image and the second image to instruct the first image and the second image to perform image processing. match. 9. A computer device, characterized in that the computer device includes a processor, a memory and a computer program stored in the memory and executable on the processor. When the processor executes the computer program Implement the image feature point matching and screening method as described in any one of claims 1 to 7. 10. A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that, when the computer program is executed by a processor, the image characteristics as claimed in any one of claims 1 to 7 are realized. Point match filtering method.