CN119445540A

CN119445540A - A mechanical parking space detection method, device, electronic equipment, medium and product

Info

Publication number: CN119445540A
Application number: CN202510034885.5A
Authority: CN
Inventors: 杨建飞; 由强; 王斌
Original assignee: Zhejiang Geely Holding Group Co Ltd; Zhejiang Zeekr Intelligent Technology Co Ltd
Current assignee: Zhejiang Geely Holding Group Co Ltd; Zhejiang Zeekr Intelligent Technology Co Ltd
Priority date: 2025-01-09
Filing date: 2025-01-09
Publication date: 2025-02-14

Abstract

The application provides a method and a device for detecting a mechanical parking space, electronic equipment, a medium and a product. According to one example of the method, multiple-frame parking space images are detected to obtain point line information in the multiple-frame parking space images, wherein the multiple-frame parking space images are obtained by shooting a mechanical parking space through a monocular camera, the point line information comprises an entrance angular point of the mechanical parking space and a parking space inner line, the direction of the parking space inner line is a direction from a parking space entrance to a parking space end, the entrance angular point is matched with the parking space inner line to obtain point line pairs of the mechanical parking space, the point line pairs are subjected to rectangular processing to obtain parking space lines of the mechanical parking space, and the parking space lines of the mechanical parking space can be accurately identified through multiple-frame image processing and point line matching, so that the parking space detection precision is improved.

Description

Detection method and device for mechanical parking space, electronic equipment, medium and product

Technical Field

The application relates to the technical field of auxiliary driving, in particular to a method and a device for detecting a mechanical parking space, electronic equipment, media and products.

Background

With the acceleration of the urban process, land resources are increasingly tense, and the mechanical parking space is used as a parking mode for efficiently utilizing space resources, so that the method is widely applied to various parking lots. However, the mechanical parking spaces are often compact in design and complex in structure, the spaces between the parking spaces are narrow, and shielding objects such as upright posts and cross beams are often accompanied.

Monocular cameras are widely used in parking space detection because of their advantages of relatively low cost, simplicity in installation, and the like. However, the field of view of the monocular camera is limited, and the monocular camera can only capture image information within a certain range of the direction opposite to the camera, and various shielding objects exist around the mechanical parking space, so that the parking space information cannot be completely acquired, and the parking space detection precision is reduced.

Disclosure of Invention

In order to overcome the problems in the related art, the application provides a method, a device, electronic equipment, a medium and a product for detecting a mechanical parking space.

According to a first aspect of any embodiment of the present application, there is provided a method for detecting a mechanical parking space, the method including:

Detecting multi-frame parking space images to obtain dot line information in the multi-frame parking space images, wherein the multi-frame parking space images are obtained by shooting a mechanical parking space through a monocular camera, and the dot line information comprises an entrance angular point of the mechanical parking space and a parking space inner line, and the direction along the parking space inner line is a direction along a parking space entrance to a parking space end;

Matching the entrance angular points with the inner line of the parking space to obtain a dotted line pair of the mechanical parking space;

And carrying out rectangular treatment on the dot line pairs to obtain the parking space lines of the mechanical parking spaces.

According to a second aspect of any embodiment of the present application, there is provided a detection device for a mechanical parking space, the device including:

the detection module is used for detecting a plurality of frames of parking space images to obtain dot line information in the plurality of frames of parking space images, wherein the plurality of frames of parking space images are obtained by shooting a mechanical parking space through a monocular camera, and the dot line information comprises an entrance angular point of the mechanical parking space and a parking space inner line, and the direction along the parking space inner line is the direction along the parking space entrance to the parking space end;

the matching module is used for matching the entrance angular points with the inner line of the parking space to obtain a dotted line pair of the mechanical parking space;

And the processing module is used for carrying out rectangular processing on the dot line pairs to obtain the parking space lines of the mechanical parking spaces.

According to a third aspect of any embodiment of the present application, there is provided an electronic device comprising:

A processor;

a memory for storing processor-executable instructions;

wherein the processor implements the method described in any of the embodiments of the application by executing the executable instructions.

According to a fourth aspect of any of the embodiments of the present application there is provided a computer readable storage medium having stored thereon computer instructions which when executed by a processor implement a method as described in any of the embodiments of the present application described above.

According to a fifth aspect of any of the embodiments of the present application there is provided a computer program product having stored thereon a computer program/instruction which when executed by a processor implements a method as described in any of the embodiments of the present application described above.

The technical scheme provided by the application can comprise the following beneficial effects:

according to the embodiment, the dot line information in the multi-frame parking space image is obtained by detecting the multi-frame parking space image, the entrance angular points in the dot line information are matched with the inner line of the parking space to obtain the dot line pair of the mechanical parking space, the dot line pair is subjected to rectangular processing to obtain the parking space line of the mechanical parking space, and the parking space line of the mechanical parking space can be accurately identified by multi-frame image processing and dot line matching, so that the parking space detection precision is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

FIG. 1 is a flow chart of a method for detecting a mechanical parking space according to an exemplary embodiment of the present application;

FIG. 2 is a schematic diagram of dotted information in a multi-frame parking space image according to an exemplary embodiment of the present application;

FIG. 3 is a flow chart illustrating a method for detecting a line in a parking space according to an exemplary embodiment of the present application;

FIG. 4 is a schematic diagram of a maximum viewing angle difference according to an exemplary embodiment of the present application;

FIG. 5 is a schematic diagram of a pair of matching dotted lines according to an exemplary embodiment of the present application;

FIG. 6 is a schematic diagram of a search matching dotted pair according to an exemplary embodiment of the present application;

FIG. 7 is a schematic diagram of a squaring process for dotted pairs according to an exemplary embodiment of the present application;

FIG. 8 is a flow chart illustrating another method of detecting a mechanical parking spot according to an exemplary embodiment of the present application;

Fig. 9 is a schematic diagram of an electronic device according to an exemplary embodiment of the present application;

fig. 10 is a block diagram of a detection device for a mechanical parking space according to an exemplary embodiment of the present application.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the application. The term "if" as used herein may be interpreted as "at..once" or "when..once" or "in response to a determination", depending on the context.

At present, various shielding objects exist around a mechanical parking space, the visual field range of a monocular camera is limited by the installation position and the angle of the monocular camera, and the parking space information cannot be obtained completely. This results in the accuracy in the parking spot detection being affected, and erroneous or missed judgment is likely to occur.

In order to solve the above-mentioned problems, the present application provides a method for detecting a mechanical parking space, and for further explaining the present application, the following embodiments are provided:

Referring to fig. 1, fig. 1 is a flowchart illustrating a method for detecting a mechanical parking space according to an exemplary embodiment of the present application. The detection method of the mechanical parking space can be executed by a parking space detection system, and the parking space detection system can be applied to vehicles and cloud service ends.

The detection method of the mechanical parking space can comprise the following steps:

Step 102, detecting multi-frame parking space images to obtain dot line information in the multi-frame parking space images, wherein the multi-frame parking space images are obtained by shooting a mechanical parking space through a monocular camera, and the dot line information comprises an entrance corner point of the mechanical parking space and a line in the parking space, and the direction along the line in the parking space is a direction along the entrance of the parking space to the end of the parking space.

In this step, the parking stall detecting system can shoot machinery parking stall through the monocular camera, obtains multiframe parking stall image. The parking space detection system can utilize the deep learning model to carry out fusion detection on multiple frames of parking space images, detect points and lines in the multiple frames of parking space images, de-duplicate the points and lines, and obtain dot line information in the multiple frames of parking space images.

The parking space detection system can also identify the dot line information in the multi-frame parking space images by utilizing image processing technologies such as edge detection, corner detection, straight line detection and the like.

When the parking space detection system detects multi-frame parking space images, the relative position of the dot line information in the mechanical parking space can be detected by analyzing the spatial positions and the directions of the dot line and the relative relation between the dot line and the line, for example, whether the entrance corner point and the line in the parking space belong to the left side or the right side of the mechanical parking space is determined.

The multi-frame parking space image is an image sequence obtained by shooting the mechanical parking space for many times through a monocular camera at different time points or different visual angles, and is used for acquiring parking space information such as the position, the size and the shape of the mechanical parking space.

A monocular camera refers to a camera system that captures image information through only one lens, captures light through an image sensor, and generates a two-dimensional image. The monocular camera may be a monocular camera, a pinhole camera, or the like.

The dot line information refers to key points and line information extracted from multi-frame parking space images. The dotted line information can comprise an entrance corner point of the mechanical parking space and a line in the parking space. The entrance corner point is a characteristic point of a parking space entrance, and is used for positioning the entrance position of a mechanical parking space.

The line along the parking stall is the line along parking stall entry to parking stall end direction in the parking stall, is used for showing the length and the direction of machinery parking stall along the line in the parking stall, and the direction along the line in the parking stall is along the parking stall entry to the direction of parking stall end. The inner lines of the parking spaces can comprise a left inner line and a right inner line of the mechanical parking spaces.

The point line information in the multi-frame parking space images is detected through fusion, the inner line information on two sides of the mechanical parking space can be obtained, the situation that the inner edges on two sides of the parking space cannot be seen at the same time due to shielding is avoided, and accordingly the parking space detection precision is improved.

The method for removing the duplication of points and lines in the multi-frame parking space image can be used for determining that two points are duplicated and one point is reserved when the distance between the two points is smaller than a threshold value according to Euclidean distance or Manhattan distance between the points, and determining that two lines are duplicated and one line is reserved according to the angle, slope or position relation between the two lines if the slopes of the two lines are similar and the positions of the two lines are overlapped.

For another example, feature descriptors of points and lines may be extracted for detected points and lines, such as feature point descriptors of Scale-invariant feature transforms (Scale-INVARIANT FEATURE TRANSFORM, SIFT), accelerated robust features (Speeded Up Robust Features, SURF), segment-based feature descriptors, and the like. A feature matching algorithm is used to match points and lines in different frames, and if the feature descriptors of two points or lines are very similar, it is determined that the two points or lines are duplicate.

Referring to fig. 2, fig. 2 shows a schematic diagram of dot line information in a multi-frame parking space image. Illustratively, the parking space detection system may capture a mechanical parking space through a monocular camera to obtain a multi-frame parking space image, and detect a left inner line 21 (blue line) and a right inner line 22 (yellow line), and a left entrance corner point 23 (blue point) and a right entrance corner point 24 (yellow point) in the multi-frame parking space image.

And 104, matching the entrance angular points with the inner line of the parking space to obtain a dotted line pair of the mechanical parking space.

In the step, the parking space detection system can extract point characteristic information such as position coordinates and directions of entrance angular points, and extract line characteristic information such as slope, line segment starting point and line segment ending point along the line in the parking space. And matching the entrance corner points with the inner line of the parking space based on the point characteristic information and the line characteristic information to obtain a plurality of point-line pairs of the mechanical parking space.

For example, the parking space detection system may calculate the distance from the entry corner to each point along the parking space, and select the point with the smallest distance as the matching point. The spatial relationship between the entrance corner point and the inner line of the parking space can be judged, the entrance corner point is located at or near the starting end of the inner line of the parking space, and the inner line of the parking space extends along the direction from the entrance corner point to the end of the parking space.

The dot-line pair is an information unit capable of reflecting the basic shape and the position of the mechanical parking space and is used for reflecting an inlet and the extending direction of the mechanical parking space, and the dot-line pair can be a left dot-line pair of the mechanical parking space or a right dot-line pair of the mechanical parking space.

The left dotted line pair is a dotted line pair formed by the left inner line of the mechanical parking space and the left entrance angular point, and the right dotted line pair is a dotted line pair formed by the right inner line of the mechanical parking space and the right entrance angular point.

With continued reference to fig. 2, the parking space detection system may match the entry corner points in the multi-frame parking space image with the inner line of the parking space, to obtain a left dot line pair and a right dot line pair of the mechanical parking space. Wherein the left dotted line pair comprises a left inner line 21 and a left entry corner point 23 and the right dotted line pair comprises a right inner line 22 and a right entry corner point 24.

And 106, carrying out rectangular treatment on the pair of dot lines to obtain a parking space line of the mechanical parking space.

In the step, the parking space detection system carries out rectangular treatment on a plurality of point line pairs obtained by matching to obtain a parking space line of the mechanical parking space.

For example, rectangular processing is performed on the left dot line pair and the right dot line pair of the same mechanical parking space, and the entrance corner points in the left dot line pair and the right dot line pair and the inner line of the parking space are converted into standard rectangular shapes so as to describe and position the parking space more accurately and obtain the parking space line of the mechanical parking space.

According to the detection method for the mechanical parking space, the multi-frame parking space image is detected to obtain the point line information in the multi-frame parking space image, the entrance corner points in the point line information are matched with the inner line of the parking space to obtain the point line pair of the mechanical parking space, the point line pair is subjected to rectangular processing to obtain the parking space line of the mechanical parking space, and the parking space line of the mechanical parking space can be accurately identified through multi-frame image processing and point line matching, so that the precision of parking space detection is improved.

In the foregoing embodiment, it is described that by performing the dot line detection, matching, and squaring processing on the multi-frame parking space image, the parking space line of the mechanical parking space is accurately detected. In the following embodiments, a more detailed description will be made regarding the detection process of the dot line information, and may be applied to any of the above embodiments.

In one embodiment, fig. 3 shows a flow chart for detecting a line in a parking space. As shown in fig. 3, the parking space detection system may input each frame of parking space image into a point line detection model, and perform rolling and pooling operations on the parking space image by using a Backbone network (backhaul) such as ResNet, mobileNet in the point line detection model, so as to extract internal line features in multiple frames of parking space images.

And generating a first thermodynamic diagram corresponding to the internal line characteristics by using one or more convolution layers in the dotted line detection model, wherein each pixel value on the first thermodynamic diagram represents the probability that the corresponding position is the internal line of the parking space.

And selecting a first candidate point along the line in the parking space from the first thermodynamic diagram, and taking a pixel point higher than a first preset threshold value in the first thermodynamic diagram as the first candidate point. The number of first candidate points may be further reduced by non-maximum suppression or the like to eliminate redundant and overlapping points in the first thermodynamic diagram.

The parking space detection system can also generate a first offset corresponding to the internal line characteristics by using the dotted line detection model, and add the first offset to the original predicted position of the first candidate point in the first thermodynamic diagram to obtain an adjusted first candidate point.

And grouping the first candidate points by using a clustering algorithm such as K-means, a Density-based clustering method with noise (Density-Based Spatial Clustering of Applications with Noise, DBSCAN) and the like, and grouping the first candidate points into a set of points along the inner line of a parking space of the mechanical parking space.

Fitting the point sets along the parking space by using fitting algorithms such as polynomial fitting, least square fitting and the like to obtain two-dimensional parking space inner line output by the point line detection model.

The internal line features are feature information of the internal line of the parking space. The first candidate points are pixel points which form a line in the parking space in the first thermodynamic diagram. The dotted line detection model is a timing model based on deep learning.

The first thermodynamic diagram is a thermodynamic diagram (Heatmap) which visually represents a first candidate point along the inner line of the parking space and is used for reflecting the probability that each pixel point in the parking space image belongs to the inner line of the parking space. The first Offset is an Offset (Offset) of the predicted position of the first candidate point with respect to the true position, and is used for adjusting the predicted position of the first candidate point to the actual in-line position.

As described above, by extracting the internal line features in the multi-frame parking space images, the first thermodynamic diagram and the first offset corresponding to the internal line features are generated, the first candidate points along the internal line of the parking space are determined based on the first thermodynamic diagram and the first offset, the positions of the internal line can be thinned, the first candidate points are clustered and fitted, the internal line of the parking space is obtained, the conditions of false detection and missing detection along the internal line can be effectively reduced, and the reliability of parking space detection is improved.

In an embodiment, please continue to refer to fig. 3, the parking space detection system uses the backbone network in the dotted line detection model to perform rolling and pooling operations on the parking space images, and extracts the point features in the multi-frame parking space images.

One or more convolution layers in the point-line detection model may be utilized to generate a second thermodynamic diagram corresponding to the point features, each pixel value on the second thermodynamic diagram representing a probability that the corresponding location is an entry corner.

And selecting a second candidate point of the entry corner point from the second thermodynamic diagram, and taking the pixel point higher than a second preset threshold value in the second thermodynamic diagram as the second candidate point.

The parking space detection system can also generate a second offset corresponding to the point characteristic by using the point line detection model, and add the second offset to the original predicted position in the second thermodynamic diagram in the second candidate point to obtain an adjusted second candidate point.

And deleting repeated points in the second candidate points by using a non-maximum value inhibition method, and taking the second candidate points as entrance angular points to obtain two-dimensional entrance angular points output by the dotted line detection model.

Wherein the point feature is feature information of the entry corner point. The second candidate points are the pixel points that make up the entry corner point in the second thermodynamic diagram.

The second thermodynamic diagram is a thermodynamic diagram representing a second candidate point of the entry corner point and is used for reflecting the probability that each pixel point in the parking space image belongs to the entry corner point. The second offset is an offset of the predicted position of the second candidate point with respect to the actual position, and is used for adjusting the predicted position of the second candidate point to the actual entry corner position.

As described above, by extracting the point features in the multi-frame parking space image and generating the second thermodynamic diagram and the second offset corresponding to the point features, the positioning error caused by factors such as image noise and distortion can be reduced, the second candidate points of the entrance angular points are determined based on the second thermodynamic diagram and the second offset, the repeated points in the second candidate points are deleted, and the second candidate points are used as the entrance angular points, so that repeated calculation or misjudgment in subsequent processing can be avoided, and the accuracy of parking space detection is improved.

In an embodiment, the parking space detection system may perform distortion correction on the detected two-dimensional inner line and entrance corner points of the parking space, map the inner line and entrance corner points of the parking space in the distorted parking space image onto the undistorted image through geometric transformation, and obtain corrected inner line and entrance corner points of the parking space.

According to the internal reference matrix of the monocular camera, calculating an inverse matrix of the internal reference matrix through a matrix inversion algorithm. And converting the line and entrance angular points in the parking space from an image coordinate system to a camera coordinate system by using an inverse matrix of the internal reference matrix. And projecting the image coordinates along the line and the entrance angular point in the parking space to a normalization plane to obtain the normalized image coordinates along the line and the entrance angular point in the parking space, which are not influenced by the internal parameters and distortion of the camera.

As described above, by correcting distortion of the detected line and entrance angular points in the parking space, the influence of the distortion on the line and entrance angular points in the parking space can be reduced, and according to the inverse matrix of the internal reference matrix of the monocular camera, the image coordinates of the line and entrance angular points in the parking space are projected to the normalization plane to obtain the normalized line and entrance angular points in the parking space, so that the calculation complexity and error sources are reduced, and the subsequent detection of the parking space line is simpler, more convenient and more accurate.

In an embodiment, before matching the entrance corner point with the internal line of the parking space, the parking space detection system may obtain position, orientation, etc. pose information of the own vehicle corresponding to the parking space image according to the timestamp of the parking space image by means of a vehicle positioning system, a map matching algorithm, etc.

According to the pose information of the vehicle and the external parameters of the monocular camera, a two-dimensional point or line under the visual angle of the camera is converted into a three-dimensional position or direction in a world coordinate system by using a point and line triangulation algorithm, and the three-dimensional position of the point line information is determined.

The parking space detection system can determine the three-dimensional position of the entrance angular point by using a point triangulation algorithm. For example, by minimizing projection errors, it is ensured that two-dimensional entry corner points observed from a plurality of frame parking space images are matched to the same three-dimensional point, and the three-dimensional positions of the entry corner points are determined.

The parking space detection system can determine the three-dimensional position along the line in the parking space by using a line triangulation algorithm. For example, the three-dimensional straight line can be fitted by triangulating points on the line in the parking space and then using a least square method or the like, or the three-dimensional direction or position of the line in the parking space can be estimated by directly using the geometric constraint of the linear feature in the image of the parking space.

Since the own vehicle and the camera move continuously during running, the viewing angle of the same point information changes. The parking space detection system analyzes the observation view angles of multi-frame parking space images based on three-dimensional positions of entrance angular points and the inner line of the parking space, and determines the maximum view angle difference of the entrance angular points in the dot line information and the inner line of the parking space in any two frames.

When the maximum viewing angle difference is larger than the preset threshold value, the dotted line information is obviously changed in position among the parking space images of different frames, which is more likely to be caused by real movement or structural change, and the dotted line information can be reserved, and the dotted line information maintains relatively stable and obvious characteristics under different viewing angles.

And deleting the dot line information under the condition that the maximum visual angle difference is smaller than or equal to a preset threshold value, and carrying out dot line matching and squaring processing without using the dot line information in the multi-frame parking space images.

The external parameters of the monocular camera refer to the position and the gesture of the monocular camera in the world coordinate system, and are used for defining the conversion relationship between the camera coordinate system and the world coordinate system, and the external parameters can be fixed or can be obtained through camera calibration.

The maximum angle difference is formed according to the observation angle of the multi-frame parking space images. The preset threshold is used for distinguishing whether the visual angle change is large enough to influence the accuracy of parking space detection.

Referring to fig. 4, fig. 4 is a schematic diagram showing a maximum viewing angle difference. The observation frames of the multi-frame parking space image comprise an observation frame A and an observation frame B, and the angle difference formed by the observation frame A and the observation frame B on the left entrance corner 23 is the maximum angle difference.

And comparing the maximum visual angle difference with a preset threshold value by the parking space detection system, and if the maximum visual angle difference is larger than the preset threshold value, performing subsequent point line matching and squaring processing by using the point line information in the multi-frame parking space images.

As described above, by determining the three-dimensional position of the dot line information according to the pose information of the own vehicle and the external parameters of the monocular camera, determining the maximum viewing angle difference of the dot line information based on the three-dimensional position, and retaining the dot line information when the maximum viewing angle difference is greater than a preset threshold, the visibility and consistency of the dot line information at different viewing angles can be more accurately evaluated, thereby improving the accuracy of the subsequent dot line matching.

In the foregoing embodiment, it is described that by using the thermodynamic diagram and the offset corresponding to the dot line features, and verifying the maximum viewing angle difference of the dot line information, the accuracy of detecting the parking space line of the mechanical parking space is further improved. In the following embodiments, the dot-line matching and squaring process will be described in more detail, and may be applied to any of the embodiments as above.

In an embodiment, the entry corner points may comprise a first entry corner point and a second entry corner point. The parking space inner line can comprise a first inner line and a second inner line, wherein the first inner line is the parking space inner line closest to the first entrance angular point, and the second inner line is the parking space inner line closest to the second entrance angular point.

The parking space detection system can acquire the inner line of the parking space closest to the entrance angular points according to each entrance angular point. Detecting a distance value between an entrance angular point and a line in a parking space closest to the entrance angular point, and comparing the distance value with a distance threshold value to obtain a matched dotted line pair.

And forming a dotted line pair of the first entrance corner point and the first inner line under the condition that a first distance value between the first entrance corner point and the first inner line is smaller than a distance threshold value. And forming a dotted line pair between the second entrance corner point and the second inner line under the condition that a second distance value between the second entrance corner point and the second inner line is smaller than a distance threshold value.

Wherein the first distance value is a distance value between the first entry corner point and the first inner line. The first distance value may comprise at least one of a horizontal distance value, a vertical distance value, a distance value of the nearest endpoint of the first entry corner point to the first inner line.

The second distance value is the distance value between the second entry corner point and the second inner line. The second distance value may comprise at least one of a horizontal distance value, a vertical distance value, a distance value of the second entry corner point from the nearest end point along the second inner line.

The first interior line is the line that is close to one side boundary in the machinery parking stall inside, and the direction along the parking stall entry to the direction of parking stall end in the first interior line can be the left interior line of machinery parking stall.

The second inner line is a line, which is close to the boundary of the other side, inside the mechanical parking space, the direction of the second inner line is the direction from the entrance of the parking space to the end of the parking space, and the second inner line can be the right inner line of the mechanical parking space.

The first inner line and the second inner line are relatively parallel or approximately parallel, and jointly define an inner space of the mechanical parking space.

The first entry corner is a corner at the entrance of the mechanical parking space and marks the beginning of a boundary of one side of the parking space. The first entry corner may be a left entry corner of the machine parking space. The second entrance corner corresponds to the first entrance corner, and the second entrance corner is another corner at the entrance of the mechanical parking space and marks the beginning of the boundary of the other side of the parking space. The second entry corner may be a right entry corner of the machine parking space.

Referring to fig. 5, fig. 5 shows a schematic diagram of a pair of matching dotted lines. Illustratively, the first entry corner is a left entry corner 23 and the second entry corner is a right entry corner 24. The first inner line closest to the left entry corner point 23 is the left inner line 21 and the second inner line closest to the right entry corner point 24 is the right inner line 22.

In case that both the horizontal distance value and the vertical distance value between the left entry corner point 23 and the left inner line 21 are smaller than the distance threshold value, the left entry corner point 23 and the left inner line 21 are formed into a left dotted line pair. In case that both the horizontal distance value and the vertical distance value between the right entry corner point 24 and the right inner line 22 are smaller than the distance threshold value, the right entry corner point 24 and the right inner line 22 are formed into a right dotted line pair.

As described above, under the condition that the first distance value between the first entrance angular point and the first inner line is smaller than the distance threshold value, the first entrance angular point and the first inner line closest to the first entrance angular point form a left dot line pair, under the condition that the second distance value between the second entrance angular point and the second inner line is smaller than the distance threshold value, the second entrance angular point and the second inner line closest to the second entrance angular point form a right dot line pair, the angular points can be accurately paired with the corresponding inner lines, incorrect pairing caused by errors is filtered, and the boundary of a mechanical parking space is more accurately determined, so that the accuracy of parking space detection is improved.

In one embodiment, the dot line pairs may include a plurality of first dot line pairs and a plurality of second dot line pairs. Wherein the first dotted pair may be a dotted pair consisting of the first entry corner point and the first inner line, i.e. a left dotted pair. The second dotted pair may be a dotted pair consisting of the second entry corner point and the second inner line, i.e., a right dotted pair.

The parking space detection system can match a plurality of first point line pairs with a plurality of second point line pairs, and the matched first point line pairs and the matched second point line pairs form matched point line pairs of the same mechanical parking space.

Referring to fig. 6, fig. 6 shows a schematic diagram of searching for matching dot-line pairs. For each first pair of points, the second pair of points closest to the right is searched, and for each second pair of points, the first pair of points closest to the left is searched. And taking the first point line pair and the second point line pair which are closest to each other as matching point line pairs.

The parking space detection system can also check whether the distance value between the first point line pair and the second point line pair closest to the parking space detection system exceeds a preset distance value, and the first point line pair and the second point line pair closest to the parking space detection system are used as matching point line pairs under the condition that the distance value between the first point line pair and the second point line pair closest to the parking space detection system exceeds the preset distance value.

The parking space detection system can carry out rectangular processing on the first point line pair and the second point line pair of each matching point line pair, and a parking space line of the mechanical parking space corresponding to each matching point line pair is obtained.

The matching point line pair consists of a first point line pair and a second point line pair of the same mechanical parking space and is used for marking the position and boundary information of the mechanical parking space.

As described above, by searching the second point line pair closest to the right for each first point line pair and searching the first point line pair closest to the left for each second point line pair and using the first point line pair closest to the second point line pair and the second point line pair closest to the first point line pair as the matching point line pair, mismatching due to factors such as noise, shielding or viewing angle change can be reduced, and the first point line pair and the second point line pair in each matching point line pair are subjected to squaring processing respectively to obtain the parking space line corresponding to the matching point line pair, so that each mechanical parking space can be ensured to be completely and accurately detected.

In one embodiment, referring to fig. 7, fig. 7 shows a schematic diagram of a squaring process of dotted pairs. The parking space detection system can acquire the average value of the included angles between the first point line pair and the second point line pair which form the same mechanical parking space, and the direction of the average value of the included angles is used as the parking space direction of the mechanical parking space.

The vehicle position direction is used instead of the direction of the first dot-line pair and the second dot-line pair. And rotating the first dot line pair by taking the entrance angular point in the first dot line pair as a reference point to obtain a left boundary line parallel to the parking space direction. And rotating the second dot line pair by taking the entrance angular point in the second dot line pair as a reference point to obtain a right boundary line parallel to the parking space direction.

The first entry corner in the first pair of points is projected to the parking space direction, resulting in a left projection point 71. And projecting the second entrance angular point in the second dot line pair to the parking space direction to obtain a right projection point 72. The midpoint between the left projection point and the right projection point is taken as the projection midpoint 73.

Based on the projected midpoint, the left boundary line, and the right boundary line, the front boundary line is constructed. The closest point from the projection midpoint 73 to the left boundary line is taken as the final left entry corner point, the closest point from the projection midpoint to the right boundary line is taken as the final right entry corner point, and the line segment between the final left entry corner point and the right entry corner point is taken as the front boundary line.

The included angle average value is an average value of included angles formed by a first point line pair and a second point line pair in the point line pairs. The projection midpoint is the midpoint between the left projection point and the right projection point. The space line may include a front boundary line, a left boundary line, and a right boundary line of the mechanical space.

As described above, the direction of the average value of the included angles formed by the first dot line pair and the second dot line pair in the dot line pair is used as the parking space direction of the mechanical parking space, the first dot line pair and the second dot line pair are respectively rotated to obtain the left boundary line and the right boundary line parallel to the parking space direction, recognition errors caused by angle deviation are reduced, the first entrance angular point and the second entrance angular point are respectively projected to the parking space direction to obtain the left projection point and the right projection point, and the boundary line before construction is provided with an accurate reference point for the boundary line before construction based on the projection midpoint between the left projection point and the right projection point, the left boundary line and the right boundary line, so that the detection precision of the mechanical parking space is improved.

In an embodiment, during the process of the self-vehicle driving into the mechanical parking space, the parking space detection system can continuously detect the end corner point and the gear position of the mechanical parking space in the latest multi-frame parking space image by using a deep learning model, image processing and other modes.

Based on the end corner point and the chock position, a rear boundary line is determined. For example, by connecting the end corner points and the chock positions, a rear boundary line is constructed using a linear interpolation or curve fitting method.

The end corner points are points of intersection of the rear boundary line of the mechanical parking space and the inner line of the parking space and are used for ending points of the mechanical parking space in the length direction, and the end corner points can comprise a left end corner point and a right end corner point of the mechanical parking space. The wheel block position is a specific position of the wheel block in the parking space image.

As described above, by detecting the end corner point in the multi-frame parking space image, determining the rear boundary line of the mechanical parking space based on the end corner point and the gear position of the mechanical parking space, marking the rear boundary by using the end corner point, and combining the gear position, the rear boundary line of the mechanical parking space can be determined more accurately.

To further describe the method for detecting a mechanical parking space, fig. 8 shows a flowchart of another method for detecting a mechanical parking space, where the method for detecting a mechanical parking space may include the following steps:

step 802, generating a first thermodynamic diagram and a first offset corresponding to internal line characteristics based on the internal line characteristics in the multi-frame parking space images.

In the step, the parking space detection system inputs each frame of parking space image into a point line detection model, and utilizes the point line detection model to carry out fusion high-precision inner line detection on multiple frames of parking space images, so as to extract inner line characteristics in the multiple frames of parking space images and generate a first thermodynamic diagram and a first offset corresponding to the inner line characteristics.

And 804, clustering and fitting the first candidate points along the parking space to obtain the inner line of the parking space.

In the step, a parking space detection system selects a first candidate point along the inner line of a parking space in a first thermodynamic diagram, adjusts the first candidate point by using a first offset, and clusters and fits the adjusted first candidate point to obtain the inner line of the parking space.

Step 806, generating a second thermodynamic diagram and a second offset corresponding to the point features based on the point features in the multi-frame parking space images.

In the step, the parking space detection system utilizes a point line detection model to perform fusion high-precision angular point detection on a plurality of frames of parking space images, extracts point features in the plurality of frames of parking space images, and generates a second thermodynamic diagram and a second offset corresponding to the point features.

Step 808, deleting the repeated points in the second candidate points of the entry corner point, and taking the second candidate points as the entry corner point.

In this step, the parking space detection system selects a second candidate point of the entry corner point in the second thermodynamic diagram, adjusts the second candidate point by using the second offset, deletes the duplicate point in the adjusted second candidate point, and uses the second candidate point as the entry corner point.

And 810, reserving an entrance angular point and a parking space inner line under the condition that the maximum visual angle difference is larger than a preset threshold value.

In the step, the parking space detection system determines the three-dimensional position of the dot line information according to the pose information of the vehicle and the external parameters of the monocular camera. And determining the maximum visual angle difference formed by the observation visual angles of the multi-frame parking space images based on the three-dimensional position of the dot line information.

And screening the spot line information based on the maximum visual angle difference, and reserving entrance angular points in the multi-frame parking space images and the inner line of the parking space under the condition that the maximum visual angle difference is larger than a preset threshold value.

Step 812, forming a dotted line pair from the first entrance corner point and the first inner line in the case that the first distance value is smaller than the distance threshold value.

In this step, the parking space detection system forms a dotted line pair between the first entrance angular point and the first inner line under the condition that a first distance value between the first entrance angular point and the first inner line is smaller than a distance threshold value.

Step 814, forming a dotted line pair from the second entry corner point and the second inner line when the second distance value is smaller than the distance threshold value.

In this step, the parking space detection system forms a dotted line pair between the second entrance angular point and the second inner line under the condition that a second distance value between the second entrance angular point and the second inner line is smaller than a distance threshold value.

Step 816, using the first and second closest point-line pairs as matching point-line pairs.

In this step, the parking space detection system searches for the second closest point-line pair to the right for each first point-line pair, and searches for the first closest point-line pair to the left for each second point-line pair. And when the distance value between the first point line pair and the second point line pair which are closest to each other exceeds the preset distance value, the first point line pair and the second point line pair which are closest to each other are used as matching point line pairs.

And 818, carrying out rectangular processing on the first point line pair and the second point line pair in the matched point line pair to obtain the parking space line corresponding to the matched point line pair.

In the step, the parking space detection system takes the direction of the average value of the included angles of the first point line pair and the second point line pair in the matched point line pairs as the parking space direction of the mechanical parking space. And rotating the first dot line pair to obtain a left boundary line parallel to the parking space direction. And rotating the second dot line pair to obtain a right boundary line parallel to the parking space direction.

And projecting the first entrance angular point in the first dot line pair to the parking space direction to obtain a left projection point, and projecting the second entrance angular point in the second dot line pair to the parking space direction to obtain a right projection point.

The nearest point from the projection midpoint to the left boundary line is taken as a final left entrance corner point, the nearest point from the projection midpoint to the right boundary line is taken as a final right entrance corner point, and a line segment between the final left entrance corner point and the right entrance corner point is taken as a front boundary line.

Step 820, determining the rear boundary line based on the end corner point and the gear position.

In the step, in the process of entering the mechanical parking space when the self-vehicle runs, the parking space detection system continuously detects the end corner point in the latest multi-frame parking space image and the gear position of the mechanical parking space, and determines a rear boundary line based on the end corner point and the gear position.

Fig. 9 is a schematic structural view of an electronic device according to an exemplary embodiment of the present application. The electronic device may be, for example, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a personal digital assistant, a server, a smart appliance, a car set, etc. Referring to fig. 9, at the hardware level, the electronic device includes a processor 902, an internal bus 904, a network interface 906, memory 908, and non-volatile storage 910, although other hardware required for other services is also possible. The processor 902 reads a corresponding computer program from the nonvolatile memory 910 into the memory 908 and then runs the computer program, and forms a detection device of the mechanical parking space on a logic level. Of course, other implementations, such as logic devices or combinations of hardware and software, are not excluded from the present application, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or logic devices.

Fig. 10 is a block diagram of a detection device for a mechanical parking space according to an exemplary embodiment of the present application. Referring to fig. 10, the apparatus may include a detection module 1002, a matching module 1004, and a processing module 1006, wherein:

The detection module 1002 is configured to detect multiple-frame parking space images to obtain dot line information in the multiple-frame parking space images, where the multiple-frame parking space images are obtained by shooting a mechanical parking space with a monocular camera, and the dot line information includes an entrance angular point of the mechanical parking space and a line in the parking space, and a direction along the line in the parking space is a direction along a parking space entrance to a parking space end;

the matching module 1004 is configured to match the entry corner point with the inner line of the parking space, so as to obtain a dotted line pair of the mechanical parking space;

the processing module 1006 is configured to perform rectangular processing on the pair of dotted lines, to obtain a parking space line of the mechanical parking space.

In one example, the matching module 1004 further includes determining a three-dimensional position of the point line information according to pose information of a vehicle and external parameters of the monocular camera before the point line pair of the mechanical parking space is obtained by matching the entrance angular point and the internal line of the parking space, determining a maximum viewing angle difference of the point line information based on the three-dimensional position, where the maximum viewing angle difference is a maximum angle difference formed according to an observation viewing angle of the multi-frame parking space image, and retaining the point line information if the maximum viewing angle difference is greater than a preset threshold.

In one example, the detection module 1002 is configured to detect a multi-frame parking space image to obtain dot line information in the multi-frame parking space image, and includes extracting features along an inner line in the multi-frame parking space image, generating a first thermodynamic diagram and a first offset corresponding to the features along the inner line, determining a first candidate point along the inner line of the parking space based on the first thermodynamic diagram and the first offset, and clustering and fitting the candidate point to obtain the inner line of the parking space.

In one example, the detection module 1002 is configured to detect a multi-frame parking space image to obtain dot line information in the multi-frame parking space image, and includes extracting point features in the multi-frame parking space image, generating a second thermodynamic diagram and a second offset corresponding to the point features, determining a second candidate point of the entry corner point based on the second thermodynamic diagram and the second offset, and deleting a repeated point in the second candidate point, where the second candidate point is used as the entry corner point.

In one example, the entrance corner point comprises a first entrance corner point and a second entrance corner point, the parking space inner line comprises a first inner line and a second inner line, the first inner line is the parking space inner line closest to the first entrance corner point, the second inner line is the parking space inner line closest to the second entrance corner point, the matching module 1004 is used for matching the entrance corner point with the parking space inner line to obtain a point line pair of the mechanical parking space, the point line pair comprises the first entrance corner point and the first inner line when a first distance value is smaller than a distance threshold value, the second entrance corner point and the second inner line are combined into the point line pair when a second distance value is smaller than the distance threshold value, and the first distance value is a distance value between the first entrance corner point and the first inner line, and the second distance value is a distance value between the second entrance corner point and the second inner line.

In one example, the dot line pairs include a plurality of first dot line pairs and a plurality of second dot line pairs, the processing module 1006, before being configured to perform a squaring process on the dot line pairs to obtain the parking space line of the mechanical parking space, further includes searching for each first dot line pair to the right for a second dot line pair closest to the first dot line pair, searching for each second dot line pair to the left for a first dot line pair closest to the first dot line pair, and using the first dot line pair and the second dot line pair closest to the first dot line pair as matching dot line pairs, and the processing module 1006, when being configured to perform a squaring process on the dot line pairs to obtain the parking space line of the mechanical parking space, includes performing a squaring process on the first dot line pair and the second dot line pair in each matching dot line pair to obtain the parking space line corresponding to each matching dot line pair.

In an example, the processing module 1006 is further configured to detect a dead-end corner point in the multi-frame parking space image and a gear position of the mechanical parking space, where the dead-end corner point is an intersection point between a line in the parking space and a rear boundary line of the mechanical parking space, and determine the rear boundary line based on the dead-end corner point and the gear position.

In one example, the parking space line includes a front boundary line, a left boundary line and a right boundary line, the processing module 1006, when configured to perform a squaring process on the pair of dot lines to obtain a parking space line of the mechanical parking space, includes projecting a direction of an average angle value, which is an average value of angles formed by a first dot line pair and a second dot line pair in the pair of dot lines, to the parking space direction, rotating the first dot line pair to obtain the left boundary line parallel to the parking space direction, rotating the second dot line pair to obtain the right boundary line parallel to the parking space direction, projecting a first entrance corner point in the pair of first dot lines to the parking space direction to obtain a left projection point, projecting a second entrance corner point in the pair of second dot lines to the parking space direction to obtain a right projection point, and constructing the front boundary line based on a projection midpoint, which is a midpoint between the left projection point and the right projection point.

The implementation process of the functions and roles of each unit in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.

For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present application. Those of ordinary skill in the art will understand and implement the present application without undue burden.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as a memory, comprising instructions executable by a processor of a detection device of a machine parking stall to implement a method as described in any of the above embodiments.

Wherein the non-transitory computer readable storage medium may be a ROM, random-access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and the application is not limited thereto.

In an exemplary embodiment, a computer program product is also provided comprising a computer program/instruction executable by a processor of a detection device of a machine parking space to implement a method as described in any of the above embodiments.

The foregoing describes certain embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The application is not limited to the precise construction which has been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

The foregoing description of the preferred embodiments of the application is not intended to be limiting, but rather to enable any modification, equivalent replacement, improvement or the like to be made within the spirit and principles of the application.

Claims

1. The method for detecting the mechanical parking space is characterized by comprising the following steps:

2. The method of claim 1, wherein prior to said matching said entry corner point with said in-car line to obtain a dotted pair of said mechanical car space, said method further comprises:

Determining the three-dimensional position of the point line information according to the pose information of the vehicle and the external parameters of the monocular camera;

determining the maximum visual angle difference of the dot line information based on the three-dimensional position, wherein the maximum visual angle difference is formed according to the observation visual angle of the multi-frame parking space image;

And under the condition that the maximum visual angle difference is larger than a preset threshold value, retaining the dotted line information.

3. The method of claim 1, wherein the detecting the multi-frame parking space image to obtain the dot line information in the multi-frame parking space image comprises:

extracting internal line characteristics in the multi-frame parking space images, and generating a first thermodynamic diagram and a first offset corresponding to the internal line characteristics;

Determining a first candidate point along the line in the parking space based on the first thermodynamic diagram and the first offset;

and clustering and fitting the candidate points to obtain the internal parking space line.

4. The method of claim 1, wherein the detecting the multi-frame parking space image to obtain the dot line information in the multi-frame parking space image comprises:

Extracting point features in the multi-frame parking space images, and generating a second thermodynamic diagram and a second offset corresponding to the point features;

determining a second candidate point of the entry corner point based on the second thermodynamic diagram and the second offset;

and deleting repeated points in the second candidate points, and taking the second candidate points as the entrance corner points.

5. The method of claim 1, wherein the entry corner points comprise a first entry corner point and a second entry corner point, and wherein the intra-space line comprises a first intra-space line and a second intra-space line, the first intra-space line being an intra-space line closest to the first entry corner point, and the second intra-space line being an intra-space line closest to the second entry corner point;

The matching of the entrance angular points and the inner line of the parking space is carried out to obtain a dotted line pair of the mechanical parking space, which comprises the following steps:

When the first distance value is smaller than the distance threshold value, the first entrance angular point and the first inner line form the dotted line pair;

when the second distance value is smaller than the distance threshold value, the second entrance angular point and a second inner line form the dotted line pair;

the first distance value is a distance value between the first entrance angular point and the first inner line, and the second distance value is a distance value between the second entrance angular point and the second inner line.

6. The method of claim 1, wherein the pair of dot lines comprises a plurality of first pair of dot lines and a plurality of second pair of dot lines;

before the rectangular processing is performed on the dotted line pair to obtain the parking space line of the mechanical parking space, the method further comprises:

For each first pair of points, searching right for a second pair of points nearest to the first pair of points;

for each second dot-line pair, searching the first dot-line pair closest to the left;

The first point line pair and the second point line pair which are closest to each other are used as matching point line pairs;

The rectangular treatment is carried out on the dotted line pair to obtain a parking space line of the mechanical parking space, and the rectangular treatment comprises the following steps:

And carrying out rectangular treatment on the first point line pair and the second point line pair of each matching point line pair to obtain the parking space line corresponding to each matching point line pair.

7. The method according to claim 1, wherein the method further comprises:

detecting a dead end corner point in the multi-frame parking space image and a gear position of the mechanical parking space, wherein the dead end corner point is an intersection point between a line in the parking space and a rear boundary line of the mechanical parking space;

and determining the rear boundary line based on the end corner point and the gear position.

8. The method of claim 1, wherein the space line comprises a front boundary line, a left boundary line, and a right boundary line;

taking the direction of the average value of the included angle as the parking space direction of the mechanical parking space, wherein the average value of the included angle is the average value of the included angle formed by the first point line pair and the second point line pair in the point line pairs;

rotating the first dotted line pair to obtain the left boundary line parallel to the parking space direction;

rotating the second dot line pair to obtain the right boundary line parallel to the parking space direction;

Projecting a first entrance angular point in the first point line pair to the parking space direction to obtain a left projection point;

projecting a second entrance corner point in the second dot line pair to the parking space direction to obtain a right projection point;

The front boundary line is constructed based on a projection midpoint, which is a midpoint between the left projection point and the right projection point, the left boundary line, and the right boundary line.

9. A device for detecting a mechanical parking space, the device comprising:

10. An electronic device, comprising:

A processor;

a memory for storing processor-executable instructions;

Wherein the processor is configured to implement the method of any of claims 1-8 by executing the executable instructions.

11. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the method of any of claims 1-8.

12. A computer program product having stored thereon a computer program/instruction which, when executed by a processor, implements the method according to any of claims 1-8.