WO2020160861A1 - Calibration of a sensor for a vehicle on the basis of object-side and image-side identification indices of a reference object - Google Patents

Abstract

Apparatus (10) for calibrating a sensor (22, 24, 26, 28) for a vehicle (21, 23), comprising an input interface (12) for inputting image data (18) containing at least one image of a predefined reference object (34, 42) that is recorded by a sensor (22, 24, 26, 28), wherein the reference object (34, 42) comprises a plurality of object-side reference parts (P1-P11, J1-J9), to each of which a world coordinate index and an object-side identification index are assigned; an image processing unit (14) for capturing a plurality of image-side reference parts (Q1-Q5, K1-K7) corresponding to the object-side reference parts (P1-P11, J1-J9), wherein the image processing unit (14) is designed to detect an image coordinate index and an image-side identification index linked to the particular object-side identification index for each of the image-side reference parts (Q1-Q5, K1-K7) in order to determine a correlation between the image coordinate indices and the world coordinate indices on the basis of the object-side and image-side identification indices; and an output interface (16) for outputting the detected image-side identification indices.

Description

Calibration of a sensor for a vehicle based on object-side and image-side identification indices of a reference object

The present invention relates to a device for calibrating a sensor for a vehicle according to claim 1. Furthermore, the invention relates to a sensor device for detecting the surroundings of a vehicle according to claim 13. The invention also relates to the use of such a device or such a sensor device in one Vehicle according to claim 14. The invention also relates to a method for calibrating a sensor for a vehicle according to claim 15. Finally, the invention relates to a computer program product according to claim 17.

Sensor systems that include one or more sensors are increasingly being used in vehicles, especially in their driver assistance systems and security systems, in order to detect the surroundings of the vehicle. Such sensors can, for example, be cameras, radars and / or light detection and ranging devices (hereinafter: lidars). Such a sensor is used to take pictures of the surroundings of the vehicle and to use an image evaluation to recognize objects that represent a potential obstacle for the vehicle when driving. The distance between the vehicle and the object can be determined on the basis of such an object detection in order to take a countermeasure if the distance falls below a critical rule, for example to brake the vehicle or to trigger a warning signal. In this way, collisions between the vehicle and objects, such as people, buildings, other vehicles or vegetation, can be avoided or at least the risk thereof can be reduced.

However, in order to achieve reliable object detection, the sensors must be calibrated. Calibration is a process in which the correlation between an image coordinate system, in which the images recorded by the respective sensor are displayed and which is specific for each sensor, and a world coordinate system, which is sensor-unspecific and generally valid for all sensors, is determined. The respective image coordinate system is therefore with the world coordinate system is set in a transmission ratio, based on which each image coordinate can be represented by means of a corresponding world coordinate.

Calibration devices and methods are known, for example, from WO 2018/000037 A1, which discloses a system and method for determining a camera position within a vehicle scene. The method there includes a first step in which an image of the vehicle scene is recorded by the camera.

The method further comprises a second step in which reference data are loaded which indicate the vehicle scene, the reference data comprising positions and orientations of known features within the vehicle scene. The known method also includes a third step in which the geometric appearance of one or more of the known features within the image are identified. Finally, the known method includes a fourth step in which the three-dimensional position and orientation of the camera relative to the known features that were identified in the third step are determined based on the geometric appearance and a camera position is calculated within the vehicle scene.

The known calibration devices and methods, however, have the disadvantage that the calibration of the sensors by means of these devices or processes cannot meet the ever-increasing demands on accuracy and reliability. For example, these calibration devices and methods are susceptible to obscuring object parts in which the reference objects cannot be completely detected by the sensor due to their position or other objects located in the field of view of the sensors. This can lead to incorrect distance measurements, which impair the functionality of the driver assistance and safety systems and cause safety-critical situations.

The object of the present invention is therefore to improve the known calibration devices and methods in such a way that their accuracy and reliability are increased. The object is achieved by a device for calibrating a sensor for a vehicle having the features of claim 1. Furthermore, the object is achieved by a sensor device for detecting the surroundings of a vehicle with the characteristics of claim 13. In addition, the object is achieved by Use of such a device in a vehicle with the features of claim 14. In addition, the object is achieved by a method for calibrating a sensor for a vehicle with the features of claim 15. Finally, the object is achieved by a computer program product with the features of claim 16.

The sensor, which is preferably attached to the vehicle, is designed to record an image of the reference object. The sensor is, for example, a camera, a radar sensor, a lidar sensor, an ultrasonic sensor or another imaging device of the device. The sensor can be integrated in a sensor system comprising a plurality of sensors that are, for example, placed in different positions of the vehicle. The multiple sensors include a camera, a radar sensor, a lidar sensor, an ultrasound sensor and / or another imaging device. In this case, the device according to the invention is preferably designed to calibrate the sensor system.

The image data generated by the sensor contain at least one image of the predefined reference object. The reference object has a certain geometric shape, for example a two-dimensional or three-dimensional shape, and / or a certain two-dimensional or three-dimensional pattern. The reference object is preferably circular, elliptical, square, hexagonal or sickle-shaped. Furthermore, the reference object preferably has a circular, elliptical or square core area which comprises a zigzag-shaped edge area with several corner areas which are each radially delimited by two sides intersecting at a corner point.

The reference object can be wholly or partially contained in the image (or in the images). This means that the respective images show the reference object of a certain perspective of the respective sensors out completely or partially.

The object-side reference parts are spatially or flatly distributed on the reference object (in the case of a two-dimensional reference object). The object-side reference parts are, for example, several reference areas of the surface of the three-dimensional reference object or several reference areas of the two-dimensional reference object. Alternatively or additionally, at least one of the object-side reference parts can comprise a reference point which is arranged in one of the reference areas. This also includes the case that a reference point is arranged in an edge section of a reference area or on an edge of a reference area. At least one reference point preferably comprises a corner point of the aforementioned zigzag shape.

The world coordinate index comprises coordinate information of the respective object-side reference parts. Here, the coordinate information relates to a world coordinate system. The world coordinate system is a reference system that differs from the image coordinate system of the sensor. The world coordinate system can be, for example, a coordinate system of the vehicle, the surroundings of the vehicle, the sensor housing or another housing. The world coordinate system can be a Cartesian or a spherical coordinate system. The world coordinate system is preferably selected such that one of the coordinates, for example along the z-axis, assumes the value zero for several of the object-side reference parts of the reference object (z = 0).

In the event that an object-side reference part is a reference area, the world coordinate index includes, for example, coordinate information of a point located in this reference area or of the world coordinate system.

In the event that an object-side reference part is a reference point, the world coordinate index includes coordinate information of the reference point or the world coordinate system. The object-side identification index is used to specify the object-side reference parts in addition to their world coordinate index. The object-side identification index relates to one or more properties of the respective object-side reference parts. For example, the object-side identification index can relate to a position, orientation, shape and / or an appearance property of the respective object-side reference parts.

The world coordinate index and / or the object-side identification index can at least partially belong to the image data. Alternatively or additionally, the world coordinate index and / or the object-side identification index can belong to a data packet which is separate from the image data and which can be fed to the image processing unit, for example, via the input interface.

By recording the predefined reference object by means of the sensor, the object-side reference parts are mapped into the image-side reference parts. In this respect, the reference parts on the image side correspond to the reference parts on the object side.

The image processing unit is designed to take the predefined reference object, in particular the image-side reference parts, from the image data. The object-side reference parts are determined by the design of the reference object.

The image-side reference parts can be different from image to image depending on the perspective of the sensor used and external parameters such as brightness.

The image coordinate index comprises coordinate information of the respective reference parts on the image side. The coordinate information relates to an image coordinate system. The image coordinate system is a reference system relative to which the image points of the image recorded by the sensor, for example pixels and / or voxels, are spatially or flatly defined on an image display surface. The coordinate information in the image coordinate system can include one or more coordinates of the image coordinate system.

In the event that a reference part on the image side is a reference area, the image coordinate index includes, for example, coordinate information of one in this Reference area of the lying point in the image coordinate system. In the event that a reference part on the image side is a reference point, the image coordinate index includes coordinate information of the reference point in the image coordinate system.

The correlation between the world coordinate indices and the image coordinate indices corresponds to the transmission ratio between the world coordinate system and the image coordinate system. The calibration device according to the invention is thus designed to determine a correlation between the world coordinate system and the image coordinate system, based on which coordinates of one of the two coordinate systems can be translated into the other of the two coordinate systems and / or vice versa.

The correlation can be determined by the image processing unit. Alternatively or additionally, the correlation can be determined by an external processing unit, for example a cloud system, to which the object-side and image-side identification indices can be supplied, based on these.

The determined correlation can be used to determine spatial parameters such as positions and / or orientations of the sensor or the sensors in the world coordinate system (so-called extrinsic parameters). For example, the determined correlation can be further optimized in a further information processing step by means of the non-linear optimization to be carried out by the image processing unit or an additional processing unit. As an alternative or in addition, so-called intrinsic parameters can be used by the image processing unit or the additional computing unit in order to determine the extrinsic parameters of the sensor.

The calibration of the sensor is improved by taking into account the identification indices of the reference parts. In particular, calibration can be carried out with sufficient accuracy and reliability even if the reference object is partially covered or only a small part of the reference parts of the reference object can be detected by the sensor. In this case, the hidden reference parts cannot be taken into account in the calibration. This information deficit can can be advantageously compensated by the additional information contained in the identification indexes of the non-covered reference parts.

The calibration device according to the invention is designed to be used in an offline calibration. A calibration platform (for example a vehicle), which is provided with a sensor system to be calibrated, is arranged in a location in which at least one reference object is in the field of view of the sensor system. The image data generated by the sensor system can be fed to the device according to the invention for the purpose of calibration. The calibration device can be attached to the calibration platform. Alternatively, the calibration device can be arranged externally.

Alternatively or in addition, the calibration device according to the invention can be used in an online calibration. The calibration platform is the vehicle on which the sensor system to be calibrated is attached. The vehicle can travel along a route in the vicinity of which at least one reference object is arranged in the field of view of the sensor system.

Advantageous refinements and developments are given in the subclaims.

In a preferred embodiment, the object-side identification index comprises an object-side distance, measured in the world coordinate system, of the respective object-side reference parts to a predefined object-side reference point, the image-side identification index comprising an image-side distance measured in the image coordinate system between the respective image-side reference parts and an image-side reference point determined by the image processing unit.

The predefined object-side reference point is a reference point in the world coordinate system from which the object-side distance for the respective object-side reference parts is measured. The object-side reference point is preferably marked on the reference object, so that the image-side reference point can be determined in a simplified manner by the image processing unit in the image of the reference object. If the ref- reference object assumes a circular shape, an elliptical shape or a square shape (for example with zigzag-shaped edge areas), the object-side reference point is preferably the center of the circular, elliptical or square shape.

If an object-side reference part is a reference point, the object-side distance is preferably the distance from the reference point to the object-side reference point. If an object-side reference part is a reference area, the object-side distance is preferably the distance from a center point, for example the geographical center point, or a focus of the reference area to the object-side reference point.

The image-side reference point is a reference point in the image coordinate system from which the image-side distance for the respective image-side reference parts is measured. The image-side reference point can be made, for example, by comparing the image of the reference object with a circular shape, an elliptical shape or a square in the image coordinate system generated by the image processing unit or a computing unit that interacts with it. This is advantageous if the reference object has a circular, elliptical or square shape, the outer edge of which is, for example, zigzag-shaped. By varying the size and / or position of the circular, elliptical or square shape, the image-side reference point can be identified particularly easily. For example, the image-side reference point can be identified as the center point of the circular, elliptical or square shape, in which the circular, elliptical or square shape corresponds most to the shape of the image of the reference object.

Advantageously, the object-side and image-side distances provide geometric information about the reference object that is used to calibrate the sensor to increase the accuracy. In a further preferred embodiment, the image processing unit is designed to carry out a homographic transformation on the captured image-side reference parts.

The homographic transformation of the reference parts on the image side serves to obtain a new image of the reference object, specifically in such a way as if the image had been recorded from a different perspective. For example, the image-side reference parts are projected onto a plane to be determined by the image processing unit. In this way, unwanted visual effects such as distortions and / or rotations of the reference object in the image can at least be reduced. The homographic transformation can preferably be carried out before the image-side identification indices and / or the image coordinates of the image-side reference parts are detected. This is particularly advantageous because, for example, the definition of the reference point on the image side is simplified in that the shape of the reference object is retained in the new image due to the elimination of the distortions or rotations.

In a further preferred embodiment, the predefined reference object has several, preferably at least four, colors.

In addition to the world coordinates, the color information of the reference object can advantageously be used to calibrate the sensor to increase the accuracy. The use of four colors enables calibration with reduced computing effort while maintaining reliability.

In a further preferred embodiment, the object-side identification index comprises an object-side color index of the respective object-side reference parts, the image-side identification index including an image-side color index of the respective image-side reference parts determined in a color index system of the sensor.

The reference object has several colors so that a specific color is assigned to each object-side reference part. For example, the reference object is related or its surface is divided into several reference areas, each reference area being assigned a color. In the event that a reference part is a reference area, the object-side color index of this reference part is the color of the associated reference area. In the event that a reference part is a reference point, the object-side color index of this reference part is the color of the reference area in which the reference point (for example a corner point) is located. The object-related color index can, for example, be a color designation (for example “red”, “green” or “blue”). Alternatively or additionally, the object-side color index can be based on an object-side color index system (such as the RGB system). The colors are to be understood in such a way that two different configurations of the same superordinate color with different contrast values (for example color configurations “dark blue” and “light blue” for the superordinate color “blue”) are to be regarded as two different colors. The image-side color indices are preferably created when the object-side color indices are detected by the sensor. The color indices on the image side are thus preferably based on a color index system of the sensor, for example an RGB system. The calibration of the sensor is therefore more precise thanks to the color information.

In a further preferred embodiment, the image processing unit is designed to determine an image-side sequence of the image-side identification indices along a course of the image-side reference parts.

The image processing unit is designed, for example, to first define the course of the image-side reference parts and then to determine the image-side sequence of the image-side identification indices. Here, for example, the image coordinate indices of the image-side reference parts result in the course in the image coordinate system. Alternatively or additionally, the course can be in the form of a numbering of the image-side reference parts.

In a further preferred embodiment, the image processing unit is designed to receive an object-side sequence of the object-side identification indices along a course of the object-side reference parts. Here, for example, the world coordinate indices of the object-side reference parts result in the course in the world coordinate system. Alternatively or additionally, the course can be present in the form of a numbering of the object-side reference parts.

In a further preferred embodiment, the image processing unit is designed to compare the image-side sequence with the object-side sequence in order to determine a correlation value between the image-side and object-side sequence.

The correlation value indicates the extent to which the object-side identification indices of the object-side sequence correspond to the image-side identification indices of the image-side sequence. The correlation value is based, for example, on the number of object-side identification indices in the object-side sequence, the sequence of which corresponds to a sequence of the image-side identification indices of the image-side sequence.

In the event that the object-side identification index is an object-side distance, the correlation value indicates the extent to which the image-side sequence of the image-side distances can be obtained in terms of amount (i.e. dimensionless) by multiplying the object-side sequence of the object-side distances by a common factor. The image processing unit is preferably designed to display the object-side and image-side distances in each case as a unit of an object-side or image-side distance. In this case, a comparison between the object-side distances and the image-side distances is dimensionless and simplified.

In a further preferred embodiment, the image processing unit is designed to determine the correlation value for several object-side sequences in each case along an associated course of the object-side reference parts in order to identify a maximum and / or an extreme of the correlation value.

Under certain circumstances, the number of reference parts on the image side can be less than the number of reference parts on the object side. This is the case, for example, when the reference object is positioned during the image acquisition by the sensor of the sensor or the reference object is partially covered by an obstacle. In this case, there can be more object-side sequences with the same number of identification indices as in the image-side sequence with which the image-side sequence can be compared. The correlation value can vary depending on the sequence on the property side. The maximum or extreme indicates a high degree of correspondence between the sequence on the image side and the sequence on the object side. This reduces the risk of an incorrect choice of the object-side sequence on which the correlation to be determined between the world coordinate system and the image coordinate system is based. The calibration is therefore more reliable.

In a further preferred embodiment, the image processing unit is designed to determine the correlation between the image coordinate indices and the world coordinate indices based on an odometric correction factor.

This measure provides a reference variable with which the result of the calibration can be compared in order to reduce any errors in the calibration of the sensor. Odometric data are preferably fed in parallel to the image data or as part of the image data via the input interface of the image processing unit. In this case, the detection of the reference object from the image data can also take into account the odometric data, so that the calibration is fail-safe from an early phase on.

In a further preferred embodiment, the image data contain a plurality of images which are each recorded synchronously by the sensor and by at least one further sensor at a plurality of times.

At each of the multiple points in time, the sensors take an image synchronously. This creates an image sequence which contains several groups of images, the images of each of these groups being recorded synchronously by the sensors at a specific point in time and the groups being offset in time with respect to one another. The image processing unit is preferably designed to process the plurality of images in such a way that, taking into account the time sequence of the images, to perform the calibration. Compared to images recorded only at a single point in time, the temporal sequence of the images provides additional information that can be used when recognizing the reference parts on the image side and their spatial and temporal information.

In a further preferred embodiment, the image processing unit comprises an artificial neural network module.

The artificial neural network module (ANN module) is based on an artificial neural network (ANN), which comprises one or more algorithms. The algorithms are trained with data in such a way that it is suitable to be used in the image processing unit. This enables a reliable calibration method.

In a further preferred embodiment, the artificial neural network module is designed to classify the reference object against a background in the image.

This is done for example by means of image segmentation, preferably semantic segmentation, which is particularly simple and precise.

In a further preferred embodiment, the artificial neural network module is designed to detect the image-side identification indices and / or the image coordinate indices and / or to determine the correlation between the image coordinate indices and the world coordinate indices.

This enables an at least partially automated calibration method, which simplifies the calibration of the sensor with at least the same accuracy.

The training device according to the invention is designed to train an ANN so that it is able to capture several image-side reference parts based on image data that contain at least one sensor-generated image of a predefined reference object, with one image-side identification in each case - dex and an image coordinate index, and / or based on the object-side and image-side identification indices to determine a correlation between the image coordinate indices and the world coordinate indices.

The computer program product according to the invention is designed to be loaded into a memory of a computer and comprises software code sections with which the method steps of the method according to the invention for calibrating a sensor or the method steps of the method according to the invention for training an artificial neural network are carried out when the compu program product is running on the computer.

A program is part of the software of a data processing system, for example an evaluation device or a computer. Software is a collective term for programs and associated data. The complement to software is hardware. Hardware describes the mechanical and electronic alignment of a data processing system. A computer is an evaluation device.

Computer program products usually comprise a sequence of instructions which, when the program is loaded, cause the hardware to carry out a specific method that leads to a specific result. If the program in question is used on a computer, the computer program product produces a technical effect, namely to increase the accuracy and reliability of the calibration of the sensor

The computer program product according to the invention is platform independent. That means it can run on any computing platform. The computer program product is preferably executed on a device according to the invention for calibrating a sensor for a vehicle or on a device according to the invention for training an artificial neural network.

The software code sections are written in any programming language, for example in Python. The invention is illustrated by way of example in the figures. Show it:

1 shows a schematic representation of an inventive device for calibrating a sensor for a vehicle according to an exemplary embodiment;

2 shows a schematic representation of a sensor system comprising a plurality of sensors;

3 shows a schematic representation of a reference object comprising a plurality of reference points;

FIG. 4 shows a schematic representation of an image of the reference object from FIG. 3 taken by one of the sensors from FIG. 2;

5 shows a schematic representation of a reference object comprising a plurality of reference points;

6 shows a schematic illustration of an image of the reference object from FIG. 5 taken by one of the sensors from FIG. 2;

7 shows a schematic representation of a method for calibrating the sensor according to an exemplary embodiment; and

Fig. 8 is a schematic representation of a sensor device according to an Ausfüh approximately example.

In the figures, the same reference symbols relate to the same or functionally similar reference parts. The relevant reference parts are identified in the individual figures.

1 shows a schematic representation of a device 10 according to the invention for calibrating a sensor 22, 24, 26, 28 for a vehicle 21 according to an embodiment leadership example. The sensor 22, 24, 26, 28 and the vehicle 21 are shown schematically in FIG. 2.

The device 10 has an input interface 12 for inputting image data 18 which contain at least one image, but here, for example, several images 19-1, 19-2, 19-3,... 19-n. The image 19-1, 19-2, 19-3, ... 19-n is recorded by the sensor 22, 24, 26, 28 and comprises a predefined colored reference object 34, 42 with several object-side reference parts P1 -P1 1, J1-J9, as shown schematically in FIGS. 3 and 5 by way of example. Each of the object-side reference parts P1 -P1 1, J1-J9 are assigned a world coordinate index and an object-side identification index.

The device 10 also has an image processing unit 14 which is designed to detect several image-side reference parts Q1 -Q5, K1-K7 (see FIGS. 4 and 6) going back to the object-side reference parts P1-P11, J1 -J9. In addition, the image processing unit 14 is designed to detect an image coordinate index and an image-side identification index for each of the image-side reference parts Q1 -Q5, K1 -K7 in order to establish a correlation between the image coordinate indices of the image-side reference parts based on the object-side identification indices and the image-side identification indices Q1 -Q5, K1 -K7 and the world coordinate indices of the associated object-side reference parts P1 -P1 1, J1 -J9 to be determined.

The device 10 furthermore has an output interface 16 for outputting the determined correlation to an external entity. The external entity shown in Fig.

1 is not shown, a further unit for performing further information processing steps can be. For example, the further unit can be designed to carry out a non-linear optimization of the determined correlation and / or to obtain extrinsic parameters for the sensor 22, 24, 26, 28 to be calibrated from the determined correlation using intrinsic parameters.

2 shows a schematic representation of a sensor system 25 comprising a plurality of sensors 22, 24, 26, 28. The sensors 22, 24, 26, 28 can comprise a camera, a radar sensor, a lidar sensor and / or an ultrasonic sensor. As shown schematically and by way of example in FIG. 2, the reference object 34 is partially in the field of view of the sensor 22. The detection of the reference object 34, 42 results in an image 19 of the reference object 34, 42, which is shown schematically in more detail in FIG.

3 shows a schematic representation of a reference object 34. Here, the object-side reference parts are shown as reference points P1-P11 by way of example. The reference points P1 -P1 1 are each located in a reference area R1 -R1 1 into which the surface of the reference object 34 is divided. In Fig. 3, the reference areas R1 -R1 1 are shown by way of example. At least four colors can be used, whereby adjacent reference areas R1 -R1 1 can have the same or different colors.

An associated world coordinate index is assigned to each of the reference points P1 -P11 based on their position in a world coordinate system 36. The world coordinate system 36 is shown by way of example in FIG. 3 as a Cartesian coordinate system. Alternatively, another, for example a spherical, coordinate system can also be used for this purpose.

In addition, an object-side color index is assigned to each of the reference points P1 -P1 1. For example, the color of a reference area R1 -R1 1 is assigned to the associated reference point P1 -P1 1.

FIG. 4 shows a schematic representation of the image 19 of the reference object 34 from FIG. 3 recorded by the sensor 22, 24, 26, 28 from FIG. 2 . Correspondingly, a part of the reference parts Q1 -Q5 on the image side can be seen as reference points and their associated reference areas T1 -T5 in image 19. Each of the image-side reference points Q1 -Q5 corresponds to one of the object-side reference points P1 -P1 1, each of the image-side reference areas T 1 -T5 corresponding to one of the object-side reference areas R1 -R1 1. For each of the image-side reference points Q1 -Q5, an image coordinate index is identified in an image coordinate system 38 when the reference object 34 is detected by the sensor 22. In Fig. 4, a coordinate system with three image axes for the image coordinates u, v and w is shown as the image coordinate system. However, this is not limitative of the present invention. Any desired, for example a two-dimensional image coordinate system can be used.

For each of the image-side reference points Q1 -Q5, an image-side color index that goes back to the object-side color indices of the object-side reference points P1 -P1 1 is identified when the reference object 34 is detected by the sensor 22. This is done on the basis of a sensor's own color index system, such as the RGB system.

The image processing unit 14 is designed to extract (detect) the image coordinate indices and the image-side color indices of the image-side reference points Q1 -Q5 from the image data 18.

5 shows a schematic representation of a further reference object 42. Here, the object-side reference parts are shown by way of example as reference points J1-J9. The reference points J1 -J9 each lie in a reference area M1 -M9 into which the surface of the reference object 42 is divided. The reference areas M1 -M9 are shown by way of example in FIG.

An associated world coordinate index is assigned to each of the reference points J1 -J9 based on their position in the world coordinate system 36. The world coordinate system 36 is shown in Fig. 3 by way of example as a Cartesian coordinate system. Alternatively, a different, for example a spherical, coordinate system can be used for this purpose.

Furthermore, an object-side distance d 1, d5, d7 is assigned to each of the reference points J1-J9. For reasons of clarity, not all object-side distances are marked with a reference symbol in FIG. 5. The respective object-side distance d 1, d5, d7 is measured in the world coordinate system 36, with this is the distance between an object-side center point A and the respective object-side reference point J1 -J9. The object-side center point A is here, for example, the center of a circle 44 from which the reference areas M1 -M9 extend radially outward to the respective reference point J1 -J9 with decreasing width.

FIG. 6 shows a schematic representation of the image 27 of the reference object 42 from FIG. 5 recorded by the sensor 22, 24, 26, 28 from FIG. 2. The image 27 shows a part of the reference object 42 which is in the field of vision of the sensor 22 on the object side . Correspondingly, part of the image-side reference parts K1-K7 as reference points and their associated reference areas N1-N7 can be seen in image 27. Each of the image-side reference points K1 -K7 corresponds to one of the object-side reference points J1 -J9, each of the image-side reference areas N1 -N7 corresponding to one of the object-side reference areas M1 -M9.

For each of the image-side reference points K1 -K7, an image coordinate index is identified in an image coordinate system 38 when the reference object 42 is detected by the sensor 22. In Fig. 4, a coordinate system with three image axes for the image coordinates u, v and w is shown as the image coordinate system. However, this is not limitative of the present invention. Any desired, for example a two-dimensional image coordinate system can be used.

For each of the image-side reference points K1 -K7, an image-side distance 11, 15, I7 that goes back to the object-side distances of the object-side reference points J1 -J9 is detected by the image processing unit 14. The respective image-side distance 11, I5, 17 is measured in the image coordinate system 38, this being the distance between an image-side center point B and the respective reference point K1 -K7. The image-side center point B is shown by way of example as the center point of the circle 46 of the image of the reference object 42. The circle 46 here goes back to the circle 44 of the reference object 42 in FIG. 5. The image processing unit 14 is designed to extract (detect) the image coordinate indices and the image-side distances of the image-side reference points K1 -K7 from the image data 18.

7 schematically shows a method 100 according to the invention for calibrating the sensor 22, 24, 26, 28. In a first step S1, image data 18 are entered which contain at least one image 19-1, 2, recorded by a sensor 22, 24, 26, 28 , 3, ..., n of a predefined reference object 34, 42. The reference object 34 comprises several object-side reference parts P1 -P1 1, J1 -J9, each of which is assigned a world coordinate index. In a second step S2, several image-side reference parts Q1 -Q5, K1-K7 are acquired from the image data, which go back to the several object-side reference parts P1-P11, J 1 -J9. In addition, an image coordinate index and an image-side identification index are detected from the image data for each of the captured image-side reference parts Q1 -Q5, K1 -K7. In a third step S3, the image-side identification indexes are output.

Based on the object-side and image-side identification indices, a correlation between the image coordinate indices of the image-side reference parts Q1 -Q5, K1 -K7 and the world coordinate indices of the object-side reference parts P1-P1 1, J1 -J9 is then determined.

In order to determine the correlation between the image coordinates of the image-side reference points Q1 -Q5, K1 -K7 and the world coordinates of the associated object-side reference points P1 -P1 1, J1 -J9, the image processing unit 14 is preferably designed to provide an image-side sequence of the image-side identification indices to be determined. This image-side sequence is the sequence of the image-side identification indices of the image-side reference points.

For the image 19 shown in FIG. 4, this image-side sequence corresponds, for example, to the (image-side) course of the image-side reference points Q1 -Q2-Q3-Q4-Q5 (in this order). The associated image-side identification indices here are image-side color indices FB1, FB2, FB3, FB4, FB5. Therefore it is the image-side Sequence for the example shown in Fig. 4 around the color index sequence FB1 -FB2-FB3-FB4-FB5.

For the image shown in FIG. 6, the image-side sequence corresponds, for example, to the image-side course K1 -K2-K3-K4-K5-K6-K7 (in this order). The associated image-side identification indices here are image-side distances 11, 12, I3, 14, 15, I6, I7. Therefore, the image-side sequence for the example shown in FIG. 6 is the distance sequence 11 -I2-I3-I4-I5-I6-I7.

The image processing unit 14 is also preferably designed to determine an object-side sequence of the object-side identification indices.

The object-side sequence for the reference object 34 shown in FIG. 3 corresponds, for example, to any (object-side) course of the object-side reference points P1 -P1 1, with the number of object-side reference points P1 -P1 1 being the number of image-side reference points Q1 to Q5 preferably corresponds to five. For example, the object-side sequence can correspond to the object-side course P1-P2-P3-P4-P5 (in this order). The associated object-side identification indices are object-side color indices F01, F02, F03, F04, F05. The exemplary object-side sequence for the example shown in FIG. 3 is therefore the color index sequence F01-F02-F03-F04-F05.

The object-side sequence for the reference object 42 shown in FIG. 5 corresponds, for example, to any (object-side) course of the object-side reference points J1 -J9, with the number of object-side reference points J1 -J9 preferably corresponding to the number of image-side reference points K1-K7 and thus is seven. For example, the object-side sequence can correspond to the object-side course J1 -J2-J3-J4-J5 (in this order). The associated object-side identification indices are object-side distances d1, d2, d3, d4, d5, d6, d7. The exemplary object-side sequence for the example shown in FIG. 5 is therefore the distance sequence d1-d2-d3-d4-d5-d6-d7. Since the number of reference points on the object side is greater than the number of reference points on the image side, there can be several possible object-side sequences. For example, the object-side sequence for the example shown in FIG. 3 can be F06-F07-F08-F09-F010. For example, the object-side sequence for the example shown in FIG. 5 can be d3-d4-d5-d6-d7-d8-d9.

The image processing unit 14 is preferably designed to compare the image-side sequence with each of the multiple possible object-side sequences and to determine a correlation value in each case. The correlation value is preferably measured by the extent to which an object-side sequence corresponds to the image-side sequence.

For the example shown in Fig. 3-4, the image-side color index sequence FB1 -FB2-FB3-FB4-FB5 is compared with the object-side color index sequence F01 -F02-F03-F04-F05. The image-side and object-side color indices are preferably translated into a common color index system so that a direct comparison is possible. If the order of the two color index sequences is the same for all five color indexes, the correlation value is the highest. If the order of the two color index sequences is the same for four of the five color indexes, the correlation value is lower. The higher the number of color indices in which the two color index sequences match one another, the higher the correlation value.

For the example shown in Fig. 5-6, the image-side distance sequence 11 -I2-I3-I4-I5-I6-I7 is compared with the object-side distance sequence d3-d4-d5-d6-d7-d8-d9. The two distance sequences are preferably compared with one another in terms of absolute value or dimensionlessly, so that a direct comparison between the distance sequences is possible. If the two distance sequences agree with one another for all seven distances in their order except for a factor common to all seven values, the correlation value is the highest. If the two distance sequences in six of the seven distances agree in their order except for the common factor, the correlation value is lower. The higher the number of distances in which the two distance follow each other except for a common factor, the higher the correlation value.

The image processing unit 14 is preferably designed to change the object-side profile by one or more object-side reference parts after determining the correlation value and then to determine a correlation value corresponding to the changed object-side profile. This process can continue iteratively until a maximum or extreme of the correlation value is determined.

The object-side sequence with the highest correlation value is preferably used as the basis for determining the correlation. Here, the image coordinate indices of the image-side reference parts are related to the world coordinate indices of the object-side reference parts of this particular object-side sequence in order to obtain a transmission ratio between the two coordinate systems.

8 schematically shows a sensor device 50 comprising the sensor system 25 shown in FIG. 2 with the multiple sensors 22, 24, 26, 28 and the calibration device 10 shown in FIG. 1 for calibrating the sensors 22, 24, 26, 28. The sensor device 50 is attached to a calibration platform 23, which preferably includes a vehicle. An offline and / or online calibration is thus possible by means of the calibration device 10 according to the invention.

Reference sign

10 calibration device

12 input interface

14 Image processing unit

16 Output interface

18 image data

19, 27 fig

19-1, -2, ... -n images

21 vehicle

22, 24, 26, 28 sensors

23 calibration platform

25 sensor system

32 Field of View

34, 42 reference object

36 World coordinate system

38 Image coordinate system

44 Circular shape of the reference object

46 Circular shape of the image of the reference object

50 sensor device

100 calibration procedures

P1 to P11, J1 to J9

object-side reference points

R1 to R11, M1 to M9

object-side reference areas

Q1 to Q5, K1 to K7

reference points on the image side

T1 to T5, N1 to N7

Image-side reference areas

S1 to S3 process steps

Claims

Claims 1 . Device (10) for calibrating a sensor (22, 24, 26, 28) for a vehicle (21, 23), comprising: an input interface (12) for entering image data (18) containing at least one image of a predefined reference object (34, 42) recorded by a sensor (22, 24, 26, 28), the reference object (34, 42) having several whether the project-side reference parts (P1 -P11, J1 -J9), which are each assigned a world coordinate index and an object-side identification index; an image processing unit (14) for capturing a plurality of image-side reference parts (Q1 -Q5, K1-K7) corresponding to the object-side reference parts (P1 -P1 1, J1 -J9), the image processing unit (14) being designed for each of the image-side reference parts gen reference parts (Q1 -Q5, K1 -K7) to detect an image coordinate index and an image-side identification index linked to the respective object-side identification index in order to determine a correlation between the image coordinate indices and the world coordinate indices based on the object-side and image-side identification indices; and an output interface (16) for outputting the detected image-side identification indices. 2. Device (10) according to claim 1, wherein the object-side identification index comprises a measured object-side distance in the world coordinate system of the respective object-side reference parts (J1 -J9) to a predefined object-side reference point (A), the image-side identification index being a measured in the image coordinate system image-side distance of the respective image-side reference parts (K1 -K7) to an image-side reference point (B) determined by the image processing unit (14). 3. Device (10) according to claim 1 or 2, wherein the image processing unit (14) is designed to carry out a homographic transformation on the detected image-side reference parts (K1 -K7). 4. Device (10) according to one of claims 1 to 3, wherein the predefined Refe renzobject (34) is colored and carries several, preferably at least four, colors. 5. The device (10) according to claim 4, wherein the object-side identification index egg NEN object-side color index of the respective object-side reference parts (P1 -P11), the image-side identification index an image-side determined in a color index system of the sensor (22, 24, 26, 28) Includes color index of the respective reference parts on the picture (Q1-Q5). 6. Device (10) according to one of claims 1 to 5, wherein the image processing unit (14) is designed to determine an image-side sequence of the image-side identification sindizes along a course of the image-side reference parts (Q1 -Q5, K1 -K7) . 7. The device (10) according to claim 6, wherein the image processing unit (14) is formed to determine an object-side sequence of the object-side identification indices along a course of the object-side reference parts (P1-P11, J1 -J9). 8. The device (10) according to claim 7, wherein the image processing unit (14) is formed to compare the image-side sequence with the object-side sequence in order to determine a correlation value between the image-side and object-side sequence. 9. The device (10) according to claim 8, wherein the correlation value is based on a number of the object-side identification indices of the object-side sequence, the order of which corresponds to a sequence of the image-side identification indices of the image-side sequence. 10. The device (10) according to claim 8 or 9, wherein the image processing unit (14) is designed to determine the correlation value for several object-side sequences in each case along an associated course of the object-side reference parts (P1-P11, J1 -J9) in order to achieve a To identify maximum and / or an extremum of the correlation value. 1 1. Device (10) according to one of claims 1 to 10, wherein the image processing unit (14) is designed to determine the correlation between the image coordinate indices and the world coordinate indices based on an odometric correction factor. 12. Device (10) according to one of claims 1 to 11, wherein the image data contain a plurality of re images that are each recorded synchronously by the sensor (22, 24, 26, 28) and by at least one wide Ren sensor at several times. 13. Sensor device (50) for detecting the surroundings of a vehicle (21, 23), comprising: a sensor (22, 24, 26, 28) for recording an image of a predefined reference object (34, 42); and a device (10) according to one of claims 1 to 1 2 for calibrating the sensor (22, 24, 26, 28). 14. Use of a device (10) according to one of claims 1 to 12 and / or a sensor device (50) according to claim 13 in a vehicle (21, 23). 15. A method for calibrating a sensor (22, 24, 26, 28) for a vehicle (21, 23), comprising: a first step (S1), in which image data (18) are entered which contain at least one image of a predefined reference object (34, 42) recorded by a sensor (22, 24, 26, 28), the reference object (34 42) comprises several object-side reference parts (P1 -P1 1, J1 -J9), each of which is assigned a world coordinate index and an object-side identification index; a second step (S2), in which several image-side reference parts (Q1 -Q5, K1 -K7) corresponding to the object-side reference parts (P1 -P1 1, J1 -J9) are detected, the image processing unit (14) being designed to for each of the image-side reference parts (Q1 -Q5, K1 -K7) an image coordinate index and an image-side identification index linked to the respective object-side identification index to be detected, based on the object-side and image-side identifiers tion indices to determine a correlation between the image coordinate indices and the world coordinate indices; and a third step (S3) in which the detected image-side identification indices are output. 16. Computer program product that is stored in a data carrier and is designed to carry out the method according to claim 15 when the computer program product is executed by a computer.