[go: up one dir, main page]

WO2016157802A1 - Appareil de traitement d'informations, système de traitement d'informations, procédé de traitement d'informations et support d'informations - Google Patents

Appareil de traitement d'informations, système de traitement d'informations, procédé de traitement d'informations et support d'informations Download PDF

Info

Publication number
WO2016157802A1
WO2016157802A1 PCT/JP2016/001595 JP2016001595W WO2016157802A1 WO 2016157802 A1 WO2016157802 A1 WO 2016157802A1 JP 2016001595 W JP2016001595 W JP 2016001595W WO 2016157802 A1 WO2016157802 A1 WO 2016157802A1
Authority
WO
WIPO (PCT)
Prior art keywords
coordinate conversion
coordinates
coordinate
conversion table
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2016/001595
Other languages
English (en)
Japanese (ja)
Inventor
高橋　徹
哲夫 井下
大輔 西脇
吉洋 服部
達哉 安部
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2017509251A priority Critical patent/JP6683195B2/ja
Publication of WO2016157802A1 publication Critical patent/WO2016157802A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models

Definitions

  • the present invention relates to information processing, and more particularly, to an information processing apparatus, an information processing system, an information processing method, and a recording medium that process position information.
  • a wide area monitoring system generally detects a vehicle and / or a person from a captured image and grasps the position of the vehicle and / or the person.
  • the wide area monitoring system preliminarily calculates the coordinates of each pixel in the camera coordinate system in which the vehicle is captured, the coordinates in the world coordinate system, and Must be associated with each other. That is, the wide area monitoring system needs to calculate the coordinate conversion between the coordinates of each pixel in the camera coordinate system and the coordinates of the world coordinate system.
  • the road surface on which the vehicle travels is often not a simple plane.
  • the road surface shape projected on the camera coordinate system is complicatedly distorted. Therefore, the coordinate transformation in this case cannot be expressed using a simple linear transformation. Therefore, accurate association between the camera coordinate system and the world coordinate system is not easy.
  • the correspondence table is a table for converting a two-dimensional coordinate table that is a camera coordinate system into a three-dimensional coordinate table that is a world coordinate system.
  • methods 1 to 3 described below are mainly conceivable as a method for performing this association.
  • Method 1 The method described below is referred to as “Method 1”.
  • Method 1 determines the position of the main target (hereinafter referred to as “Landmark”) on the camera field of view in the world coordinate system based on general maps, aerial photographs or road drawings. To derive.
  • An example of a landmark is an illumination column.
  • the method 1 compares the position of the derived landmark in the world coordinate system (coordinates such as latitude, longitude, and altitude) with the position on the captured image (that is, coordinates in the camera coordinate system). And associate the two.
  • a continuous correspondence table is created by interpolating the coordinates between the landmarks.
  • Method 2 The method described below is referred to as “Method 2”.
  • Method 2 derives the position (coordinates) of the landmark on the camera field of view in the world coordinate system using road surveying equipment such as Total Station. Then, in the method 2, the position (coordinate) of the derived landmark in the world coordinate system is compared with the position on the photographed image (that is, sitting in the camera coordinate system), and the two are associated with each other. Method 2 creates a continuous correspondence table by interpolating the coordinates between the landmarks.
  • Method 3 The method described below is referred to as “Method 3”.
  • Method 3 uses a survey vehicle equipped with a GPS (Global Positioning System) / inertial navigation system, stereo camera, laser radar (Laser radar) or the like.
  • Method 3 derives three-dimensional continuous road shape data (map data) as a position (coordinates) in the world coordinate system. Then, in the method 3, the map data values of the landmarks on the camera field of view are compared with the positions on the image obtained by photographing them (that is, the coordinates in the camera coordinate system), and the two are associated with each other.
  • Method 3 does not use interpolation or the like for the coordinates between the landmarks.
  • Method 3 creates a continuous correspondence table using the map data for the coordinates between the landmarks.
  • Patent Document 1 discloses the following method.
  • the method described in Patent Document 1 records the coordinates of the world coordinate system recorded by a survey vehicle equipped with GPS and the frame of a moving image captured by the camera in complete synchronization. This method creates a continuous correspondence table between the coordinates of the world coordinate system and the coordinates of the camera coordinate system without performing the road surface shape surveying based on the above record.
  • Patent Document 2 discloses the following method.
  • the method described in Patent Document 2 analyzes a three-dimensional point group representing the shape of a plurality of features and the like based on statistical processing as three-dimensional topographic data.
  • the coordinate axis used to associate the three-dimensional terrain data with the coordinates of the world coordinate system is estimated based on the analysis result.
  • Japanese Patent Application Laid-Open No. H10-228707 describes clustering point groups that are relatively close to each other.
  • Patent Document 2 describes that a point group is separated for each side surface based on the distribution of normal vectors on the side surface of the building.
  • the area associated with the position (coordinates) in the camera coordinate system is limited to a city area where a landmark such as a high-rise building or a lighting column exists, or a place where a road drawing exists. It will be. Therefore, when a place where a landmark does not exist (for example, a remote island, a port, or a mountainous area) is monitored in a wide area, Method 1 and Method 2 can associate the coordinates of the camera coordinate system with the coordinates of the world coordinate system. Absent. That is, in the above method, the method 1 and the method 2 cannot obtain the coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system. Furthermore, when there are undulations between landmarks, method 1 and method 2 need to interpolate the coordinate transformation of the point according to the undulations. Therefore, when a simple linear transformation is used, the estimation accuracy deteriorates in Method 1 and Method 2.
  • the area in which the camera field of view (camera coordinate system) and the real coordinates (world coordinate system) can be associated with each other is limited.
  • the associated area area where coordinate transformation can be obtained
  • the associated area is an area where a landmark exists, or a terrain area with little undulation. Or, it is limited to the area where the surveying vehicle can travel. Therefore, when monitoring a wide area outside, in the area where there are no landmarks, the area of terrain with little undulations, or the area where the surveying vehicle cannot run, the position in the world coordinate system of the object shown in the image taken by the camera ( It was difficult to accurately obtain the coordinates.
  • the methods 1 to 3 and the method described in Patent Document 1 have a problem in that the coordinates of the camera coordinate system and the coordinates of the world coordinate system cannot be associated with each other in the region. That is, the methods 1 to 3 and the method described in Patent Document 1 have a problem that coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system cannot be obtained in the region.
  • the method described in Patent Document 2 is a method of determining a gravity method from the outer shape of a building and cannot solve the above problem.
  • An object of the present invention is to solve the above-mentioned problems, and an information processing apparatus, an information processing system, and an information processing method capable of calculating coordinate transformation between the coordinates of the camera coordinate system and the coordinates of the world coordinate system regardless of the target region And providing a recording medium.
  • An information processing apparatus includes a small area formed on the basis of a point having three-dimensional information regarding a position on the terrain corresponding to an area included in captured image data captured by the imaging apparatus.
  • An information processing system includes the above information processing apparatus, display means for displaying three-dimensional information and / or measurement point candidates received from the information processing apparatus, three-dimensional information and / or Input means for acquiring candidate information in the measurement point candidates.
  • an information processing method includes a small area formed on the basis of a point having three-dimensional information regarding a position on the terrain corresponding to an area included in captured image data captured by the imaging apparatus. Based on the normal direction and altitude of the area, it is divided into small area groups that are groups of small areas, and using the small area group and candidate information that is information related to coordinate transformation, the coordinates of the camera coordinate system that is the coordinates in the captured image data A coordinate conversion table that is information on coordinate conversion between the coordinates and coordinates in the world coordinate system that is coordinates in a small area corresponding to the captured image data is created.
  • a recording medium including a small area formed based on a point having three-dimensional information regarding a position on the terrain corresponding to the area included in the captured image data captured by the imaging device.
  • the camera coordinate system that is the coordinates in the captured image data using the process of dividing into small area groups that are groups of small areas based on the normal direction and elevation of the image, and candidate information that is information related to the small area groups and coordinate transformation
  • a computer readable recording process for creating a coordinate conversion table, which is information of coordinate conversion between the coordinates of the image and the coordinates in the world coordinate system, which are the coordinates in the small area corresponding to the captured image data, and the program to be executed by the computer are recorded.
  • FIG. 1 is a block diagram illustrating an example of a configuration of a coordinate association system including a coordinate conversion table creation device according to a first embodiment of the present invention.
  • FIG. 2 is a schematic diagram illustrating an example of 3D terrain data stored in the 3D terrain database storage unit.
  • FIG. 3 is a flowchart illustrating an example of the operation in the coordinate conversion table creating apparatus according to the first embodiment.
  • FIG. 4 is a schematic diagram illustrating an example of a small region group according to the first embodiment.
  • FIG. 5 is a block diagram showing an example of the configuration of the object position measuring apparatus according to the first embodiment.
  • FIG. 6 is a schematic diagram illustrating a situation in which coordinates in the camera coordinate system of an object such as a suspicious vehicle in the object position measurement apparatus according to the first embodiment are converted into coordinates in the world coordinate system.
  • FIG. 7 is a block diagram illustrating an example of a configuration of a coordinate association system according to the second embodiment.
  • FIG. 8 is a flowchart illustrating an example of the operation of the coordinate conversion table creation device according to the second embodiment.
  • FIG. 9 is a schematic diagram illustrating an example of a group obtained as a result of step S203 in FIG.
  • FIG. 10 is a block diagram illustrating an example of a configuration of a coordinate association system according to the third embodiment.
  • FIG. 11 is a flowchart illustrating an example of the operation of the coordinate conversion table creating apparatus according to the third embodiment.
  • FIG. 12 is a schematic diagram for explaining a method of extracting measurement point candidates by the measurement point extraction unit according to the third embodiment.
  • FIG. 13 is a block diagram illustrating an example of a configuration of a coordinate association system according to the fourth embodiment.
  • FIG. 14 is a flowchart illustrating an example of the operation of the coordinate conversion table creation device according to the fourth embodiment.
  • FIG. 15 is a schematic diagram illustrating an example of extraction of measurement point candidates in the fourth embodiment.
  • FIG. 16 is a block diagram illustrating an example of a configuration of a coordinate association system according to the sixth embodiment.
  • FIG. 17 is a flowchart illustrating an example of the operation of the coordinate conversion table creation device according to the sixth embodiment.
  • FIG. 18 is an explanatory diagram illustrating an example of a measurement point candidate and a calculated shortest path in the sixth embodiment.
  • FIG. 19 is a block diagram illustrating an example of a configuration of a coordinate association system according to the seventh embodiment.
  • FIG. 20 is a flowchart illustrating an example of the operation of the coordinate conversion table creation device according to the seventh embodiment.
  • FIG. 21 is a schematic diagram for explaining the operation in step S606 in the seventh embodiment.
  • FIG. 22 is a block diagram illustrating an example of a configuration of a coordinate association system according to the eighth embodiment.
  • FIG. 23 is a flowchart illustrating an example of the operation of the coordinate conversion table creation device according to the eighth embodiment.
  • FIG. 24 is a schematic diagram for explaining the operation in step S708 in FIG.
  • FIG. 25 is a block diagram illustrating an example of a hardware configuration of the coordinate conversion table creation device according to each embodiment of the present invention.
  • FIG. 26 is a block diagram showing an example of the configuration of the information processing apparatus according to the embodiment of the present invention.
  • FIG. 27 is a block diagram showing an example of the configuration of the information processing system according to the embodiment of the present invention.
  • the information processing apparatus creates information related to coordinate transformation (correspondence) between the coordinates of the camera coordinate system and the coordinates of the world coordinate system.
  • the information processing apparatus is included in, for example, a predetermined information processing system (for example, a coordinate association system).
  • the information processing system performs an operation related to coordinate transformation (correspondence) created by the information processing apparatus. Therefore, in the following description, as an example of the information processing apparatus, coordinates for creating a coordinate conversion table (correspondence relationship table) that is information on coordinate conversion (correspondence relationship) between coordinates in the camera coordinate system and coordinates in the world coordinate system. This will be described using a conversion table creation device.
  • the information processing apparatus As an example of the information processing system, a description will be given using a coordinate association system including a device that transmits information used for the operation of the coordinate conversion table creation device.
  • the information processing apparatus according to each embodiment is not limited to the coordinate conversion table creation apparatus.
  • the information processing system according to each embodiment is not limited to the coordinate correspondence system.
  • the following description is demonstrated using a coordinate conversion table.
  • the coordinate conversion table may be called a correspondence table.
  • FIG. 1 is a block diagram showing an example of a configuration of a coordinate association system 100 including a coordinate conversion table creation device 1 according to the first embodiment of the present invention.
  • a coordinate association system 100 according to the first embodiment includes an imaging device 101, a three-dimensional terrain database storage unit 102, a coordinate conversion table creation device 1, a display device 104, a coordinate input device 105, and a storage device 107.
  • the coordinate conversion table creation device 1 includes a small area grouping unit 103 and a coordinate conversion table creation unit 106.
  • the imaging apparatus 101 is an apparatus (for example, a camera) that is installed at a monitoring target point and images a monitoring target area. An image photographed by the photographing apparatus 101 is referred to as “photographed image data”.
  • the 3D terrain database storage unit 102 is a storage device that stores terrain data in the monitoring target area (hereinafter referred to as “3D terrain data”) in advance.
  • the three-dimensional terrain data is a small area (for example, three points) formed based on point data including three-dimensional information (for example, latitude, longitude, and elevation values) and a predetermined number of points. Data).
  • the 3D terrain data stored in the 3D terrain database storage unit 102 is not limited to the above.
  • the data format of the three-dimensional terrain data is not particularly limited.
  • the three-dimensional landform database storage unit 102 may store, for example, three-dimensional landform data using a database format.
  • FIG. 2 is a schematic diagram showing an example of 3D terrain data stored in the 3D terrain database storage unit 102.
  • the three-dimensional terrain database storage unit 102 may store a three-dimensional map database owned by the Geographical Survey Institute or the like, or three-dimensional terrain data created based on the topographic survey of the monitoring target area.
  • the 3D terrain database storage unit 102 includes at least data corresponding to a region (location) included in an image (captured image data) captured by the imaging apparatus 101 as 3D terrain data.
  • the small area grouping unit 103 divides small areas included in the 3D terrain data into groups in the 3D terrain data corresponding to the locations included in the captured image data. That is, the small area grouping unit 103 creates a group in the small area included in the three-dimensional terrain data corresponding to the captured image data.
  • the created small area group may be simply referred to as a “small area group”.
  • the display device 104 displays 3D terrain data grouped into small area groups.
  • the coordinate input device 105 acquires information (candidate information) related to coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system regarding the three-dimensional terrain data displayed on the display device 104.
  • the coordinate input device 105 may include an input device (not shown) and acquire an operator's input operation as information related to coordinate conversion.
  • the coordinate input device 105 acquires information (candidate information) regarding coordinate conversion between the camera coordinate system and the world coordinate system for at least four arbitrary points within the small region for each small region group.
  • the coordinate conversion table creation unit 106 converts the coordinates of the camera coordinate system, which are the coordinates in the captured image data, into the coordinates of the world coordinate system, which is the coordinates of the three-dimensional terrain data, using the small area group and information related to the coordinate conversion.
  • a coordinate conversion table that is information to be created is created. That is, the coordinate conversion table is a table of information related to coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system in the captured image data.
  • the coordinate conversion table creation device 1 creates a coordinate conversion table.
  • the storage device 107 stores the created coordinate conversion table.
  • the small area grouping unit 103 and the coordinate conversion table creating unit 106 are realized by using, for example, a CPU (Central Processing Unit) of a computer that operates based on a coordinate conversion program.
  • the CPU may read a coordinate conversion program from a program recording medium such as a program storage device (not shown) and operate as each of the above elements based on the read coordinate conversion program.
  • the elements are the small region grouping unit 103 and the coordinate conversion table creation unit 106.
  • the computer will be described in detail later.
  • FIG. 3 is a flowchart showing an example of the operation in the coordinate conversion table creating apparatus 1 according to the first embodiment. Note that the image capturing apparatus 101 captures the monitoring target area, generates captured image data, and sends the captured image data to the coordinate conversion table creation unit 106.
  • the small area grouping unit 103 acquires the 3D terrain data corresponding to the location included in the captured image data from the 3D terrain database storage unit 102 (step S101).
  • the small area grouping unit 103 performs grouping (grouping) on the small areas of the acquired three-dimensional terrain data based on the normal direction and elevation information in each small area included in the acquired three-dimensional terrain data ( Step S102).
  • FIG. 4 is a schematic diagram illustrating an example of a small region group according to the first embodiment.
  • the small area grouping unit 103 performs grouping using the following method.
  • the small area grouping unit 103 obtains normal vectors (for example, normalized normal vectors) of all small areas included in the target three-dimensional terrain data and the average elevation of the small areas.
  • the small area grouping unit 103 divides all small areas into pairs according to a predetermined rule.
  • the small region grouping unit 103 calculates the inner product of the normal vectors and the elevation difference (the above-described difference in average elevation) for each pair of all the small regions.
  • the small area grouping unit 103 uses a pair of small areas in which the inner product and the altitude difference are included in a certain range of values as grouping targets.
  • the small area grouping unit 103 selects a small area pair whose normal direction is substantially the same using the inner product value, and further uses a difference in altitude to form a pair of small areas having the same altitude. Select. In other words, the small area grouping unit 103 selects a pair of small areas that have substantially the same elevation in substantially the same direction. Then, the small area grouping unit 103 groups the small area pairs using the altitude. Note that the small area grouping unit 103 may select a small area whose normal direction is a predetermined direction (for example, a vertical direction) as a processing target, and execute the grouping. For example, as illustrated in FIG.
  • the small region grouping unit 103 may select a substantially horizontal small region (small region pair) and divide it into groups.
  • a set of small areas with low elevation is a set of small areas included in group 1.
  • a set of small areas whose altitude is higher than a certain value than the altitude of group 1 is a set of small areas included in group 2.
  • the method of grouping small areas is not limited to the above method.
  • the small area grouping method may be any grouping technique that uses the normal direction and elevation information of each small area, and may be an existing grouping technique or clustering technique.
  • the display device 104 displays 3D terrain data grouped into small area groups.
  • the coordinate input device 105 acquires information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system for each small region group with respect to the three-dimensional terrain data displayed on the display device 104 (step). S103).
  • the coordinate input device 105 may include an input device (not shown) and acquire an operator's input operation as information related to coordinate conversion.
  • the coordinate input device 105 acquires information related to coordinate conversion between the camera coordinate system and the world coordinate system for at least four arbitrary points within the small region for each small region group.
  • the coordinate conversion table creation unit 106 coordinates the coordinates of the camera coordinate system, which is the coordinate system of the imaging apparatus 101, and the coordinates of the world coordinate system (for example, latitude, longitude, and altitude). Ask for conversion. Specifically, the coordinate conversion table creation unit 106 obtains coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system for all pixels on which the ground surface in the captured image is displayed. Then, the coordinate conversion table creation unit 106 creates a coordinate conversion table including the coordinate conversion of all the small area groups based on the calculated coordinate conversion (step S104).
  • the coordinate conversion table creation unit 106 may obtain coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system as follows, for example.
  • the coordinate conversion table creation unit 106 uses information regarding coordinate conversion between the camera coordinate system and the world coordinate system of each plane acquired by the coordinate input device 105 and the plane that approximates the small region group.
  • the coordinate conversion table creation unit 106 obtains a plane projective conversion parameter for performing plane projective conversion to the coordinates of a plane that approximates the coordinates of the camera coordinate system for each small region group.
  • the coordinate conversion table creation unit 106 performs planar projection corresponding to the small region group with respect to the position of the pixel corresponding to the ground surface in the captured image data in which the location corresponding to the small region group is captured (the coordinates in the camera coordinate system). A coordinate transformation using the transformation parameter is calculated. As described above, the coordinate conversion table creation unit 106 creates a coordinate conversion table using coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system.
  • the parameters related to the coordinate conversion of the small area group are not limited to the parameters of the planar projective conversion.
  • the parameter relating to coordinate transformation may be a parameter that is an approximation of affine transformation or other linear transformation.
  • the coordinate conversion table creation unit 106 obtains continuous coordinate conversion for pixels in each small region group using nonlinear conversion such as spline interpolation or TPS (ThinSPlate Spline) interpolation. May be.
  • the coordinate input device 105 acquires information related to the coordinate transformation between the camera coordinate system and the world coordinate system regarding the number of points necessary and sufficient to estimate each transformation parameter. Assuming that
  • the coordinate conversion table creation unit 106 stores the created coordinate conversion table in the storage device 107.
  • the small area grouping unit 103 divides the small areas into groups (small area groups) that can be approximated to a plane based on the normal direction and the elevation information. Then, the coordinate conversion table creation unit 106 uses the plane that approximates the small region group to convert the coordinates of the camera coordinate system of the imaging device 101 arranged at the monitoring target point and the coordinates of the world coordinate system. Create a conversion table. As described above, the coordinate conversion table creation device 1 can create a coordinate conversion table using conversion using a plane (for example, plane projection conversion) regardless of the topography of the monitoring target region.
  • a plane for example, plane projection conversion
  • the coordinate conversion table creation device 1 can determine the coordinates of the camera coordinate system and the world even if the monitoring target area is an area where no landmark is present, a complex terrain with undulations, or an area where a survey vehicle cannot travel. Coordinate conversion with the coordinates of the coordinate system can be realized.
  • the coordinate conversion table can be used to measure the position coordinates of an object such as a suspicious vehicle.
  • FIG. 5 is a block diagram illustrating an example of a configuration of the object position measurement device 2001 and the coordinate association system 100 according to the first embodiment.
  • the object position measuring apparatus 2001 illustrated in FIG. 5 is suspicious in the captured image data captured by the imaging apparatus 101 using the coordinate conversion table created by the coordinate conversion table creating apparatus 1 and stored in the storage device 107. Measure the position coordinates of an object such as a vehicle. Therefore, the object position measurement device 2001 includes an object detection device 2002 and an object position coordinate conversion device 2003.
  • the object detection device 2002 detects an object in the captured image data captured by the imaging device 101.
  • the object position coordinate conversion device 2003 uses the coordinate conversion table to convert object coordinates (camera coordinate system coordinates) in the captured image data into world coordinate system coordinates.
  • the object position measuring apparatus 2001 measures the position of the object (for example, coordinates such as latitude, longitude, and altitude) in the captured image data captured by the image capturing apparatus 101.
  • FIG. 6 illustrates a situation in which coordinates in the camera coordinate system (position in the captured image data) of an object such as a suspicious vehicle in the object position measurement apparatus 2001 according to the first embodiment are converted into coordinates in the world coordinate system. It is a schematic diagram. The coordinates of the camera coordinate system of each car included in the photographed image data shown on the left side of FIG. 6 are converted into the coordinates of the world coordinate system shown on the right side of FIG.
  • the coordinate conversion table stored in the storage device 107 includes a setting change of the photographing apparatus 101 (a change in resolution, a change in focus or angle of view accompanying a lens change, or a change in position or orientation of the photographing apparatus). It does not need to be updated unless it occurs. Therefore, when the coordinate conversion table is stored in the storage device 107, the object detection device 2002 can operate using the coordinate conversion table unless the photographing apparatus 101 is changed. That is, the object detection device 2002 uses the captured image data obtained from the imaging device 101 even after the coordinate conversion table creation device 1 does not operate unless the imaging device 101 is changed after the creation of the coordinate conversion table.
  • the coordinates of the contained object in the world coordinate system can be obtained.
  • the object is, for example, a suspicious person or a suspicious vehicle.
  • the coordinate conversion table creation device 1 can achieve the effect of calculating coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system regardless of the target region.
  • the small area grouping unit 103 divides the small areas in the 3D terrain data corresponding to the captured image data into groups based on the 3D terrain data stored in the 3D terrain database storage unit 102. More specifically, the small area grouping unit 103 divides the small areas into groups using the normal direction and the altitude of the small areas. Then, the coordinate conversion table creation unit 106 calculates coordinate conversion (coordinate conversion table) for converting the coordinates of the camera coordinate system into the coordinates of the world coordinate system for each group of small regions. Here, the small area grouping unit 103 divides the small areas into small area groups so as to be substantially flat based on the normal direction and the altitude.
  • the coordinate conversion table creation unit 106 can create a coordinate conversion table for converting the coordinates of the camera coordinate system to the coordinates of the world coordinate system using a plane that approximates each small region group.
  • the coordinate conversion table creation device 1 is for creating a coordinate conversion table using plane approximation regardless of the target region.
  • FIG. 7 is a block diagram illustrating an example of a configuration of the coordinate association system 200 according to the second embodiment.
  • the coordinate association system 200 includes an imaging device 201, a three-dimensional landform database storage unit 202, a coordinate conversion table creation device 2, a display device 205, a coordinate input device 206, and a storage device 208.
  • the coordinate conversion table creation device 2 includes a position / orientation acquisition unit 203, a small area grouping unit 204, and a coordinate conversion table creation unit 207.
  • the imaging device 201, the three-dimensional terrain database storage unit 202, and the storage device 208 are the same as the imaging device 101, the three-dimensional terrain database storage unit 102, and the storage device 107, respectively, according to the first embodiment.
  • the imaging device 201 outputs information regarding the imaging device 201 to the coordinate conversion table creation device 2 in addition to the captured image data.
  • the information related to the imaging apparatus 201 includes information related to the position of the imaging apparatus 201 (position information) and information related to the orientation of the imaging apparatus 201.
  • the position information is position information (for example, latitude, longitude, and altitude) in the world coordinate system of the photographing apparatus 201.
  • the information related to the orientation of the image capturing apparatus 201 is information related to the image capturing direction of the image capturing apparatus 201 (orientation information).
  • the position / orientation acquisition unit 203 acquires position information and direction information from the photographing apparatus 201.
  • the small area grouping unit 204 generates a small area of the 3D terrain data based on the 3D terrain data corresponding to the location included in the captured image data and the distance from the imaging device 201 to the location (3D terrain data). Create a group.
  • the display device 205 displays 3D terrain data grouped into small area groups.
  • the coordinate input device 206 acquires information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system regarding the three-dimensional terrain data displayed on the display device 205.
  • the coordinate input device 206 may include an input device (not shown) and acquire an operator's input operation as information related to coordinate conversion.
  • the coordinate input device 206 acquires information related to coordinate conversion between the camera coordinate system and the world coordinate system for at least four arbitrary points within the small region for each small region group.
  • the coordinate conversion table creation unit 207 creates a coordinate conversion table for converting the coordinates of the camera coordinate system in the captured image data into the coordinates of the world coordinate system using the small area group and the information related to the coordinate conversion.
  • the position / orientation acquisition unit 203, the small area grouping unit 204, and the coordinate conversion table creation unit 207 are realized using, for example, a CPU of a computer that operates based on a coordinate conversion program.
  • the CPU may read a coordinate conversion program from a program recording medium such as a program storage device (not shown) and operate as each of the above elements based on the read coordinate conversion program.
  • the computer will be described in detail later.
  • FIG. 8 is a flowchart showing an example of the operation of the coordinate conversion table creation device 2 according to the second embodiment. Note that the image capturing apparatus 201 captures the monitoring target area, generates captured image data, and sends the captured image data to the coordinate conversion table creation unit 207.
  • the position / orientation acquisition unit 203 acquires position information and direction information of the photographing apparatus 201. Similar to the small region grouping unit 103 of the first embodiment, the small region grouping unit 204 acquires three-dimensional landform data corresponding to the location of the captured image data from the three-dimensional landform database storage unit 202 (step S201). .
  • the position / orientation acquisition unit 203 may acquire the position information of the image capturing apparatus 201 using, for example, Exif (Exchangeable image file format) information of the captured image data. Further, the position / orientation acquisition unit 203 may acquire position information of the image capturing apparatus 201 from a GPS attached to the image capturing apparatus 201. Further, the position / orientation acquisition unit 203 may acquire the orientation information of the imaging apparatus 201 from a magnetic compass attached to the imaging apparatus 201. However, the method by which the position / orientation acquisition unit 203 acquires the position information and the direction information of the imaging apparatus 201 is not limited to the above. The position / orientation acquisition unit 203 may acquire the position information and the direction information of the photographing apparatus 201 from other sensors (not shown). Further, the position / orientation acquisition unit 203 may acquire the position information and the direction information of the photographing apparatus 201 based on an operation of an operator on an input device (not shown).
  • Exif Exchangeable image file format
  • the small region grouping unit 204 groups small regions of the acquired three-dimensional terrain data based on the normal direction and elevation information of each small region (step S202).
  • the small area grouping unit 204 obtains a distance from the photographing apparatus 201 to the small area for each small area based on the position information and the azimuth information of the photographing apparatus 201 acquired by the position / orientation acquisition unit 203.
  • the small area grouping unit 204 groups the small areas again based on the distance from the photographing apparatus 201 to the small area (step S203).
  • the small area grouping unit 204 further subdivides the group created in step S202 according to the distance from the imaging device 201 to the small area.
  • the small region grouping unit 204 divides at least a part of the groups into a plurality of groups based on the distance.
  • FIG. 9 is a schematic diagram showing an example of a group obtained as a result of step S203 in FIG. 9, in FIG. 9, a set of small areas in a short distance in a small area having a low altitude is group 1.
  • a group of small areas at a long distance is group 2.
  • a set of small areas at a short distance is Group 3.
  • a group of small areas at a long distance (on the right side in FIG. 9) in the small area having a high altitude is group 4.
  • groups 1 and 2 are divided into two groups based on the distance from the photographing apparatus 101.
  • the display device 205 displays the three-dimensional terrain data grouped in the subdivided small area groups.
  • the coordinate input device 206 acquires information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system for each small region group with respect to the three-dimensional terrain data displayed on the display device 205 (step). S204).
  • the coordinate input device 206 may include an input device (not shown) and acquire an operator's input operation as information related to coordinate conversion.
  • the coordinate input device 206 acquires information related to coordinate conversion between the camera coordinate system and the world coordinate system for at least four arbitrary points within the small region for each small region group.
  • the coordinate conversion table creation unit 207 creates a coordinate conversion table based on the small area group, similarly to the coordinate conversion table creation unit 106 of the first embodiment (step S205).
  • the coordinate conversion table creation unit 207 stores the created coordinate conversion table in the storage device 208.
  • the small area grouping unit 204 groups small areas using the distance. That is, the small area group of the second embodiment includes small areas having the same distance from the imaging device 201.
  • the accuracy of the coordinate conversion obtained by the coordinate conversion table creation unit 207 may be reduced.
  • the coordinate conversion table creation unit 207 creates a coordinate conversion table using the small area groups grouped based on the distance. Therefore, the coordinate conversion table creation device 2 can create a coordinate conversion table with high accuracy. Therefore, even when an object far from the photographing apparatus 201 is detected in the photographed image data, the coordinate conversion table creating apparatus 2 creates a coordinate conversion table using a small area group that is substantially the same distance as the object. is doing. Therefore, the coordinates of the object in the world coordinate system are calculated with high accuracy.
  • each embodiment described below may be configured not to execute this operation. That is, in each embodiment described below, the small areas may be grouped without using the distance from the imaging device to the small areas.
  • FIG. 10 is a block diagram illustrating an example of a configuration of a coordinate association system 300 according to the third embodiment.
  • a coordinate association system 300 according to the third embodiment includes an imaging device 301, a three-dimensional landform database storage unit 302, a coordinate conversion table creation device 3, a display device 306, a coordinate input device 307, and a storage device 309. including.
  • the coordinate conversion table creation device 3 includes a position / orientation acquisition unit 303, a small area grouping unit 304, a measurement point extraction unit 305, and a coordinate conversion table creation unit 308.
  • the photographing device 301, the three-dimensional landform database storage unit 302, and the storage device 309 are the same as the photographing device 201, the three-dimensional landform database storage unit 202, and the storage device 208 in the second embodiment.
  • the position / orientation acquisition unit 303 and the small region grouping unit 304 are the same as the position / orientation acquisition unit 203 and the small region grouping unit 204 in the second embodiment.
  • the measurement point extraction unit 305 extracts measurement point candidates in each small region group based on the small region grouping result in the small region grouping unit 304.
  • a measurement point candidate is a point from which information (candidate information) regarding coordinate transformation between the coordinates of the camera coordinate system and the corresponding coordinates (position) of the world coordinate system is acquired.
  • a measurement point candidate may be simply referred to as a “measurement point”. Note that the measurement point candidates are displayed on the display device 306, as will be described later.
  • Display device 306 displays measurement point candidates. For example, the display device 306 indicates a measurement point candidate to the worker.
  • the coordinate input device 307 is a device that acquires candidate information on measurement point candidates.
  • the coordinate input device 307 is a device for acquiring the candidate information based on the operation of the worker.
  • the operator's operation specifically includes information (coordinate conversion information) about a pair of coordinates (the coordinates of the camera coordinate system and the coordinates of the world coordinate system) in the measurement point candidate displayed on the display device 306. ) Input. That is, the coordinate input device 307 acquires information (candidate information) related to coordinate conversion in a coordinate pair of the coordinates of the camera coordinate system and the coordinates of the world coordinate system at the measurement point candidate.
  • the coordinate conversion table creation unit 308 acquires candidate information related to the measurement point candidate from the coordinate input device 307, and based on the acquired candidate information, coordinate conversion related to coordinate conversion from the coordinates of the camera coordinate system to the coordinates of the world coordinate system. Create a table.
  • the position / orientation acquisition unit 303, the small area grouping unit 304, the measurement point extraction unit 305, and the coordinate conversion table creation unit 308 are realized using, for example, a CPU of a computer that operates based on a coordinate conversion program.
  • the CPU may read a coordinate conversion program from a program recording medium such as a program storage device (not shown) and operate as each of the above elements based on the read coordinate conversion program.
  • the CPU executes an operation of transmitting information (candidate for measurement point) to the display device 306 and an operation of acquiring information (candidate information) from the coordinate input device 307 based on the coordinate conversion program.
  • This also applies to other embodiments described later.
  • the computer will be described in detail later.
  • FIG. 11 is a flowchart showing an example of the operation of the coordinate conversion table creation device 3 according to the third embodiment. Note that the imaging apparatus 301 captures the monitoring target area, generates captured image data, and sends the captured image data to the coordinate conversion table creation unit 308.
  • the position / orientation acquisition unit 303 acquires the position information and the direction information in the world coordinate system of the image capturing apparatus 301 in the same manner as the position / orientation acquisition unit 203 of the second embodiment. Further, the small area grouping unit 304 acquires the three-dimensional terrain data corresponding to the location of the captured image data from the three-dimensional terrain database storage unit 302, similarly to the small region grouping unit 204 of the second embodiment (step S301). ).
  • the small region grouping unit 304 divides the small regions of the acquired three-dimensional terrain data into groups based on the normal direction and elevation information of each small region (Ste S302).
  • the small region grouping unit 304 is a place corresponding to a small region from the imaging device 301 based on the position information and orientation information of the imaging device 301 acquired by the position / orientation acquisition unit 303. Find the distance to.
  • the small area grouping unit 304 groups the small area groups again based on the distance from the photographing apparatus 301 to the place corresponding to the small area (step S303).
  • the measurement point extraction unit 305 extracts a measurement point candidate for each small area group (step S304).
  • FIG. 12 is a schematic diagram for explaining a method of extracting measurement point candidates by the measurement point extraction unit 305 in the third embodiment.
  • FIG. 12 shows the measurement point candidates by using the group shown in FIG. 4 instead of the group regrouped based on the distance shown in FIG. 9 for easy understanding of the operation. It was.
  • the measurement point extraction unit 305 calculates the position (coordinates) of the center of gravity for each small area group. Next, the measurement point extraction unit 305 selects a plurality of points whose distance from the center of gravity is a certain value or more. Then, the measurement point extraction unit 305 extracts, as measurement point candidates, a predetermined number of points at which the angle formed by all line segments connecting each point and the center of gravity is equal to or greater than a predetermined angle from the selected points. That is, the measurement point extraction unit 305 extracts the measurement point candidate points so that they are separated from the center of gravity of the small region group by a predetermined distance or more and are not adjacent to each other.
  • the measurement point extraction unit 305 extracts a plurality of measurement point candidates for each small region group.
  • the predetermined number of measurement point candidates extracted here is a predetermined number or more, or a predetermined number of ranges. Then, the measurement point extraction unit 305 transmits the measurement point candidate to the display device 306.
  • the display device 306 displays the acquired measurement point candidates. At this time, the display device 306 may display the measurement point candidates together with the three-dimensional terrain data, as shown in FIG.
  • the coordinate input device 307 acquires information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system in the measurement point candidates displayed on the display device 306 (step S305).
  • the coordinate input device 307 may include an input device (not shown) and acquire an operator's input operation as information related to coordinate conversion.
  • the worker confirms the captured image data displayed on the display device 306 and the map data or the aerial image. Then, the operator may input information (candidate information) related to coordinate transformation between the coordinates of the camera coordinate system and the coordinates of the world coordinate system at the position corresponding to the measurement point candidate to the coordinate input device 307.
  • the worker has a sensor (for example, GPS) and a marker that can acquire coordinates in the world coordinate system, and goes to a measurement point candidate.
  • an operator measures the coordinate of the world coordinate system of a measurement point candidate using a sensor. Further, the operator places a marker on the measurement point candidate.
  • the imaging device 301 acquires captured image data in this state.
  • the operator obtains information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system at the position corresponding to the measurement point candidate, and sends the information to the coordinate input device 307. You may enter.
  • the coordinate input device 307 transmits information (candidate information) related to coordinate conversion of the coordinates of the camera coordinate system and the coordinates of the world coordinate system at the position corresponding to the acquired measurement point candidate to the coordinate conversion table creation unit 308.
  • the coordinate conversion table creation unit 308 uses the acquired candidate information to calculate the coordinates of the camera coordinate system of the small area group and the coordinates of the world coordinate system.
  • a coordinate conversion table is created (step S306). That is, the coordinate conversion table creation unit 308 creates a coordinate conversion table using information (candidate information) related to coordinate conversion of measurement point candidates.
  • the coordinate conversion table creation unit 308 causes the storage device 309 to store the created coordinate conversion table.
  • the measurement point extraction unit 305 of this embodiment extracts measurement point candidates in each small area group. Then, the coordinate conversion table creation unit 308 acquires information (candidate information) related to coordinate conversion in the above measurement point candidates.
  • the candidate information is information relating to coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system at the measurement point candidate.
  • the candidate information includes, for example, coordinate information measured by an operator or the like. That is, the candidate information (information regarding coordinate conversion) includes at least information regarding accurate coordinate conversion in the measurement point candidate.
  • the coordinate conversion table creation device 3 creates the coordinate conversion table using the candidate information, so that the accuracy of the coordinate conversion table can be improved.
  • the coordinate conversion table creation device 3 can improve the accuracy of the coordinate conversion table using the candidate information.
  • the acquired candidate information includes information related to accurate coordinate conversion in the small area group.
  • the processing amount for ensuring a predetermined accuracy in coordinate transformation is generally inversely proportional to the accuracy of the initial value. When the accuracy of the initial value is high, the processing amount decreases. Therefore, the coordinate conversion table creation device 3 can reduce the calculation cost when creating the coordinate conversion table using the acquired candidate information.
  • FIG. 13 is a block diagram illustrating an example of a configuration of a coordinate association system 400 according to the fourth embodiment.
  • a coordinate association system 400 according to the fourth embodiment includes an imaging device 401, a three-dimensional terrain database storage unit 402, a coordinate conversion table creation device 4, a display device 407, a coordinate input device 408, and a storage device 410. including.
  • the coordinate matching system 400 of the fourth embodiment includes a measurement ease database storage unit 405.
  • the coordinate conversion table creation device 4 includes a position / orientation acquisition unit 403, a small area grouping unit 404, a measurement point extraction unit 406, and a coordinate conversion table creation unit 409.
  • the imaging device 401, the three-dimensional terrain database storage unit 402, and the storage device 410 are the same as the imaging device 301, the three-dimensional terrain database storage unit 302, and the storage device 309 in the third embodiment.
  • the display device 407 and the coordinate input device 408 are the same as the display device 306 and the coordinate input device 307 in the third embodiment.
  • the position / orientation acquisition unit 403, the small region grouping unit 404, and the coordinate conversion table creation unit 409 are the same as the position / orientation acquisition unit 303, the small region grouping unit 304, and the coordinate conversion table creation unit 308 in the third embodiment. It is.
  • the measurement ease database storage unit 405 is a storage device that stores in advance data obtained by quantifying the ease of measurement of coordinates in the world coordinate system at each point included in the three-dimensional terrain data.
  • the storage format of data obtained by digitizing the measurement ease is not particularly limited.
  • the measurement ease database storage unit 405 may store measurement ease data as a database format.
  • data obtained by quantifying measurement ease may be simply referred to as “measurement ease data”.
  • a region such as a pond or a swamp is a region where it is difficult for an operator to go to that point and measure the coordinates of the world coordinate system.
  • the measurement ease data of the points included in the difficult region is a small value.
  • a region such as a flat land or a road is a region where an operator can easily go to that point and measure the coordinates of the world coordinate system.
  • the measurement ease data of the points included in such a region is a large value. As described above, in the following description, it is assumed that a point where the coordinates of the world coordinate system are easy to measure has a larger value of measurement ease data.
  • the measurement point extraction unit 406 extracts measurement point candidates in the small region group based on the small region group grouped by the small region grouping unit 404 and the measurement ease data. That is, the measurement point extraction unit 406 extracts measurement point candidates that can be easily measured.
  • the position / orientation acquisition unit 403, the small area grouping unit 404, the measurement point extraction unit 406, and the coordinate conversion table creation unit 409 are realized using a CPU of a computer that operates based on a coordinate conversion program, for example.
  • the CPU may read a coordinate conversion program from a program recording medium such as a program storage device (not shown) and operate as each of the above elements based on the read coordinate conversion program.
  • the computer will be described in detail later.
  • FIG. 14 is a flowchart showing an example of the operation of the coordinate conversion table creation device 4 according to the fourth embodiment. Note that the imaging apparatus 401 captures the monitoring target area, generates captured image data, and sends the captured image data to the coordinate conversion table creation unit 409.
  • the position / orientation acquisition unit 403 acquires the position information and the direction information in the world coordinate system of the imaging apparatus 401, as with the position / orientation acquisition unit 303 of the third embodiment.
  • the small area grouping unit 404 acquires the 3D terrain data corresponding to the location of the captured image data from the 3D terrain database storage unit 402, similarly to the small area grouping unit 304 of the third embodiment (step S401). ).
  • the small region grouping unit 404 groups small regions of the acquired three-dimensional terrain data based on the normal direction and elevation information of each small region (step) S402).
  • the small region grouping unit 404 calculates the distance from the image capturing device 401 to the small region based on the position information and the direction information of the image capturing device 401 acquired by the position / orientation acquiring unit 403. Ask.
  • the small area grouping unit 404 divides the small area group according to the distance from the photographing apparatus 401 to the small area (step S403).
  • the measurement point extraction unit 406 extracts a plurality of measurement point candidates for each small region group, similarly to the measurement point extraction unit 305 of the third embodiment (step S404).
  • the measurement point extraction unit 406 ranks the measurement point candidates extracted in step S404 based on the measurement ease data of the measurement point candidates.
  • the measurement point extraction unit 406 performs this process for each small area group. That is, the measurement point extraction unit 406 ranks the measurement point candidates in the order in which the coordinates of the world coordinate system are easily measured in each small region group.
  • the measurement point extraction unit 406 may rank the measurement point candidates in descending order of the measurement ease data for each small region group. Then, the measurement point extraction unit 406 extracts up to a predetermined number of measurement point candidates for each small region group based on the rank (step S405).
  • the measurement point extraction unit 406 is set to extract measurement point candidates from the highest level to a predetermined number. In this case, the measurement point extraction unit 406 extracts (selects) the measurement point candidates from the highest level to the predetermined number for each small region group.
  • the aspect of extracting the measurement point candidates based on the rank is not limited to the above.
  • the measurement point extraction unit 406 may extract measurement point candidates whose measurement ease data value is greater than a predetermined threshold.
  • FIG. 15 is a schematic diagram showing an example of extraction of measurement point candidates in the fourth embodiment.
  • FIG. 15 shows a measurement point candidate that is desired to be extracted (selected) with a low rank as a triangular marker, and shows the extracted measurement point candidate using an X-type mark.
  • the upper part of FIG. 15 shows positions corresponding to measurement point candidates in the captured image data.
  • the lower diagram in FIG. 15 shows the positions of the measurement point candidates in the three-dimensional terrain data.
  • the lower figure of FIG. 15 is not shown in FIG. 4 but a group regrouped based on the distance shown in FIG. Using groups, candidate measurement points were shown.
  • the measurement point extraction unit 406 has extracted a measurement point candidate in step S404, and a measurement point candidate is further extracted from the measurement point candidates in step S405.
  • the operation of the measurement point extraction unit 406 is not limited to such an operation.
  • the measurement point extracting unit 406 may extract the measurement point candidates from the top to the predetermined order in descending order of the value of the measurement ease data for each small region group without executing Step S404.
  • the operation of the measurement point extraction unit 406 is the same in each embodiment described later.
  • the display device 407 displays the measurement point candidates extracted (selected) in step S405, similarly to the display device 306 of the third embodiment.
  • the coordinate input device 408 operates in the same manner as the coordinate input device 307 of the third embodiment. That is, the coordinate input device 408 obtains information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system in the measurement point candidates displayed on the display device 407 (step S406).
  • the coordinate conversion table creation unit 409 creates a coordinate conversion table between the coordinates of the camera coordinate system and the coordinates of the world coordinate system, using the acquired candidate information, similarly to the coordinate conversion table creation unit 308 of the third embodiment. (Step S407).
  • the coordinate conversion table creation unit 409 causes the storage device 410 to store the created coordinate conversion table.
  • the fourth embodiment has an effect of facilitating acquisition of candidate information for further improving accuracy.
  • the reason is as follows.
  • the measurement point extraction unit 406 of the fourth embodiment extracts measurement point candidates that can be easily measured based on the measurement ease data stored in the measurement ease database storage unit 405. Therefore, the candidate information acquired by the coordinate input device 408 is information that can be easily obtained. Therefore, 4th Embodiment also has an effect of reducing a worker's load.
  • the coordinate conversion table creation unit 409 can acquire highly accurate candidate information.
  • each embodiment as in the case of the fourth embodiment, a case will be described in which measurement point candidates are ranked, and measurement point candidates are extracted based on the ranks.
  • each embodiment described below may be configured not to perform this operation.
  • a coordinate association system 400 according to the fifth embodiment will be described.
  • the configuration of the coordinate association system 400 according to the fifth embodiment is the same as that of the coordinate association system 400 according to the fourth embodiment. Therefore, the fifth embodiment will be described below with reference to FIG.
  • the coordinate conversion table creation unit 409 creates a coordinate conversion table for at least one small area group, and then performs coordinate conversion in the remaining small area groups based on the positional relationship between the small area group and the remaining small area groups. presume. Then, the coordinate conversion table creation unit 409 creates a coordinate conversion table for the remaining small region groups based on the estimated coordinate conversion.
  • the operation of the fifth embodiment is different from the operations of steps S406 and S407 in the flowchart of FIG. 14 showing the operation of the fourth embodiment.
  • the display device 407 according to the fifth embodiment displays the measurement point candidates extracted in step S405 in the same manner as the display device 407 according to the fourth embodiment.
  • the coordinate input device 408 of the fifth embodiment acquires information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system with respect to the measurement point candidates of at least one small region group (steps). S406).
  • the coordinate conversion table creation unit 409 obtains coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system with respect to the small area group (hereinafter referred to as “small area group A ”) corresponding to the acquired candidate information. . More specifically, for the small area group A , the coordinate conversion table creation unit 409 obtains a coordinate conversion parameter for coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system. Then, the coordinate conversion table generation unit 409, by using the coordinate transformation parameters for all the pixels surface of the region corresponding to the small region group A is displayed, to create the coordinate conversion table.
  • the coordinate conversion table creation unit 409 performs coordinate conversion (coordinate conversion parameters) in another small region group (hereinafter referred to as “small region group B ”) based on the relative positional relationship with the small region group A. ).
  • the coordinate conversion table creation unit 409 calculates a rotation matrix and a translation vector representing a relative positional relationship between the small area groups based on the three-dimensional terrain data.
  • the coordinate conversion table creation unit 409 may calculate a rotation matrix and a translation vector in advance.
  • the coordinate conversion table creation unit 409 calculates the coordinate conversion parameter in the small region group B from the coordinate conversion parameter in the small region group A based on the positional relationship between the small region group B and the small region group A. Specifically, the coordinate conversion table creation unit 409 calculates the coordinate conversion parameter in the small region group B from the coordinate conversion parameter in the small region group A using the rotation matrix and the translation vector representing the positional relationship of the small region group. To do.
  • the coordinate conversion table creation unit 409 creates a coordinate conversion table for all the pixels on which the ground surface in the region corresponding to the small region group B is displayed, using the calculated coordinate conversion parameter. As described above, the coordinate conversion table creation unit 409 creates a coordinate conversion table for the small area group B based on the positional relationship between the small area groups and the coordinate conversion parameters for the small area group A for which candidate information has been acquired ( Step S407).
  • the coordinate conversion table creation unit 409 causes the storage device 410 to store the created coordinate conversion table.
  • the fifth embodiment has the effect of facilitating acquisition of candidate information.
  • the reason is as follows.
  • the coordinate conversion table creation unit 409 of the fifth embodiment estimates the coordinate conversion parameters of other small region groups based on at least some of the small region group candidate information, and the coordinate conversion table based on the estimated parameters. Create This is because the coordinate conversion table creation device 4 only needs to acquire candidate information for some of the measurement point candidates. Therefore, the fifth embodiment also has an effect of reducing the load on the worker.
  • a correspondence relationship regarding one or a part of small region groups may be used as the correspondence relationship of the small region groups.
  • FIG. 16 is a block diagram illustrating an example of a configuration of a coordinate association system 500 according to the sixth embodiment.
  • a coordinate association system 500 according to the sixth embodiment includes an imaging device 501, a three-dimensional landform database storage unit 502, a coordinate conversion table creation device 5, a display device 508, a coordinate input device 509, and a storage device 511. including.
  • the coordinate association system 500 of the sixth embodiment includes a measurement ease database storage unit 505.
  • the coordinate conversion table creation device 5 includes a position / orientation acquisition unit 503, a small area grouping unit 504, a measurement point extraction unit 506, a route calculation unit 507, and a coordinate conversion table creation unit 510.
  • the photographing device 501, the three-dimensional terrain database storage unit 502, and the storage device 511 are the same as the photographing device 401, the three-dimensional terrain database storage unit 402, and the storage device 410 in the fourth embodiment.
  • the measurement ease database storage unit 505, the display device 508, and the coordinate input device 509 are the same as the measurement ease database storage unit 405, the display device 407, and the coordinate input device 408 in the fourth embodiment.
  • the display device 508 displays the shortest path as described later.
  • the position / orientation acquisition unit 503, the small region grouping unit 504, and the coordinate conversion table creation unit 510 are the same as the position / orientation acquisition unit 403, the small region grouping unit 404, and the coordinate conversion table creation unit 409 in the fourth embodiment. It is.
  • the measurement point extraction unit 506 is the same as the measurement point extraction unit 406 in the fourth embodiment.
  • the route calculation unit 507 calculates the shortest route that passes through all the measurement point candidates extracted by the measurement point extraction unit 506. Then, the route calculation unit 507 outputs (transmits) the calculated shortest route and measurement point candidate to the display device 508.
  • the display device 508 displays the measurement point candidates and the shortest route.
  • the position / orientation acquisition unit 503, the small area grouping unit 504, the measurement point extraction unit 506, the route calculation unit 507, and the coordinate conversion table creation unit 510 are realized as follows, for example. These elements are realized using, for example, a CPU of a computer that operates based on a coordinate conversion program. In this case, the CPU may read a coordinate conversion program from a program recording medium such as a program storage device (not shown) and operate as each of the above elements based on the read coordinate conversion program. The computer will be described in detail later.
  • FIG. 17 is a flowchart showing an example of the operation of the coordinate conversion table creation device 5 according to the sixth embodiment. Note that the image capturing apparatus 501 captures the monitoring target area, generates captured image data, and sends the captured image data to the coordinate conversion table creation unit 510.
  • steps S501 to S505 shown in FIG. 17 are the same as the operations of steps S401 to S405 in the fourth embodiment shown in FIG. Therefore, the detailed description is abbreviate
  • step S505 after acquiring the measurement point candidates extracted by the measurement point extraction unit 506, the route calculation unit 507 calculates the shortest route that passes through all the measurement point candidates based on the distance between the measurement point candidates (step S506). ).
  • the method for calculating the shortest route in the measurement point extraction unit 506 is not particularly limited.
  • the route calculation unit 507 may calculate the shortest route as follows.
  • the route calculation unit 507 creates a graph in which measurement point candidates are nodes and nodes (measurement point candidates) are connected by edges. At this time, the route calculation unit 507 associates the distance information of the measurement point candidate with the edge. The route calculation unit 507 determines the shortest route that passes through all nodes (all measurement point candidates) for this graph using the Dijkstra method. Note that the route calculation unit 507 may consider not only distance information associated with an edge but also three-dimensional terrain data between nodes (measurement point candidates) in calculating the shortest route.
  • the route calculation unit 507 numerically represents the ease of travel between the nodes (measurement point candidates) (for example, the ease of travel of the vehicle whose coordinates are measured) based on the three-dimensional landform data between the nodes (measurement point candidates). Parameterized). Then, the route calculation unit 507 may perform weighting on the edge using the parameter. However, the route calculation unit 507 may use a route determination method other than the Dijkstra method.
  • the display device 508 displays the measurement point candidates extracted in step S505 and the shortest route calculated in step S506.
  • FIG. 18 is a diagram illustrating an example of a measurement point candidate and a calculated shortest route in the sixth embodiment.
  • the operator moves a moving object having means for measuring the coordinates of the world coordinate system along the displayed shortest path, and uses the moving object to measure the measurement point candidate. Measure the coordinates of the world coordinate system corresponding to. For example, an operator who inputs candidate information to the coordinate input device 509 performs the measurement of the coordinates of the world coordinate system and the arrangement of the markers at the measurement candidate points described in the third embodiment along the displayed shortest path. Execute.
  • the coordinate input device 408 operates in the same manner as the coordinate input device 408 of the fourth embodiment. That is, the coordinate input device 408 acquires information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system in the measurement point candidates displayed on the display device 508 (step S508).
  • the coordinate conversion table creation unit 510 creates a coordinate conversion table between the coordinates of the camera coordinate system and the coordinates of the world coordinate system, using the acquired candidate information, similarly to the coordinate conversion table creation unit 409 of the fourth embodiment. (Step S509).
  • the coordinate conversion table creation unit 510 stores the created coordinate conversion table in the storage device 511.
  • the sixth embodiment has the effect of further reducing the load for acquiring candidate information.
  • the reason is as follows.
  • the route calculation unit 507 of the present embodiment outputs the measurement point candidate and the shortest route to the display device 508. Therefore, the display device 508 can display the shortest path passing through the measurement point candidate in addition to the measurement point candidate. Therefore, an operator who measures candidate information can move a moving body (for example, a measurement vehicle) used for measurement along the displayed shortest route, reduce the time required for measurement, and measure coordinates. it can.
  • a moving body for example, a measurement vehicle
  • FIG. 19 is a block diagram illustrating an example of a configuration of a coordinate association system 600 according to the seventh embodiment.
  • a coordinate association system 600 according to the seventh embodiment includes a three-dimensional landform database storage unit 602, a coordinate conversion table creation device 6, a measurement ease database storage unit 605, and a storage device 611.
  • the coordinate association system 600 of the seventh embodiment includes a display device 608 and a coordinate input device 609.
  • a coordinate association system 600 according to the seventh embodiment includes a plurality of imaging devices 601.
  • the coordinate conversion table creation device 6 includes a position / orientation acquisition unit 603, a small region grouping unit 604, a measurement point extraction unit 606, a measurement point grouping unit 607, and a coordinate conversion table creation unit 610.
  • the three-dimensional landform database storage unit 602, the measurement ease database storage unit 605, and the storage device 611 are the three-dimensional landform database storage unit 402, the measurement ease database storage unit 405, and the storage device 410 in the fourth embodiment. It is the same.
  • the display device 608 and the coordinate input device 609 are the same as the display device 407 and the coordinate input device 408 in the fourth embodiment. However, the display device 608 displays the measurement point candidates for each photographing device 601.
  • the small region grouping unit 604 and the measurement point extraction unit 606 are the same as the small region grouping unit 404 and the measurement point extraction unit 406 in the fourth embodiment.
  • the individual photographing devices 601 are the same as the photographing device 401 in the fourth embodiment.
  • the position / orientation acquisition unit 603 acquires position information and direction information in the world coordinate system of the image capturing device 601 for each image capturing device 601.
  • the other points are the same as the position / orientation acquisition unit 403 in the fourth embodiment.
  • the coordinate conversion table creation unit 610 receives photographed image data for each photographing apparatus 601 and creates a coordinate conversion table. Regarding the other points, the coordinate conversion table creation unit 610 is the same as the coordinate conversion table creation unit 409 in the fourth embodiment.
  • the measurement point grouping unit 607 corresponds to each photographing apparatus 601 from the measurement point candidates extracted by the measurement point extraction unit 606 based on the reliability of coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system. Create a group to be used.
  • the reliability is a value indicating the degree of matching (for example, the magnitude of error) between the coordinates of the camera coordinate system and the coordinates of the world coordinate system.
  • the measurement point grouping unit 607 is not limited to a value used as the reliability. However, in the description of the present embodiment, as an example, a case will be described in which the distance between each imaging device 601 and a measurement point candidate is used as the reliability.
  • the measurement point candidate is farther from the image capturing device 601, the error in the coordinate system of the world coordinate system (error in the camera depth direction) that is assumed to be a deviation per pixel of the image capturing device 601 (camera) increases. Therefore, the farther the measurement point candidate is from the imaging device 601, the greater the error in the coordinate relationship between the coordinates of the camera coordinate system and the coordinates of the world coordinate system.
  • the coordinate conversion table creation unit 610 can improve accuracy when operated using candidate information in measurement point candidates close to each imaging device 601. Therefore, the measurement point grouping unit 607 can use the distance between the imaging device 601 and the measurement point candidate as the reliability.
  • the measurement point grouping unit 607 divides the measurement point candidates into groups corresponding to the respective image capturing apparatuses 601 based on the distance between the measurement point candidates and the image capturing apparatus 601. Specifically, the measurement point grouping unit 607 creates a group of measurement point candidates whose distance from the imaging device 601 is less than a predetermined threshold as a group for each imaging device 601. That is, the measurement point grouping unit 607 creates a group of measurement point candidates for each photographing apparatus 601 based on the reliability of the coordinates of the world coordinate system in the measurement point candidates and the coordinates of the camera coordinates in the image data.
  • the position / orientation acquisition unit 603, the small area grouping unit 604, the measurement point extraction unit 606, the measurement point grouping unit 607, and the coordinate conversion table creation unit 610 are realized as follows. These elements are realized using, for example, a CPU of a computer that operates based on a coordinate conversion program. In this case, the CPU may read a coordinate conversion program from a program recording medium such as a program storage device (not shown) and operate as each of the above elements based on the read coordinate conversion program. The computer will be described in detail later.
  • FIG. 20 is a flowchart showing an example of the operation of the coordinate conversion table creation device 6 according to the seventh embodiment. Note that each imaging device 601 captures a monitoring target area, generates captured image data, and sends the captured image data to the coordinate conversion table creation unit 610.
  • steps S601 to S605 shown in FIG. 20 are the same as the operations in steps S401 to S405 in the fourth embodiment shown in FIG. 14, respectively, and detailed description thereof will be omitted.
  • step S603 the small area grouping unit 604 performs the same operation as that in step S403 in FIG. That is, the small area grouping unit 604 obtains the distance from the image capturing apparatus 601 to the small area for each image capturing apparatus 601. Then, the small region grouping unit 604 further divides the small region group grouped in step S602 into groups based on the distance from the photographing device 601 to the small region for each photographing device 601.
  • step S604 the measurement point extraction unit 606 performs the same operation as that in step S404 in FIG. 14 for each photographing apparatus 601. That is, the measurement point extraction unit 606 performs a process of extracting measurement point candidates in the small area group for each photographing apparatus 601. Furthermore, in step S605, the measurement point extraction unit 606 executes the same operation as in step S405 of FIG. That is, the measurement point extraction unit 606 extracts measurement point candidates for each photographing apparatus 601.
  • the measurement point grouping unit 607 divides the measurement point candidates into groups based on the distance to each image capturing device 601 for each image capturing device 601 (step S606). Specifically, the measurement point grouping unit 607 creates a group of measurement point candidates whose distance from the imaging device 601 is less than a threshold for each imaging device 601. That is, in step S606, the measurement point grouping unit 607 determines a measurement point candidate for each photographing apparatus 601.
  • FIG. 21 is a schematic diagram for explaining the operation in step S606 in the seventh embodiment.
  • FIG. 21 uses the group shown in FIG. 4 instead of the group regrouped based on the distance shown in FIG. 9 in order to make the operation easy to understand.
  • the measurement point candidate was shown.
  • FIG. 21 illustrates a case where there are two imaging devices 601.
  • the two photographing devices 601 are denoted by alphabetical signs and are distinguished as the photographing device 601 a and the photographing device 601 b .
  • Measurement point candidate shown with a square marker in FIG. 21 is a measurement point candidate distance from the photographing apparatus 601 a and the imaging device 601 b is less than the threshold. Measurement point candidates are shown using markers triangle, the distance from the imaging device 601 a is less than the threshold value, a measurement point candidate distance from the photographing device 601 b is greater than or equal to the threshold. X-type marker measurement point candidates are shown with the distance from the imaging device 601 a and a is equal to or larger than the threshold, a measurement point candidate distance is less than the threshold value from the imaging device 601 b.
  • Measurement point grouping unit 607 based on the distance from the imaging device 601 a, as a group on imaging device 601 a, selects the measurement point candidates are shown using markers square and triangle. That is, the measurement point grouping unit 607, a group for capturing apparatus 601 a, excludes measurement point candidate indicated by X-type marker.
  • measurement point grouping unit 607 based on the distance from the imaging device 601 b, as a group on imaging device 601 b, selects the measurement point candidates are shown with triangles, and X-type marker. That is, the measurement point grouping unit 607, as a group on imaging device 601 b, excludes measurement point candidate indicated by the marker triangle.
  • the measurement point grouping unit 607 transmits to the display device 608 the measurement point candidates and information regarding the group for each photographing device 601.
  • the display device 608 displays measurement point candidates included in the corresponding group for each photographing device 601.
  • the coordinate input device 609 operates in the same manner as the coordinate input device 408 of the fourth embodiment for each imaging device 601. That is, the coordinate input device 609 obtains information (candidate information) related to coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system in the measurement point candidates displayed on the display device 608 for each photographing device 601. (Step S608).
  • the coordinate conversion table creation unit 610 uses the acquired candidate information to determine the coordinates of the camera coordinate system and the coordinates of the world coordinate system for each imaging device 601. Coordinate conversion table is created (step S609).
  • the coordinate conversion table creation unit 610 stores the coordinate conversion table created for each photographing device 601 in the storage device 611.
  • the seventh embodiment has the effect of further improving accuracy.
  • the reason is as follows.
  • the measurement point grouping unit 607 creates a group of measurement point candidates for each photographing apparatus 601 based on the distance from each photographing apparatus 601. In other words, the measurement point grouping unit 607 creates a group that does not include measurement point candidates for which the reliability of coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system is low. Then, the measurement point grouping unit 607 transmits the measurement point candidates and the created group to the display device 608.
  • Display device 608 displays the measurement point candidates and groups. That is, the display device 608 displays a measurement point candidate with high reliability for each photographing device 601.
  • the coordinate input device 609 acquires information (candidate information) related to coordinate conversion for a measurement point candidate with high reliability. Since the coordinate conversion table creation unit 610 creates the coordinate conversion table based on the candidate information acquired as described above, the accuracy of the coordinate conversion table can be improved.
  • grouping of measurement point candidates in the seventh embodiment may be used.
  • a coordinate association system 700 according to the eighth embodiment includes a plurality of imaging devices 701 as in the seventh embodiment. Then, the coordinate association system 700 of the eighth embodiment creates a coordinate conversion table for the captured image data of at least one imaging device 701. Then, the coordinate association system 700 of the eighth embodiment creates a coordinate conversion table related to the captured image data of the other imaging device 701 based on the coordinate conversion table and the relative positional relationship in the imaging device 701. To do.
  • FIG. 22 is a block diagram illustrating an example of a configuration of the coordinate association system 700 according to the eighth embodiment.
  • a coordinate association system 700 according to the eighth embodiment includes a three-dimensional landform database storage unit 702, a coordinate conversion table creation device 7, a measurement ease database storage unit 705, and a storage device 711. Furthermore, the coordinate association system 700 includes a display device 707 and a coordinate input device 708.
  • the coordinate association system 700 includes a plurality of imaging devices 701.
  • the coordinate conversion table creation device 7 includes a position / orientation acquisition unit 703, a small area grouping unit 704, a measurement point extraction unit 706, an imaging device relative position calculation unit 709, and a coordinate conversion table creation unit 710.
  • the imaging device 701 is the same as the imaging device 601 in the seventh embodiment.
  • the three-dimensional landform database storage unit 702, the measurement ease database storage unit 705, and the storage device 711 are the same as the three-dimensional landform database storage unit 602, the measurement ease database storage unit 605, and the storage device 611 in the seventh embodiment. It is the same.
  • the display device 707 and the coordinate input device 708 are the same as the display device 608 and the coordinate input device 609 in the seventh embodiment. However, the display device 707 does not use a group for each photographing device 701 when displaying the measurement point candidate for each photographing device 701.
  • the position / orientation acquisition unit 703, the small region grouping unit 704, and the measurement point extraction unit 706 are the same as the position / orientation acquisition unit 603, the small region grouping unit 604, and the measurement point extraction unit 606 in the seventh embodiment. .
  • the imaging device relative position calculation unit 709 acquires information (candidate information) on coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system at the measurement point candidate from the coordinate input device 708.
  • the imaging device relative position calculation unit 709 calculates the relative positional relationship of the imaging device 701 based on the information. That is, the imaging device relative position calculation unit 709 calculates information related to the relative position of the imaging device 701 based on the candidate information of the measurement point candidates.
  • the coordinate conversion table creation unit 710 creates a coordinate conversion table for at least one imaging device 701 using candidate information. Then, the coordinate conversion table creation unit 710 creates a coordinate conversion table related to the other imaging device 701 based on the coordinate conversion table and the relative positional relationship between the imaging devices 701. That is, the coordinate conversion table creation unit 710 creates a coordinate conversion table based on information (candidate information) related to coordinate conversion of at least one photographing apparatus 701, and based on the created coordinate conversion table and information related to the relative position, A coordinate conversion table for another imaging apparatus 701 is created.
  • the position / orientation acquisition unit 703, the small area grouping unit 704, the measurement point extraction unit 706, the imaging device relative position calculation unit 709, and the coordinate conversion table creation unit 710 are realized as follows. These elements are realized using, for example, a CPU of a computer that operates based on a coordinate conversion program. In this case, the CPU may read a coordinate conversion program from a program recording medium such as a program storage device (not shown) and operate as each of the above elements based on the read coordinate conversion program. The computer will be described in detail later.
  • FIG. 23 is a flowchart showing an example of the operation of the coordinate conversion table creation device 7 according to the eighth embodiment.
  • each imaging device 701 captures a monitoring target region, generates captured image data, and sends the captured image data to the coordinate conversion table creation unit 710.
  • step S705 when a measurement point candidate is extracted for each photographing device 701, the display device 707 displays the measurement point candidate for each photographing device 701 as in the sixth embodiment. That is, as described above, the display device 707 does not use a group for each photographing device 701.
  • the coordinate input device 708 operates in the same manner as the coordinate input device 609 of the sixth embodiment for each imaging device 701. That is, the coordinate input device 708 acquires, for each photographing device 701, information (candidate information) regarding coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system at the measurement point candidate displayed on the display device 707. (Step S707).
  • the imaging device relative position calculation unit 709 calculates the relative positional relationship of the imaging device 701 for each imaging device 701 based on the candidate information of the measurement point candidates for each imaging device 701 (step S708). Specifically, the imaging device relative position calculation unit 709 calculates a rotation matrix and a translation vector that represent the relative positional relationship of the imaging device 701.
  • FIG. 24 is a schematic diagram for explaining the operation in step S708 in FIG.
  • FIG. 24 uses the group shown in FIG. 4 instead of the group regrouped based on the distance shown in FIG. 9 in order to make the operation easy to understand.
  • the measurement point candidate was shown.
  • FIG. 24 illustrates a case where there are two imaging devices 701.
  • the two photographing devices 701 are denoted by alphabetical signs and are distinguished as photographing devices 701 a and 701 b .
  • Imaging device relative position calculating unit 709 obtains a candidate information at measurement point candidate corresponding to the imaging apparatus 701 a. Similarly, imaging device relative position calculating unit 709 obtains a candidate information at measurement point candidate corresponding to the photographing apparatus 701 b.
  • the imaging device relative position calculation unit 709 is a rotation matrix that represents the relative positional relationship between the imaging devices 701 a and 701 b based on the candidate information of the imaging device 701 a and the candidate information of the imaging device 701 b. And a translation vector is calculated.
  • FIG. 24 shows two imaging devices 701. However, this is exemplary. Three or more imaging devices 701 may be included.
  • the imaging device relative position calculation unit 709 determines, for example, a reference imaging device 701 (hereinafter referred to as “imaging device 701 a ”).
  • the photographing device relative position calculation unit 709 may calculate a relative positional relationship between the photographing device 701 a and the other photographing device 701 (hereinafter referred to as “photographing device 701 b ”).
  • the coordinate transformation table creating unit 710 first creates a coordinate conversion table of the imaging apparatus 701 a.
  • the imaging device relative position calculation unit 709 may use a plurality of imaging devices 701 as the imaging device 701 used as a reference. Alternatively, the imaging device relative position calculation unit 709 may calculate the positional relationship of the imaging device 701 in a column.
  • the coordinate conversion table creation unit 710 creates a coordinate conversion table for at least one imaging device 701 based on information (candidate information) related to coordinate conversion. Hereinafter, to simplify the description, it is assumed that the coordinate conversion table creation unit 710 first obtains a coordinate conversion table for one imaging device 701 (imaging device 701 a ).
  • Coordinate conversion table generating unit 710 with respect to imaging device 701 a, using the methods already described, to create a coordinate conversion table. That is, the coordinate transformation table creating unit 710, for each small region groups associated with imaging device 701 a, (more specifically coordinate transformation parameters) coordinate transformation between the camera coordinate system of coordinates and the world coordinate system of the coordinates determining the . Then, the coordinate conversion table creation unit 710 uses the coordinate conversion parameters for each small region group to determine the coordinates of the camera coordinate system and the coordinates of the world coordinate system for all pixels on which the ground surface in the captured image data is displayed. Find the coordinate transformation of. Then, the coordinate conversion table creation unit 710 creates a coordinate conversion table based on the coordinate conversion.
  • the coordinate conversion table generation unit 710 based on the coordinate conversion table created, and information indicating a relative positional relationship between the imaging device 701 a and another imaging device 701 b, other imaging devices 701 b A coordinate conversion table is created (step S709). That is, the coordinate transformation table generation unit 710, based on the information representing the relative positional relationship between the imaging apparatus 701 a and the imaging apparatus 701 b (the rotation matrix and translation vector), the coordinate transformation was developed for capturing apparatus 701 a Convert the table. The converted coordinate transformation table, the coordinate transformation table for imaging apparatus 701 b. Coordinate conversion table generation unit 710 similarly to all of the imaging device 701 b, to create a coordinate conversion table.
  • the coordinate conversion table creation unit 710 stores the created coordinate conversion table in the storage device 711.
  • the eighth embodiment has the effect of reducing the load for creating the coordinate conversion table. The reason is as follows.
  • the imaging device relative position calculation unit 709 calculates information (rotation matrix and translation vector) regarding the relative position of the imaging device 701. Then, the coordinate conversion table creation unit 710 creates a coordinate conversion table for one or more other image capturing apparatuses 701 based on the coordinate conversion table for one or more image capturing apparatuses 701 and information related to the relative position.
  • the process of creating the coordinate conversion table based on the information on the relative position is matrix multiplication and translation vector addition. This process is less burdensome than the process of creating a coordinate conversion table using information (candidate information) related to coordinate conversion acquired from the coordinate input device 708. Therefore, the coordinate conversion table creation device 7 can reduce the processing load for creating the coordinate conversion table.
  • FIG. 25 is a block diagram illustrating an example of a hardware configuration of the coordinate conversion table creation device according to each embodiment of the present invention.
  • the coordinate conversion table creating apparatus shown in FIG. 25 is configured using a computer 1000.
  • the computer 1000 includes a CPU 1001, a main storage device 1002, an auxiliary storage device 1003, an interface 1004, a display device 1005, and an input device 1006.
  • the CPU 1001 realizes the functions of each embodiment based on a program (coordinate conversion program) for realizing the operation of the coordinate conversion table creation device of each embodiment.
  • the CPU 1001 reads a coordinate conversion program from the auxiliary storage device 1003, develops it in the main storage device 1002, and executes the above-described operation based on the program.
  • the CPU 1001 may acquire the program via a communication line (not shown) and store the program in the auxiliary storage device 1003 or expand the program in the main storage device 1002 to execute the above operation.
  • the program acquired via this communication line may be a program for realizing a part of the processing of the CPU 1001 described above.
  • the program acquired via this communication line may be a program (difference program) used in combination with a program stored in the auxiliary storage device 1003.
  • the main storage device 1002 temporarily stores programs executed by the CPU 1001 and data.
  • the main storage device 1002 is, for example, a D-RAM (Dynamic-RAM).
  • the auxiliary storage device 1003 stores the program (coordinate conversion program) and fixed data of each embodiment.
  • the auxiliary storage device 1003 is a non-volatile tangible medium that can be read by a computer.
  • the auxiliary storage device 1003 may be a storage device connected via the interface 1004.
  • the auxiliary storage device 1003 may operate as a storage device, a three-dimensional terrain database storage unit, and / or a measurement ease database storage unit.
  • the auxiliary storage device 1003 is, for example, a magnetic disk, a magneto-optical disk, a CD (Compact Disc) -ROM (Read Only Memory), a DVD (Digital Versatile Disc) -ROM, or a semiconductor memory.
  • the interface 1004 relays transmission and reception of information between the CPU 1001 and an external device.
  • the interface 1004 is, for example, a USB (Universal Serial Bus) card or a LAN (Local Area Network) card.
  • the display device 1005 is a device that displays information to the operator of the computer 1000.
  • the display device 1005 is, for example, a liquid crystal display.
  • the input device 1006 is a device that receives an input operation from an operator of the computer 1000.
  • the input device 1006 is, for example, a keyboard, a mouse, or a touch panel.
  • the coordinate matching system of each embodiment may be realized using the computer 1000 shown in FIG.
  • the display device 1005 may operate as a display device.
  • the input device 1006 may operate as a coordinate input device.
  • each component of the coordinate conversion table creation device may be configured with a hardware circuit. Further, in the coordinate conversion table creation device, each component may be configured using a plurality of devices connected via a network (not shown). In the coordinate conversion table creation device, the plurality of components may be configured with a single piece of hardware.
  • FIG. 26 is a block diagram showing an example of the configuration of the information processing apparatus 10 corresponding to the outline of the coordinate conversion table creation apparatuses 1 to 7 according to the embodiment of the present invention. That is, the information processing apparatus 10 is an apparatus corresponding to the minimum configuration of the coordinate conversion table creation apparatus.
  • the information processing apparatus 10 includes a small area grouping unit 71 corresponding to a small area grouping unit and a coordinate conversion table creating unit 72 corresponding to a coordinate conversion table creating unit.
  • the small area grouping means 71 converts a small area formed based on a point having three-dimensional information on the terrain corresponding to the area included in the photographed image data photographed by the photographing device into a normal direction and an altitude of the small area. Based on this, it is divided into small area groups, which are small area groups.
  • the coordinate conversion table creating means 72 uses the group of small regions and the information related to coordinate conversion, the coordinates of the camera coordinate system, which are the coordinates in the captured image data, and the coordinates of the world coordinate system, which are the coordinates of the 3D terrain data.
  • a coordinate conversion table showing the coordinate conversion of is created.
  • the coordinate conversion table may be referred to as a correspondence table.
  • the information processing apparatus 10 realizes coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system regardless of the target region, as in the first embodiment.
  • the effect of. More specifically, the information processing apparatus 10 can determine the coordinates of the camera coordinate system and the world coordinate system even in an area where no landmark exists, an area of undulating complex terrain, or an area where a survey vehicle cannot travel. Coordinates can be converted between coordinates.
  • FIG. 27 is a block diagram showing an example of the configuration of the information processing system 800 corresponding to the outline of the coordinate association systems 100 to 700 according to the embodiment of the present invention.
  • the information processing system 800 is a system corresponding to the minimum configuration of the coordinate association systems 100 to 700.
  • the information processing system 800 includes a display unit 81 corresponding to a display device, an input unit 82 corresponding to a coordinate input device, and an association unit 83 corresponding to the information processing device 10 (coordinate conversion table creation device).
  • association unit 83 corresponds to the information processing apparatus 10.
  • the display unit 81 displays the three-dimensional information received from the association unit 83 and / or the measurement point candidate.
  • the input means 82 acquires candidate information on 3D terrain data and / or measurement point candidates.
  • the information processing system 800 has the same effects as those of the first embodiment.
  • a small area formed on the basis of a point having three-dimensional information regarding the position on the terrain corresponding to the area included in the photographed image data photographed by the photographing device is determined based on the normal direction and the altitude of the small area.
  • An information processing apparatus comprising: a coordinate conversion table creating means for creating a coordinate conversion table that is conversion information.
  • (Appendix 2) It further includes position and orientation acquisition means for acquiring the distance from the imaging device to the small area and the orientation of the imaging device, Small area grouping means The information processing apparatus according to claim 1, further dividing the small area group based on the distance and the direction.
  • (Appendix 3) Calculate the center of gravity of the small area group and obtain candidate information that is information related to coordinate conversion between the coordinates of the camera coordinate system and the coordinates of the world coordinate system. Extract a point included in a predetermined number of small region groups that are separated by a predetermined distance or more and the angle between all line segments connecting each point included in the small region group and the center of gravity is equal to or greater than the predetermined angle. Further including a measuring point extracting means for The coordinate conversion table creation means The information processing apparatus according to Supplementary Note 1 or Supplementary Note 2, wherein candidate information for a measurement point candidate is acquired and a coordinate conversion table is created based on the candidate information.
  • Measurement 4 And further comprising a measurement ease data storage means for storing measurement ease data, which is data obtained by quantifying the ease of measurement of coordinates in the coordinates of the world coordinate system, Measuring point extraction means Extract measurement point candidates based on the measurement ease data.
  • measurement ease data which is data obtained by quantifying the ease of measurement of coordinates in the coordinates of the world coordinate system
  • Measuring point extraction means Extract measurement point candidates based on the measurement ease data.
  • the coordinate conversion table creation means Coordinate conversion for creating a coordinate conversion table in a small area group for which a coordinate conversion table has not been created, based on parameters used for coordinate conversion for creating a coordinate conversion table in a small area group for which a coordinate conversion table has been created.
  • the information processing apparatus according to appendix 3 or appendix 4, wherein the parameter is estimated and a coordinate conversion table of a small area group for which no coordinate conversion table has been generated is generated based on the estimated parameter.
  • Appendix 6 The information processing apparatus according to any one of appendix 3 to appendix 5, further including route calculation means for calculating a shortest route that passes through all the measurement point candidates.
  • (Appendix 7) Including multiple photographic devices, Based on the reliability of the coordinates of the world coordinate system and the coordinates of the camera coordinate system in the measurement point candidates, further includes a measurement point grouping means for creating a group of measurement point candidates for each photographing device,
  • the coordinate conversion table creation means The information processing apparatus according to any one of supplementary notes 3 to 6, wherein candidate information is acquired for each photographing apparatus and a coordinate conversion table is created.
  • (Appendix 8) Including multiple photographic devices, Based on the candidate information of the measurement point candidate, further includes an imaging device relative position calculation means for calculating information on the relative position of the imaging device,
  • the coordinate conversion table creation means A coordinate conversion table is created based on the candidate information of at least one imaging device, and the coordinate conversion table of the imaging device for which no coordinate conversion table is created based on the created coordinate conversion table and information relating to the imaging device and relative position.
  • Appendix 9 The information processing apparatus according to any one of appendices 1 to 8, Display means for displaying three-dimensional information received from the information processing apparatus and / or measurement point candidates; An information processing system including input means for acquiring candidate information on three-dimensional information and / or measurement point candidates.
  • a small area formed on the basis of a point having three-dimensional information regarding the position on the terrain corresponding to the area included in the photographed image data photographed by the photographing device is determined based on the normal direction and the altitude of the small area.
  • Divide into small area groups that are groups, Coordinates between the coordinates of the camera coordinate system, which are the coordinates in the captured image data, and the coordinates of the world coordinate system, which is the coordinates of the small area corresponding to the captured image data, using the small area group and candidate information that is information relating to coordinate transformation
  • a small area formed on the basis of a point having three-dimensional information regarding the position on the terrain corresponding to the area included in the photographed image data photographed by the photographing device is determined based on the normal direction and the altitude of the small area.
  • Processing to divide into small area groups that are groups, Coordinates between the coordinates of the camera coordinate system, which are the coordinates in the captured image data, and the coordinates of the world coordinate system, which is the coordinates of the small area corresponding to the captured image data, using the small area group and candidate information that is information relating to coordinate transformation
  • a recording medium on which a computer-readable program for causing a computer to execute a process of creating a coordinate conversion table as conversion information is recorded.
  • the present invention is suitable for associating the coordinates of the camera coordinate system in the image photographed by the camera with the coordinates of the world coordinate system in the three-dimensional terrain data.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Remote Sensing (AREA)
  • Computer Graphics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

En vue de calculer une transformation de coordonnées entre des coordonnées d'un système de coordonnées de caméra et les coordonnées d'un système de coordonnées universel, indépendamment de régions cibles, un appareil de traitement d'informations selon la présente invention comprend : un moyen de regroupement de petites régions destiné à diviser des petites régions, qui sont formées sur la base de points ayant des informations tridimensionnelles concernant un emplacement dans un terrain correspondant à une région incluse dans des données d'images capturées qui sont capturées par un dispositif de capture d'image, en groupes de petite région qui sont des groupes d'une petite région, sur la base de la direction normale et de l'altitude de la petite région ; et un moyen de création de table de transformation de coordonnées destiné à créer une table de transformation de coordonnées qui contient des informations pour la transformation de coordonnées entre les coordonnées du système de coordonnées de caméra, qui sont des coordonnées dans les données d'images capturées, et des coordonnées du système de coordonnées universel, qui sont des coordonnées dans une petite région qui correspondent aux données d'images capturées, à l'aide des groupes de la petite région et des informations de candidats qui sont associées à la transformation de coordonnées.
PCT/JP2016/001595 2015-03-27 2016-03-18 Appareil de traitement d'informations, système de traitement d'informations, procédé de traitement d'informations et support d'informations Ceased WO2016157802A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2017509251A JP6683195B2 (ja) 2015-03-27 2016-03-18 情報処理装置、情報処理システム、情報処理方法、及び、プログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2015065610 2015-03-27
JP2015-065610 2015-03-27

Publications (1)

Publication Number Publication Date
WO2016157802A1 true WO2016157802A1 (fr) 2016-10-06

Family

ID=57004900

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/001595 Ceased WO2016157802A1 (fr) 2015-03-27 2016-03-18 Appareil de traitement d'informations, système de traitement d'informations, procédé de traitement d'informations et support d'informations

Country Status (2)

Country Link
JP (1) JP6683195B2 (fr)
WO (1) WO2016157802A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109785376A (zh) * 2017-11-15 2019-05-21 富士通株式会社 深度估计装置的训练方法、深度估计设备及存储介质
JP2019121176A (ja) * 2018-01-05 2019-07-22 オムロン株式会社 位置特定装置、位置特定方法、位置特定プログラムおよびカメラ装置
CN113763520A (zh) * 2020-06-01 2021-12-07 北京沃东天骏信息技术有限公司 一种指示三维模型方位的方法和装置
CN114332228A (zh) * 2021-12-30 2022-04-12 高德软件有限公司 数据处理方法、电子设备及计算机存储介质
CN115861047A (zh) * 2021-09-23 2023-03-28 中移(上海)信息通信科技有限公司 一种遥感影像坐标系转换的控制方法、装置及终端

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08240421A (ja) * 1995-03-06 1996-09-17 Meidensha Corp カメラパラメータの推定方法およびそのパラメータ利用の物体認識方法
JPH08255239A (ja) * 1995-03-17 1996-10-01 Hitachi Denshi Ltd 地理情報処理装置
JPH09305746A (ja) * 1996-05-21 1997-11-28 Oki Electric Ind Co Ltd 地形データ補間装置
JP2004133094A (ja) * 2002-10-09 2004-04-30 Zenrin Co Ltd ３次元電子地図データの生成方法
US20110122257A1 (en) * 2009-11-25 2011-05-26 Honeywell International Inc. Geolocation of objects in an area of interest
JP2011133840A (ja) * 2009-11-27 2011-07-07 Asia Air Survey Co Ltd 監視カメラ画像を用いた連動表示計測システム
JP2013088999A (ja) * 2011-10-18 2013-05-13 Aero Asahi Corp 建物抽出装置、方法及びプログラム
JP2014137321A (ja) * 2013-01-18 2014-07-28 Nec Corp 位置座標変換システム、位置座標変換方法、車載装置、世界座標計測装置および位置座標変換プログラム

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08240421A (ja) * 1995-03-06 1996-09-17 Meidensha Corp カメラパラメータの推定方法およびそのパラメータ利用の物体認識方法
JPH08255239A (ja) * 1995-03-17 1996-10-01 Hitachi Denshi Ltd 地理情報処理装置
JPH09305746A (ja) * 1996-05-21 1997-11-28 Oki Electric Ind Co Ltd 地形データ補間装置
JP2004133094A (ja) * 2002-10-09 2004-04-30 Zenrin Co Ltd ３次元電子地図データの生成方法
US20110122257A1 (en) * 2009-11-25 2011-05-26 Honeywell International Inc. Geolocation of objects in an area of interest
JP2011133840A (ja) * 2009-11-27 2011-07-07 Asia Air Survey Co Ltd 監視カメラ画像を用いた連動表示計測システム
JP2013088999A (ja) * 2011-10-18 2013-05-13 Aero Asahi Corp 建物抽出装置、方法及びプログラム
JP2014137321A (ja) * 2013-01-18 2014-07-28 Nec Corp 位置座標変換システム、位置座標変換方法、車載装置、世界座標計測装置および位置座標変換プログラム

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109785376A (zh) * 2017-11-15 2019-05-21 富士通株式会社 深度估计装置的训练方法、深度估计设备及存储介质
CN109785376B (zh) * 2017-11-15 2023-02-28 富士通株式会社 深度估计装置的训练方法、深度估计设备及存储介质
JP2019121176A (ja) * 2018-01-05 2019-07-22 オムロン株式会社 位置特定装置、位置特定方法、位置特定プログラムおよびカメラ装置
CN113763520A (zh) * 2020-06-01 2021-12-07 北京沃东天骏信息技术有限公司 一种指示三维模型方位的方法和装置
CN115861047A (zh) * 2021-09-23 2023-03-28 中移(上海)信息通信科技有限公司 一种遥感影像坐标系转换的控制方法、装置及终端
CN114332228A (zh) * 2021-12-30 2022-04-12 高德软件有限公司 数据处理方法、电子设备及计算机存储介质

Also Published As

Publication number Publication date
JPWO2016157802A1 (ja) 2018-02-01
JP6683195B2 (ja) 2020-04-15

Similar Documents

Publication Publication Date Title
US11237005B2 (en) Method and arrangement for sourcing of location information, generating and updating maps representing the location
US10240934B2 (en) Method and system for determining a position relative to a digital map
CN107850445B (zh) 用于生成及使用定位参考数据的方法及系统
JP5389964B2 (ja) 地図情報生成装置
KR101444685B1 (ko) 영상기반 멀티센서 데이터를 이용한 차량의 위치자세 결정 방법 및 장치
JP6178704B2 (ja) 計測点高付与システム、計測点高付与方法および計測点高付与プログラム
JP6683195B2 (ja) 情報処理装置、情報処理システム、情報処理方法、及び、プログラム
WO2016113904A1 (fr) Dispositif de calcul d'informations tridimensionnelles, procédé de calcul d'informations tridimensionnelles, et dispositif mobile autonome
JP6589410B2 (ja) 地図生成装置及びプログラム
JP7431320B2 (ja) デジタル地図データを使用する方法およびシステム
KR102174729B1 (ko) 랜드마크를 활용한 주행차로 인식 방법 및 시스템
JP5814620B2 (ja) 位置補正データ生成装置、位置標定装置、位置補正データ生成装置の位置補正データ生成方法、位置標定装置の位置標定方法、位置補正データ生成プログラムおよび位置標定プログラム
US12002237B2 (en) Position coordinate derivation device, position coordinate derivation method, position coordinate derivation program, and system
JPWO2021111613A1 (ja) ３次元地図作成装置、３次元地図作成方法、及び３次元地図作成プログラム
Khoshelham et al. Vehicle positioning in the absence of GNSS signals: Potential of visual-inertial odometry
JP7682607B2 (ja) 情報処理装置、制御方法、プログラム及び記憶媒体
Koppanyi et al. Experiences with acquiring highly redundant spatial data to support driverless vehicle technologies
JP7345153B2 (ja) 飛翔体の地理座標推定装置、地理座標推定システム、地理座標推定方法、及びコンピュータプログラム
Zomrawi et al. Accuracy evaluation of digital aerial triangulation
Wu et al. An Assessment of Errors Using Unconventional Photogrammetric Measurement Technology-with UAV Photographic Images as an Example
Lin et al. Research on the production of 3D image cadastral map
JP7562816B1 (ja) 点群処理装置、点群処理方法及びプログラム
KR102612216B1 (ko) 지면정보를 활용한 관심지점지도와 6자유도 단말 자세 정합 방법 및 시스템
Geitner et al. Spatial Grid-Based Object Localization from A Single Passive Sensor: A Deep Learning-Integrated Approach
Mattia et al. A low cost solution as an alternative to traditional mobile mapping system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16771695

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017509251

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16771695

Country of ref document: EP

Kind code of ref document: A1