WO2025177925A1 - Information processing device, information processing method, and information processing system - Google Patents

Abstract

This information processing device is provided with: a display processing unit that displays, on a display unit, geographical information created on the basis of images captured by a camera provided on a moving body; an associating unit that associates re-imaging information for re-imaging by the camera with the geographic information on the basis of an input from a user; and a movement plan creating unit that creates a movement plan for the moving body for performing the re-imaging by means of the camera on the basis of the re-imaging information.

Description

Information processing device, information processing method, and information processing system

This technology relates to an information processing device, an information processing method, and an information processing system.

Prior art includes technology for creating point clouds and orthoimages representing terrain from multiple pieces of image data captured by a camera mounted on a mobile object such as a drone. Furthermore, technology has been proposed for re-photographing an object when the subject fails to be photographed using such a mobile object (Patent Document 1).

Japanese Patent Application Laid-Open No. 2020-191523

In the technology described in Patent Document 1, if a failure to capture an object is detected, the object is re-photographed from the position where the failure was detected. However, in order to create highly accurate geographic information such as point clouds, it is important to properly create a movement plan for the moving object, specifying the route and area through which the object will be moved to re-photograph the object.

This technology was developed in consideration of these problems, and aims to provide an information processing device, information processing method, and information processing system that can create a travel plan for re-photographing for creating geographic information.

In order to solve the above-mentioned problems, the first technology is an information processing device that includes a display processing unit that displays geographic information created based on images captured by a camera carried by a moving object on a display unit, an association unit that associates re-photographing information for re-photographing by the camera with the geographic information based on input from a user, and a movement plan creation unit that creates a movement plan for the moving object to re-photograph by the camera based on the re-photographing information.

The second technology is an information processing method that displays geographic information created based on images captured by a camera carried by a moving object on a display unit, associates re-photographing information for re-shooting with the geographic information based on input from the user, and creates a movement plan for the moving object to re-shoot with the camera based on the re-photographing information.

The third technology is an information processing system comprising an information processing device that includes a geographic information creation device that creates geographic information based on images captured by a camera equipped on a moving object, a display processing unit that displays the geographic information on a display unit, an association unit that associates re-photographing information for re-photographing with the camera based on input from a user with the geographic information, and a movement plan creation unit that creates a movement plan for the moving object to re-photograph with the camera based on the re-photographing information.

FIG. 1 is a diagram illustrating a configuration of an information processing system 1000. FIG. 1 is a block diagram showing the configuration of a moving body 100. FIG. 2 is a block diagram showing the configuration of a geographic information creation device 200. FIG. 2 is a block diagram showing the hardware configuration of an information processing device 300. FIG. 3 is a block diagram showing a configuration of a processing block of the information processing device 300. 10 is a flowchart showing processing in the information processing system 1000. FIG. 10 is a diagram showing geographic information displayed on the display unit 307 in 2D. FIG. 10 is a diagram showing geographic information displayed on the display unit 307 in 3D. FIG. 10 is an explanatory diagram of an input for associating re-photographing information with geographic information. FIG. 10 is an explanatory diagram of re-photographing information associated with geographic information. FIG. 1 is an explanatory diagram of oblique photography. FIG. 10 is an explanatory diagram of association of re-photographing information with an orthoimage. FIG. 1 is an explanatory diagram of a movement plan. FIG. 10 is an explanatory diagram of the reliability of geographic information. This is an example of how importance is displayed when using this technology in as-built management.

Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be made in the following order.
<Embodiment>
[Configuration of information processing system 1000]
[Configuration of moving body 100]
[Configuration of geographic information creation device 200]
[Configuration of information processing device 300]
[Processing in Information Processing System 1000]
<Modification>

<Embodiment>
[Configuration of information processing system 1000]
As shown in FIG. 1, the information processing system 1000 includes a mobile object 100, a geographic information creation device 200, and an information processing device 300.

The mobile object 100 is connected to a geographic information creation device 200. The geographic information creation device 200 is also connected to an information processing device 300. The information processing device 300 is also connected to the mobile object 100. In this embodiment, the mobile object 100 is an unmanned aerial vehicle flying in the sky, and is therefore connected to the geographic information creation device 200 and the information processing device 300 via a wireless connection. The geographic information creation device 200 is connected to the information processing device 300 via a wired or wireless connection.

Wired connection methods include, for example, HDMI (registered trademark) (High-Definition Multimedia Interface) and USB (Universal Serial Bus), while wireless connection methods include, for example, Wi-Fi, wireless LAN (Local Area Network), networks such as 4G (fourth generation mobile communication system) and 5G (fifth generation mobile communication system), Bluetooth (registered trademark), NFC (Near Field Communication), and Ethernet (registered trademark). The supply of data and information from the mobile object 100 to the geographic information creation device 200, the supply of data and information from the geographic information creation device 200 to the information processing device 300, and the supply of data and information from the information processing device 300 to the mobile object 100 may be performed via recording media such as USB flash memory and SD memory cards.

The mobile object 100 is an unmanned aerial vehicle such as a drone that flies in the sky. The mobile object 100 is equipped with a camera 105, and can perform automatic flight and automatic photographing based on a pre-set movement route (pre-movement route) or a movement plan created by the information processing device 300. The pre-set movement route can be set using existing mobile object control software, etc.

When performing automatic flight and automatic photography, route information, photography position, photography direction, photography timing, etc. are set in advance, and the mobile body 100 performs flight control and photography control in accordance with these settings. The mobile body 100 flies above the photography range, i.e., the range for which geographic information is to be created (geographic information creation range), and photographs the ground surface with the camera 105 while flying. The mobile body 100 also obtains camera position information, which is the position of the camera 105 at the time of photography, and transmits the image and camera position information to the geographic information creation device 200. In addition to automatic flight, the mobile body 100 may also be capable of being flown manually by a pilot.

The geographic information creation device 200 creates geographic information for the geographic information creation range based on images captured by the camera 105 of the mobile object 100. The geographic information may be a three-dimensional point cloud, orthoimages, etc.

A 3D point cloud is a collection of data consisting of multiple points with location and color information. It can represent terrain, objects, and more as a large collection of points, and is geographic information that can be used in a variety of fields, including civil engineering, architecture, and manufacturing.

An orthoimage is an image that has been transformed (orthotransformed) to eliminate misalignment of the images on the image and display the aerial photograph in the correct size and position without tilt, just like a map viewed from directly above. Because the shape and position of the captured image are correct in an orthoimage, it is possible to accurately measure position, area, distance, etc. on the image, and it is geospatial information that can be overlaid on map data and used.

In addition, the geographic information creation device 200 estimates the orientation of the camera 105 at the time of image capture based on the image captured by the camera 105 of the moving object 100. The geographic information creation device 200 transmits the geographic information, camera position information, and camera orientation information to the information processing device 300.

The information processing device 300 displays a GUI (Graphical User Interface) to present geographical information, camera position information, camera posture information, and other information to the user. The information processing device 300 also associates re-photographing information with geographical information based on input from the user, and creates a movement plan based on the re-photographing information.

Re-photography information is information that is associated with geographic information in order to create a travel plan. In this technology, re-photography information consists of three pieces of information: designated route information, priority area information, and unnecessary area information. Details of re-photography information will be provided later.

The movement plan is created by the information processing device 300 based on the re-imaging information, and indicates the movement route of the moving body 100 for re-imaging. Details of the movement route will be described later.

[Configuration of moving body 100]
The configuration of the moving body 100 will be described with reference to Fig. 2. Although not shown, the moving body 100 has an exterior configuration including a housing, rotors, and rotor support shafts. However, the rotors are not limited to rotors and may be fixed wings.

The UAV (Unmanned Aerial Vehicle) control unit 101 is composed of a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and controls the entire moving body 100 and each part by executing processes and issuing commands according to programs. The UAV control unit 101 also controls the moving speed, moving direction, turning direction, etc. of the moving body 100 by supplying control signals to the actuators 102 that control the output of the actuators 102.

The UAV control unit 101 also controls the output of the actuator 102 while comparing the current position of the moving body 100 with the previous movement route, thereby controlling the moving body 100 to move along the previous movement route. The UAV control unit 101 also controls the output of the actuator 102 while comparing the current position of the moving body 100 with the movement plan transmitted from the information processing device 300, thereby controlling the moving body 100 to move along the movement plan.

Actuator 102 is a drive source for driving the rotor blades, and is provided at the tip of the rotor blade support shaft, etc. Actuator 102 operates under the control of UAV control unit 101.

The gimbal 103 supports the camera 105 so that it can rotate freely, for example, at the bottom of the moving body 100. The gimbal 103 is a type of support device that provides support on, for example, two or three axes.

The gimbal control unit 104 controls the operation of the gimbal 103 that supports the camera 105. By controlling the rotation of the axis of the gimbal 103 with the gimbal control unit 104, the attitude of the camera 105 can be freely adjusted. This makes it possible to adjust the attitude of the camera 105 to match the movement plan created by the information processing device 300 and take pictures.

Camera 105 is equipped with an imaging element, signal processing circuitry, etc., and captures RGB (Red, Green, Blue) or monochrome images. The imaging element may be a CCD (Charge Coupled Device), CMOS (Complementary Metal Oxide Semiconductor), etc.

Camera 105 can capture a subject and obtain an image as digital data. Camera 105 may also have the function of performing AI-based processing on the image. Examples of this processing include image recognition processing and image detection processing.

Camera 105 is mounted, for example, on the bottom of the housing of mobile body 100 so as to be suspended via gimbal 103 in order to capture images of the ground surface within the geographic information creation range. By driving gimbal 103, camera 105 is able to point its lens in any direction from 360 degrees horizontally to vertically downwards to capture images. Note that camera 105 may be mounted on mobile body 100 in any manner as long as it is able to point its lens in any direction from 360 degrees horizontally to vertically downwards to capture images.

The location information acquisition unit 106 acquires the location (e.g., latitude, longitude, altitude) of the camera 105 using, for example, a GNSS (Global Navigation Satellite System) device or a GPS (Global Positioning System) device. Note that since the location of the camera 105 is the location of the moving body 100, it is also possible to detect the speed of the moving body 100 from changes in location information and elapsed time.

The attitude information acquisition unit 107 is configured, for example, by an IMU (Inertial Measurement Unit), and acquires the attitude of the moving body 100 and camera 105 by determining three-dimensional angular velocity and acceleration using a two-axis or three-axis acceleration sensor, an angular velocity sensor, a gyro sensor, etc. The attitude information is, for example, a three-axis rotation vector (Yaw, Pitch, Roll) calculated from the acceleration and angular velocity of the moving body 100.

The sensor control unit 108 controls the acquisition of images by the camera 105 and the acquisition of camera position information by the position information acquisition unit 106 so that these are synchronized. The sensor control unit 108 also attaches time information, such as a timestamp, to the image and camera position information to indicate that they are synchronized.

The communication unit 109 is a communication terminal or communication module for various purposes, such as processing communications via transmission paths such as the Internet, and communicating with various devices via wired/wireless communication, bus communication, etc. The mobile object 100 transmits images and camera position information to the geographic information creation device 200 via the communication unit 109. Note that the mobile object 100 may also transfer this information to the geographic information creation device 200 via an external storage medium such as a USB flash memory or SD memory card.

In addition, a recording medium such as a semiconductor memory may be used, and after the mobile body 100 lands, the recording medium may be removed from the mobile body 100 and connected to a recording medium connection slot (not shown) in the geographic information creation device 200, allowing images and camera position information to be transferred.

A mobile object 100, such as a drone, controls the output of actuators 102 to enable desired flight. When hovering in the air, the attitude information acquisition unit 107 or a separately provided gyro sensor detects tilt, and the output of the actuator 102 on the side where the aircraft is lowered is increased and the output of the actuator 102 on the side where the aircraft is raised is decreased, thereby keeping the aircraft horizontal. Furthermore, when moving forward, the output of the actuator 102 in the direction of travel is decreased and the output of the actuator 102 in the opposite direction is increased, thereby causing the aircraft to assume a forward-leaning attitude and generating propulsion in the direction of travel.

The mobile object 100 may be equipped with a 360-degree camera that can capture images of the surrounding area. The 360-degree camera can capture images of clouds and the sun, which can be used for various processing purposes.

The mobile body 100 may be equipped with an altimeter, a compass, etc. The altimeter may be a barometric altimeter or a radio altimeter that measures the altitude at which the mobile body 100 is located. The compass uses the action of a magnet to detect the direction in which the mobile body 100 is traveling.

In this embodiment, the mobile object 100 is equipped with sensors such as a camera 105, a position information acquisition unit 106, and an attitude information acquisition unit 107, but a configuration in which a standalone sensor device having these functions is mounted on the mobile object 100 is also possible.

[Configuration of geographic information creation device 200]
Next, the configuration of the geographic information creation device 200 will be described with reference to FIG.

The receiving unit 201 receives images and camera position information transmitted from the mobile object 100 and supplies them to the geographic information creation unit 202 and the attitude estimation unit 203.

The geographic information creation unit 202 creates geographic information based on the images supplied from the receiving unit 201. The geographic information creation unit 202 creates a three-dimensional point cloud using, for example, the SfM (Structure From Motion) method. The geographic information creation unit 202 may create an orthoimage instead of, or in addition to, the three-dimensional point cloud. The geographic information that the geographic information creation unit 202 creates may be set in advance, or may be selectable by the user. The orthoimage can be created using an existing method such as SfM-MVS (Structure-from-Motion/Multi-View-Stereo).

The attitude estimation unit 203 estimates the attitude of the camera 105 at the time of shooting based on the image supplied from the receiving unit 201 and outputs camera attitude information. One method for determining the attitude of the camera 105 from the captured image is SfM. Note that the camera attitude information may also be acquired by the attitude information acquisition unit 107 provided in the moving body 100.

The transmitting unit 204 transmits the geographic information, camera position information, and camera attitude information to the information processing device 300 via a transmission path such as the Internet or the communication method described above. The geographic information creation device 200 may transfer this information to the information processing device 300 via an external storage medium such as a USB flash memory or an SD memory card. The geographic information creation device 200 may also transmit this information to the cloud or a server.

The geographic information creation device 200 is installed as a fixed station within the geographic information creation range. The geographic information creation device 200 may be configured as a stand-alone device, or may be configured as an electronic device with information processing and communication functions, such as a personal computer, smartphone, or tablet terminal. The geographic information creation device 200 may also be realized by an electronic device with computer functions executing a program. The program may be pre-installed on the electronic device, or may be distributed by download or storage medium, etc., and installed by a user, etc. The geographic information creation device 200 may also be configured as a cloud server.

[Configuration of information processing device 300]
Next, the hardware configuration of the information processing device 300 will be described with reference to FIG.

CPU 301 functions as an arithmetic processing unit that performs various processes, and controls the entire information processing device 300 and each unit. CPU 301 executes various processes in accordance with programs stored in ROM 302 or non-volatile memory unit 304, or programs loaded from storage unit 308 to RAM 303. Non-volatile memory unit 304 may, for example, be an EEPROM (Electrically Erasable Programmable Read Only Memory). RAM 303 also stores data, etc., necessary for CPU 301 to execute various processes, as appropriate.

The CPU 301, ROM 302, RAM 303, and non-volatile memory unit 304 are interconnected via a bus. The bus is also connected to an input/output interface 305.

The input/output interface 305 is connected to an input unit 306, a display unit 307, a memory unit 308, a communication unit 309, and a drive 310.

The input unit 306 is, for example, a variety of controls or operating devices such as a keyboard, mouse, keys, dial, touch panel, touch pad, remote controller, etc. The input unit 306 detects user operations, and the signals corresponding to the input operations are interpreted by the CPU 301.

The display unit 307 may be configured, for example, as a display device such as a liquid crystal display or organic electroluminescence (EL) display provided on the housing of the computer device, or as a separate display device connected to the computer device. The display unit 307 may be connected to the input/output interface 305 either integrally or separately.

The display unit 307 displays various images, videos, etc. on the display screen based on instructions from the CPU 301. The display unit 307 also displays various operation menus, icons, messages, etc. based on instructions from the CPU 301. The display unit 307 also displays a GUI (Graphical User Interface) for displaying geographic information related to the present technology.

The storage unit 308 is a large-capacity storage medium such as a hard disk or flash memory. Various applications, data, information, etc. are stored in the storage unit 308.

The communication unit 309 is a communication terminal or communication module for various purposes, performing communication processing via transmission paths such as the Internet, and communicating with various devices via wired/wireless communication, bus communication, etc. The information processing device 300 receives geographic information, camera position information, and camera attitude information transmitted from the geographic information creation device 200 via the communication unit 309. The information processing device 300 also transmits a movement plan to the mobile object 100 via the communication unit 309. The information processing device 300 may transfer information and data to the geographic information creation device 200 and the mobile object 100 via external storage media such as USB flash memory or an SD memory card.

A drive 310 is connected to the input/output interface 305 as needed. A removable storage medium 311 is attached via the drive 310 as needed. Removable storage medium 311 includes magnetic disks, optical disks, magneto-optical disks, semiconductor memory, etc.

The drive 310 can read data files such as programs used for various processes from the removable storage medium 311. The read data files are stored in the storage unit 308. Images contained in the data files are displayed on the display unit 307. Computer programs read from the removable storage medium 311 are installed in the storage unit 308 as needed.

In the information processing device 300, for example, software for processing the present technology can be installed via network communication via the communication unit 309 or via removable storage medium 311. The software may also be stored in advance in ROM 302, storage unit 308, etc. Images captured by the camera 105 and the results of AI image processing may also be received, and the images and processing results may be stored in the storage unit 308 or removable storage medium 311.

Next, the configuration of the processing blocks in the information processing device 300 will be explained with reference to Figure 5.

The information acquisition unit 321 acquires the geographic information, camera position information, and camera attitude information sent from the geographic information creation device 200 and received by the communication unit 309, and supplies them to the display processing unit 323. The information acquisition unit 321 also supplies the geographic information to the anomaly detection unit 322 and the association unit 324.

The anomaly detection unit 322 detects anomalies in the geographic information. If there is a defect in the photography by the camera 105, an anomaly will occur in the geographic information. Anomalies in the three-dimensional point cloud as geographic information include sparse points, or points not existing in areas where points should be present. Furthermore, anomalies in the orthoimage as geographic information include image distortion, missing images (exposing the background), and objects hidden by shadows.

The display processing unit 323 performs processing to display geographic information, camera position information, camera attitude information, the movement path of the mobile object 100, and abnormality detection results on the display unit 307.

The association unit 324 performs processing to associate re-photographing information for creating a travel plan with geographic information based on input from the user.

The coordinate conversion unit 325 converts the re-photographing information expressed in the site coordinate system into the drone coordinate system. The site coordinate system is a coordinate system whose coordinate axes are set at an arbitrary origin set at a local location such as a construction site. The drone coordinate system is a global coordinate system expressed in latitude and longitude.

The movement plan creation unit 326 creates a movement plan based on the re-photography information associated with the geographic information.

The information processing device 300 is configured as described above. The information processing device 300 is configured from an electronic device with information processing and communication functions, such as a personal computer, smartphone, or tablet terminal. The processing blocks and information processing method of the information processing device 300 according to the present technology may be realized by these electronic devices executing a program. The program may be installed in the electronic device in advance, or may be distributed by download or storage medium, etc., and installed by a user, etc. The processing unit of the information processing device 300 may also be configured in a cloud server.

A cloud server is not limited to being composed of a single computer device, but may also be composed of a system of multiple computer devices. Multiple computer devices may be systemized, for example, using a LAN (Local Area Network). Additionally, multiple computer devices located in remote locations may be systemized using a VPN (Virtual Private Network) that uses the Internet, etc. Multiple computer devices may also include computer devices that are a group of servers (cloud) that can be used by a cloud computing service.

The functions provided by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and/or combinations thereof, programmed to provide the described functions. Processors include transistors and other circuits and are considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes a program stored in memory. In this specification, a circuit, unit, or means is hardware that is programmed to provide or executes the described functions. The hardware may be any hardware disclosed herein or any hardware that is programmed to provide or is known to execute the described functions. When the hardware is a processor, which is considered to be a type of circuitry, the circuitry, means, or unit is a combination of hardware and software used to configure the hardware and/or processor.

[Processing in Information Processing System 1000]
Next, the processing in the information processing system 1000 will be described with reference to FIG.

First, in step S11, the mobile object 100 moves (flies) above the geographic information creation range based on a pre-created preliminary movement route, and photographs are taken by the camera 105. While the mobile object 100 is moving along the pre-created movement route, the camera 105 continues to take photographs repeatedly at predetermined intervals.

Furthermore, in synchronization with the image capture by the camera 105, the position information acquisition unit 106 acquires the camera position information at the time of image capture. The mobile object 100 then transmits the image and camera position information to the geographic information creation device 200.

In addition, the mobile object 100 can take photographs while changing its travel route in response to instructions from the user via wireless communication while moving. Furthermore, it can also add or change the timing of taking photographs in response to instructions from the user via wireless communication while moving.

Next, in step S12, the attitude estimation unit 203 of the geographic information creation device 200 estimates the attitude of the camera 105 based on the image captured by the camera 105 and outputs camera attitude information.

Next, in step S13, the geographic information creation unit 202 of the geographic information creation device 200 creates geographic information based on the image. The geographic information creation device 200 transmits the geographic information, camera position information, and camera attitude information to the information processing device 300.

Next, in step S14, the anomaly detection unit 322 of the information processing device 300 detects an anomaly in the geographic information and supplies information on the location where the anomaly exists in the geographic information (anomaly location information) to the display processing unit 323 as the anomaly detection result. Anomalies in a three-dimensional point cloud as geographic information can be detected, for example, by measuring the distance between each point that makes up the point cloud and determining whether that distance is equal to or greater than a predetermined threshold. Furthermore, because the point cloud is created in advance to satisfy a predetermined density, areas where the density of the created point cloud is equal to or less than a predetermined density can be detected as an anomaly.

Next, in step S15, the display processing unit 323 of the information processing device 300 processes the geographic information to display it on the display unit 307. There are two display formats: 2D display and 3D display. In the case of 2D display, the geographic information displayed on the display unit 307 is as shown in Figure 7. In the example of Figure 7, the geographic information is a three-dimensional point cloud. In this embodiment, the ground surface photographed by the camera 105 includes the ground, roads, embankments, buildings, and trees. However, this configuration of the ground surface is merely an example, and the present technology is not limited to such a configuration of the ground surface.

In addition to the geographic information, the display processing unit 323 displays on the display unit 307 the camera position information at the time of shooting, which was acquired in synchronization with the shooting of the image used to create the geographic information. The camera position information is displayed as a point indicating the position on the geographic information. This allows the user to easily understand the position of the camera 105 at which the image was taken.

In addition to the geographic information, the display processing unit 323 also displays camera attitude information on the display unit 307. The camera attitude information is represented by a rectangular frame indicating the angle of view of the camera 105 and lines extending from points indicating the camera position information to each corner of the frame. This allows the user to easily understand the attitude of the camera 105 at which the image was taken. The angle of view can be identified from the image used to create the geographic information.

In addition to the geographic information, the display processing unit 323 also displays on the display unit 307 the movement path of the mobile object 100 in the sky above the geographic information creation range. The movement path is displayed as a line connecting multiple positions indicated by the camera position information in chronological order (photographing order). To achieve this, for example, it is advisable to link time information such as a timestamp to the camera position information on the mobile object 100 and refer to this. By displaying this movement path, the user can easily understand the route the mobile object 100 took to take the photographs used to create the geographic information.

Furthermore, the display processing unit 323 displays, in addition to the geographical information, anomaly location information, which is the anomaly detection result, on the display unit 307. This allows the user to understand where the anomaly is in the geographical information, identify locations or areas within the geographical information creation range that require re-photography, and enter information to associate with the photographing information.

The user can also find abnormalities by looking at the displayed geographical information. Therefore, displaying the abnormality location information is not essential. Displaying the abnormality location information assists the user in entering information to associate it with the shooting information. The abnormality location information may be highlighted, for example by making it easier for the user to understand. In addition to the abnormality location information, information indicating the type of abnormality may also be supplied from the abnormality detection unit 322 to the display processing unit 323 and displayed on the display unit 307.

In the example of Figure 7, camera position information, camera attitude information, movement path, and abnormality location information are displayed along with geographic information, but it is not necessary to display all of this information at the same time. The user may be able to select which information to display. Also, the display or non-display of selected information may be switched depending on the selection input from the user.

Geographical information can also be displayed in 3D. In this case, the geographical information displayed on the display unit 307 will be as shown in Figure 8. The geographical information in Figure 8 is also a three-dimensional point cloud. In the 3D display of Figure 8, camera position information, camera attitude information, movement path, and abnormality position information are displayed superimposed on the geographical information, just like the 2D display of Figure 7. The user may be able to select and switch between 2D and 3D display.

When the user sees the geographical information displayed on the display unit 307, they input information to the input unit 306 to associate it with re-photographing information.

When a user inputs information while viewing the display of geographical information and travel routes shown in Figure 7, the input results will be, for example, as shown in Figure 9.

Next, in step S16, the association unit 324 of the information processing device 300 performs a process of associating the re-photograph information with the geographic information based on the user's input results. The re-photograph information associated with the geographic information based on the input results shown in FIG. 9 becomes as shown in FIG. 10.

As shown in Figure 10, in this technology, re-photographing information includes three types of information: designated route information, priority area information, and unnecessary area information. The re-photographing information is information that specifies either or both of the route and area along which the mobile object 100 will travel in order to re-photograph using the camera 105, and also information that specifies areas into which the mobile object 100 does not need to move.

Designated route information is information that directly specifies the specific travel route of the mobile object 100. To associate designated route information with geographic information, the user inputs a line indicating the designated route on the geographic information displayed on the display unit 307. The tip of the arrow is the end point of the designated route, and the end point on the opposite side of the arrow is the start point of the designated route. The designated route may also be input by specifying the start point and end point. In this case, the straight line connecting the start point and end point becomes the designated route. Multiple designated route information may also be associated with geographic information. Multiple designated route information may be associated with distant positions, or the designated routes may be connected and associated. Travel route information may also be specified as lines other than straight lines, such as curved lines, wavy lines, or free lines.

The specified route information can include the movement speed of the mobile object 100 when moving along the specified route, and the attitude of the camera 105. The movement speed can be input as a specific speed value, or the length of time required to move along the input specified route can be input. The attitude of the camera 105 can be input as a specific angle value, or the attitude of the camera 105 corresponding to the input angle value can be visually displayed using animation or the like.

The specified route information can also be used to specify a travel route for so-called oblique photography, in which the camera 105 is tilted diagonally rather than pointing vertically downward, after photographing the area for creating geographic information by moving back and forth from one side to the other multiple times, as shown in Figure 11, and then photographing while moving diagonally relative to the direction of the round-trip route. Using images obtained through oblique photography to create geographic information can improve the accuracy of the geographic information.

Priority area information does not specify a specific route of travel like designated route information, but rather specifies an area (priority area) within the geographic information creation range that will be photographed with priority.

To associate priority area information with geographic information, the user inputs a priority area on the geographic information displayed on the display unit 307. The priority area may be input as a rectangular area as shown in Figure 9, or as an area of other shape such as a circle or freeform, or may be specified by a waypoint.

In orthoimages, as shown in Figure 12, it is preferable to associate priority area information in order to focus on photographing areas where distortion occurs or where dark or long shadows exist.

Furthermore, it is preferable to associate priority area information in order to focus on capturing areas where blind spots or shadows are created depending on the shooting direction, such as tall buildings. This priority area information makes it possible to focus on capturing areas where occlusion occurs, where the area behind is hidden by a building or object in the foreground.

Unnecessary area information is information that specifies areas (unnecessary areas) that do not need to be photographed and do not need to be included in the movement plan. Areas that do not need to be photographed include, for example, areas covered with trees such as forests and woods, water surfaces such as oceans, lakes and rivers, and snowy surfaces. However, unnecessary areas are not limited to these, and can be any area that the user considers unnecessary to photograph for the purpose of creating geographic information.

To associate unnecessary area information with geographic information, the user inputs information to identify the unnecessary area on the geographic information displayed on the display unit 307. The unnecessary area may be input as a rectangular area as shown in Figure 9, or as an area of other shape such as a circle or freeform, or may be specified by a waypoint.

Note that it is not necessary to associate all of the designated route information, priority area information, and unnecessary area information with geographic information. Only one type of information may be associated with geographic information, or two types of information may be associated with geographic information. Furthermore, multiple pieces of information may be associated with geographic information.

Next, in step S17, the coordinate conversion unit 325 converts the designated route, priority area, and unnecessary area indicated in the re-photography information associated with the geographic information from the site coordinate system to the drone coordinate system.

Next, in step S18, the movement plan creation unit 326 creates a movement plan based on the re-photography information associated with the geographic information. The movement plan is created so that the mobile object 100 moves along the designated route in the designated route information and the priority area in the priority area information, and does not move through the unnecessary areas in the unnecessary area information. The movement plan is also created as a route that connects the designated route and the priority area via the shortest distance.

As for the designated route information, the designated route, the moving speed of the moving body 100, and the attitude of the camera 105 are reflected directly in the movement plan.

Furthermore, for priority area information, movement plans can be created using multiple movement methods. These methods include round trip movement, up and down movement, and movement at reduced speed.

Reciprocating movement is a method in which the mobile object 100 travels back and forth horizontally multiple times above the priority area to capture images. This increases the number of images captured of the priority area, i.e., the images used to create geographic information, and increases the image overlap rate. Increasing the image overlap rate improves the accuracy of the geographic information, making it possible to create geographic information without abnormalities. Furthermore, the image overlap rate can be further increased by narrowing the distance between the reciprocating movement paths.

Up and down movement is a method of photographing a range of heights, such as the wall of a building, by pointing the lens of the camera 105 toward the wall and moving up and down relative to the wall. This makes it possible to photograph areas that are difficult to photograph from the air and create geographic information. Furthermore, by narrowing the distance between the round trip paths taken by the moving body 100 as it moves up and down, the image overlap rate can be increased.

The object photographed by moving up and down can be perpendicular to the ground surface, or it can be any object such as terrain or a building that is higher than the ground surface, even if it is not perpendicular.

Slower speed movement is a method of capturing images by slowing the speed of the moving body 100 moving above the priority area compared to when moving above other areas. This increases the number of times images are captured, increasing the image overlap rate and improving the accuracy of the geographic information, allowing for the creation of geographic information without any abnormalities.

The method for moving the priority area and taking photographs may be set in advance by the user, or the information processing device 300 may automatically decide using an algorithm. Furthermore, the method is not limited to one, and a movement plan may be created by combining multiple methods. For example, the movement speed may be reduced and photographs taken while moving back and forth.

The start position of the moving body 100 in the movement plan may be the current position of the moving body 100, or any position specified by the user. If the user specifies the start position of the movement, it is preferable that the user be able to input the start position of the movement along with the input of information for re-photography. If the user does not specify a position, it is preferable that the current position of the moving body 100 or a position set in advance as a default position be used as the start position of the movement.

Therefore, for example, when three pieces of re-photography information - designated route information, priority area information, and unnecessary area information - are associated with geographic information as shown in Figure 10, a travel plan is created as shown in Figure 13.

In the movement plan shown in Figure 13, the moving body 100 first moves from the movement start position to priority area A, and moves back and forth so as to focus on photographing within priority area A. Next, the moving body 100 moves from priority area A to one end of the designated route, moves along the designated route, and then moves from the other end of the designated route to priority area B. Next, the moving body 100 moves so as to focus on photographing within priority area B. Because priority area B is an area with height, such as a wall, the moving body 100 moves up and down within priority area B to photograph. Then the movement of the moving body 100 ends.

The movement plan creation unit 326 creates a movement plan based on the re-photographing information associated with the geographical information. Therefore, if the only re-photographing information associated with the geographical information is designated route information, the movement plan creation unit 326 creates a movement plan that reflects only that designated route information. Also, if the only re-photographing information associated with the geographical information is priority area information, the movement plan creation unit 326 creates a movement plan that reflects only that priority area information.

If multiple pieces of re-photographing information are associated with geographic information, the user may be able to select from that information the re-photographing information to be reflected in the travel plan. By selecting the re-photographing information to be reflected in the travel plan, the travel plan can also be adjusted or changed. In addition, multiple different travel plans can be created based on the re-photographing information associated with the geographic information.

The display processing unit 323 may display a preview of the movement plan created based on the re-shooting information on the display unit 307. By viewing this preview, the user can redo input to associate the re-shooting information, or input to add or delete re-shooting information. The preview display may be in either 2D or 3D, but displaying it in 3D allows the user to easily and intuitively grasp the movement plan, particularly the details of vertical movement.

In addition, if the user does not input the travel speed and camera attitude in the specified route information, they are automatically set to a predetermined travel speed and camera attitude. A notification prompting the user to input the travel speed and camera attitude may also be displayed on the display unit 307.

When inputting information to associate re-photographing, the user may input conditions such as the desired photographing time and weather, and the movement plan creation unit 326 may create a movement plan that reflects those conditions.

If multiple pieces of re-photographing information are associated with geographic information, the user may be able to set a priority for each piece of re-photographing information. If a priority is set for the re-photographing information, the movement plan creation unit 326 creates a movement plan according to the priority.

For example, suppose two priorities, high priority and low priority, can be set for re-photographing information. In this case, high-priority designated route information or priority area information is reflected in the travel plan regardless of the remaining battery charge of the mobile object 100. Also, low-priority designated route information or priority area information is reflected in the travel plan only if the remaining battery charge of the mobile object 100 is equal to or greater than a predetermined amount.

The movement plan creation unit 326 may create a movement plan based on the remaining battery power of the moving body 100. For example, if the remaining battery power of the moving body 100 is below a predetermined threshold, an upper limit is set on the total length of the route in the movement plan, and a movement plan is created within the upper limit of the length of the movement route, with priority given to re-photographing information associated with positions close to the starting position of the moving body 100. Also, if the remaining battery power is below a predetermined threshold, a movement plan may be created that reduces round trips in movement through priority areas.

The movement plan creation unit 326 may also create a movement plan based on the time required for image capture. For example, a time for image capture, such as 30 minutes, is set in advance, and a movement plan is created that prioritizes re-image capture information set in a position close to the start position of the movement of the moving body 100 so that the time it takes for the moving body 100 to travel the route in the movement plan is within 30 minutes. Re-image capture information that does not fit within 30 minutes is not included in the movement plan.

The movement plan may also include the start time of the movement of the moving body 100 and the time period during which the moving body 100 will move. Furthermore, the movement plan creation unit 326 may create a movement plan based on the weather. For example, on a sunny day, the movement start time and movement time period in the movement plan may be determined based on the hours of daylight and the position of the sun so as to minimize shadows.

The movement plan creation unit 326 may also create a movement plan based on wind direction. For example, if the wind is blowing from the south, it is more efficient for the moving body 100 to move from upwind to downwind, i.e., from south to north. Therefore, a movement plan is created based on wind direction, such as a movement route within the priority area that moves the moving body 100 from south to north.

In addition, the information processing device 300 can obtain weather information from the Internet, etc., in order to create a travel plan.

Returning to the explanation of the flowchart in Figure 6, the movement plan created by the movement plan creation unit 326 is transmitted from the information processing device 300 to the mobile body 100. Next, in step S19, the mobile body 100 moves above the geographic information creation range based on the movement plan and re-images using the camera 105. The UAV control unit 101 of the mobile body 100 controls the output of the actuator 102 while comparing the current position of the mobile body 100 with the movement plan based on the movement plan, thereby controlling the mobile body 100 to move in accordance with the movement plan. In addition, the gimbal control unit 104 controls the gimbal 103 based on the movement plan to adjust the attitude of the camera 105 and take images. This makes it possible to take images again based on the movement plan. The mobile body 100 then transmits the images acquired during the re-images to the geographic information creation device 200.

Then, in step S20, the geographic information creation unit 202 of the geographic information creation device 200 recreates the geographic information based on the image.

The flowchart in Figure 6 shows the re-photography based on a movement plan and the re-creation of geographic information using the images obtained from the re-photography. However, it is also possible to create a further movement plan and perform a second re-photography and re-creation of geographic information. It is possible to create a movement plan, re-photograph, and re-create geographic information any number of times.

The processing of this technology is carried out as described above. According to this technology, by associating re-photography information with geographic information, it is possible to easily create a travel plan for re-photographing for creating geographic information.

Furthermore, when entering information to associate re-photographing information with geographic information, users can check the travel routes of past photographs displayed on the geographic information, making it easier and more accurate to enter information.

If a user were to create a travel plan themselves while looking at geographic information, the geographic information and travel plan would have to be displayed on separate screens, and the user would have to compare the two while creating the travel plan. However, with this technology, the user only needs to enter information to associate the re-photographing information with the displayed geographic information, making it easy to create a travel plan.

By having the information processing device 300 equipped with an anomaly detection unit 322 and displaying the anomaly detection results on the display unit 307 in addition to the geographical information, the user can reliably recognize anomalies in the geographical information and create a movement plan to resolve the anomaly.

This technology is unique in that it displays geographic information and associates it with re-photography information, so its implementation in other companies' products can be verified by checking the GUI display of those products.

The number of moving bodies 100 moving based on the movement plan is not limited to one, and may be multiple. In other words, movement and re-photographing based on the movement plan may be performed using multiple moving bodies 100. If there are multiple moving bodies 100, the re-photographing information may be separated by type of information and a movement plan may be created for each moving body 100, or one movement plan may be created and divided for each moving body 100.

For example, if there are two moving bodies, such as moving body 100A and moving body 100B, the movement plan is divided into two parts, left and right, with the left part being the movement plan for moving body 100A and the right part being the movement plan for moving body 100B. Dividing into two parts, left and right, is merely an example, and the movement plan may also be divided into top and bottom, or in any other manner. If there are three abnormalities in moving bodies 100, the movement plan may also be divided into three or more parts.

Furthermore, when there are multiple moving bodies, such as moving body 100A and moving body 100B, a movement plan based on designated route information is created for moving body 100A, and a movement plan based on priority area information is created for moving body 100B. Furthermore, multiple moving bodies 100 may be moved based on the same movement plan.

After photographing the entire geographic information creation range, a movement plan may be created for a portion of the geographic information creation range, and that portion of the geographic information creation range may be re-photographed based on that movement plan. Alternatively, after photographing the entire geographic information creation range, a movement plan may be created for a portion of the geographic information creation range, and the entire geographic information creation range may be re-photographed again based on the movement plan for that portion of the geographic information creation range.

In a three-dimensional point cloud, the reliability of geographic information decreases around objects of a certain height, such as building walls. For example, if a blind spot occurs due to a tall object, as shown in Figure 14, the overlap rate of multiple images captured by the camera 105 of the mobile object 100 decreases, and the reliability of the three-dimensional point cloud decreases.

Furthermore, as shown in Figure 14, shadows form around objects of a certain height, such as the walls of buildings. The influence of these shadows can sometimes reduce the reliability of the 3D point cloud. In SfM, a method for creating 3D point clouds, the boundaries of shadows are likely to be treated as feature points. Feature points are used to align multiple images, so points that are treated as the same feature point in multiple images must exist at the same coordinates. However, due to the movement path of the mobile object 100, a time difference may occur when the same location is photographed at different times. If the time difference between these different times is long, the position of the shadow may have changed. If the shifted and displaced shadow boundary is treated as a feature point, the image alignment will attempt to match an incorrect position, which will have a negative impact on the creation of the 3D point cloud. For this reason, the shadow boundary areas are also considered to have low reliability in the 3D point cloud.

Therefore, the reliability of 3D point clouds can be considered low around tall objects, such as building walls, due to the reduced overlap rate caused by blind spots and the influence of changing shadows.

These low reliability areas may be calculated from the position of the sun or the attitude of the camera 105, and displayed together with the geographic information on the display unit 307. A user viewing this display can input re-photograph information to associate the low reliability areas with the re-photograph information. The movement plan creation unit 326 then creates a movement plan based on the re-photograph information, making it possible to create highly accurate geographic information even for low reliability areas.

For the same reason, the reliability of the three-dimensional point cloud may be reduced due to the influence of cloud shadows. Therefore, a 360-degree camera capable of capturing images of the surrounding area is mounted on the mobile object 100, and in addition to capturing images of the ground surface with camera 105, the 360-degree camera also captures images of clouds and the sun. The position of the cloud shadow is identified from the 360-degree image obtained from this capture. The boundary of the shadow may then be displayed on the display unit 307 together with geographical information as an area of low reliability.

When this technology is used for as-built management (managing whether a constructed structure meets the standards intended by the client), the importance of the geographic information may be displayed on the display unit 307 together with the geographic information so that the user can input information to associate with re-photography.

For example, since photographing the soil ground that will be the actual construction surface needs to be prioritized, the display unit 307 displays that it is of high importance along with the geographic information. Also, embankments, construction machinery, buildings, etc. are of medium importance, and this is displayed along with the geographic information. Furthermore, non-construction surfaces such as forests, trees, rivers, ponds, puddles, snow, and the periphery of the captured image frame are of low importance, and this is displayed along with the geographic information. The importance can be displayed, for example, by surrounding the range indicating the importance in the geographic information with a frame, as shown in Figure 15, and coloring the frame in a different color depending on the level of importance. Also, the thickness, type, and color of the frame lines may be changed depending on the level of importance.

In addition, areas where the overlap rate of the images that form the basis of the geographic information is low (due to the effects of photography failure, such as wind or weather conditions) or areas with low point cloud density are areas that should be re-photographed, and therefore the display unit 307 displays the importance of these areas together with the geographic information.

By displaying the importance in this way, users can recognize the importance and appropriately associate re-photograph information with areas that should be re-photographed.

The importance can be determined based on the results of terrain recognition within the geographic information creation range, the results of object detection within the shooting range, the overlap rate of the captured images, the density of the point cloud created as geographic information, and the shooting camera position information (information on whether it is in the center or periphery of the area). The importance can be determined based on any one of these, or a combination of several of them.

This reliability-related processing and importance-related processing may be performed by the anomaly detection unit 322, or the information processing device 300 may be provided with dedicated processing units for each.

<Modification>
Although the embodiments of the present technology have been specifically described above, the present technology is not limited to the above-described embodiments, and various modifications based on the technical concept of the present technology are possible.

Different types of geographic information, such as a 3D point cloud and an orthoimage, may be displayed simultaneously on the display unit 307. In this case, the 3D point cloud and the orthoimage may be displayed superimposed, or the display screen of the display unit 307 may be divided into two display areas, with the 3D point cloud displayed in one display area and the orthoimage displayed in the other display area.

The latest geographic information and past geographic information may be displayed simultaneously on the display unit 307. For example, the display screen of the display unit 307 may be divided into two display areas, with the latest point cloud displayed in one display area and the past point cloud displayed in the other display area.

By simultaneously displaying the latest geographic information and past geographic information, users can view and compare the latest and past geographic information, and can input key area information to correspond to areas in the latest geographic information where there are abnormal differences from the past geographic information. Then, by creating a movement plan based on that key area information and taking photographs again, the latest geographic information can be created in which the abnormal differences have been resolved.

The information processing device 300 may be equipped with a geographic information creation unit 202, or geographic information may be created on a cloud server or the like.

Geographic information may be information other than 3D point clouds and orthoimages, such as DEM (Digital Elevation Model). DEM is a digital elevation model, which is a digital representation of 3D coordinates using planar positions (2D) and elevation values measured using various surveying methods.

In addition to unmanned aerial vehicles, the mobile object 100 may also be a manned aerial vehicle, a glider, a helicopter, a balloon, an airship, a rocket, or a device that can move along rails installed in a high position such as the ceiling of a building. Furthermore, the mobile object 100 is not limited to those that move in the sky, but may also be an automobile, motorcycle, bicycle, personal mobility, airplane, ship, robot, construction machinery, agricultural machinery, etc.

The present technology can also be configured as follows.
(1)
a display processing unit that displays, on a display unit, geographic information created based on an image captured by a camera provided on the moving object;
an association unit that associates re-photographing information for re-photographing by the camera with the geographical information based on an input from a user;
an information processing device comprising: a movement plan creation unit that creates a movement plan for the moving body for re-imaging by the camera based on the re-imaging information.
(2)
The information processing device according to (1), further comprising an anomaly detection unit that detects an anomaly in the geographic information.
(3)
The information processing device according to (2), wherein the display processing unit displays the detected abnormality together with the geographic information on the display unit.
(4)
The information processing device according to any one of (1) to (3), wherein the display processing unit displays, on the display unit, a movement path of the moving object when photographing with the camera, together with the geographic information.
(5)
The information processing device according to any one of (1) to (4), wherein the display processing unit displays the position of the camera when the image was taken on the display unit together with the geographic information.
(6)
The information processing device according to any one of (1) to (5), wherein the display processing unit displays the attitude of the camera when the image was captured together with the geographic information on the display unit.
(7)
The information processing device according to any one of (1) to (6), wherein the re-photographing information is information specifying either one or both of the route and area the moving object will move in for re-photographing by the camera.
(8)
The information processing device according to (7), wherein the re-photographing information is designated route information that designates a travel route of the moving object.
(9)
The information processing device according to (8), wherein the specified route information further specifies a moving speed of the moving object.
(10)
The information processing device according to (8) or (9), wherein the specified route information further specifies the attitude of the camera.
(11)
The information processing device according to any one of (7) to (10), wherein the re-photographing information is priority area information that specifies an area to be photographed with priority by the camera.
(12)
The information processing device according to any one of (1) to (11), wherein the re-photographing information is unnecessary area information that specifies an area that does not need to be photographed by the camera.
(13)
The information processing device according to any one of (7) to (12), wherein the movement plan creation unit creates the movement plan so as to include either one or both of the route and the area specified in the re-photographing information and result in the shortest distance.
(14)
The information processing device according to any one of (1) to (13), wherein the geographic information is a three-dimensional point cloud.
(15)
The information processing device according to any one of (1) to (14), wherein the geographic information is an orthoimage.
(16)
The information processing device according to any one of (1) to (15), further comprising a geographic information creation unit that creates the geographic information.
(17)
Displaying geographic information created based on an image captured by a camera equipped on the mobile object on a display unit;
Corresponding re-photographing information for re-photographing by the camera to the geographical information based on an input from a user;
An information processing method for creating a movement plan for the moving body for re-photographing by the camera based on the re-photographing information.
(18)
a geographic information creation device that creates geographic information based on images captured by a camera equipped on a moving object;
An information processing system comprising an information processing device including: a display processing unit that displays the geographical information on a display unit; an association unit that associates re-photographing information for re-photographing by the camera with the geographical information based on input from a user; and a movement plan creation unit that creates a movement plan for the moving body for re-photographing by the camera based on the re-photographing information.

100... Mobile object 105... Camera 202... Geographic information creation unit 300... Information processing device 307... Display unit 322... Abnormality detection unit 323... Display processing unit 324... Correspondence unit 326... Movement plan creation unit 1000... Information processing system

Claims (18)

a display processing unit that displays, on a display unit, geographic information created based on an image captured by a camera provided on the moving object;
an association unit that associates re-photographing information for re-photographing by the camera with the geographical information based on an input from a user;
an information processing device comprising: a movement plan creation unit that creates a movement plan for the moving body for re-imaging by the camera based on the re-imaging information. The information processing device according to claim 1 , further comprising an anomaly detection unit that detects an anomaly in the geographic information. The information processing apparatus according to claim 2 , wherein the display processing unit displays the detected abnormality together with the geographic information on the display unit. The information processing device according to claim 1 , wherein the display processing unit displays, on the display unit, a movement path of the moving object when the image was captured by the camera, together with the geographic information. The information processing apparatus according to claim 1 , wherein the display processing unit displays, on the display unit, the position of the camera when the image was taken, together with the geographic information. The information processing apparatus according to claim 1 , wherein the display processing unit displays, on the display unit, the orientation of the camera when the image was captured, together with the geographic information. The information processing apparatus according to claim 1 , wherein the re-photographing information is information specifying one or both of a route and an area where the moving object will move in order to re-photograph the image with the camera. The information processing apparatus according to claim 7 , wherein the re-photographing information is designated route information that designates a travel route of the moving object. The information processing device according to claim 8 , wherein the specified route information further specifies a moving speed of the mobile object. The information processing apparatus according to claim 8 , wherein the specified route information further specifies the attitude of the camera. The information processing apparatus according to claim 7 , wherein the re-photographing information is priority area information that specifies an area to be photographed with priority by the camera. The information processing apparatus according to claim 1 , wherein the re-photographing information is unnecessary area information that specifies an area that does not need to be photographed by the camera. The information processing apparatus according to claim 7 , wherein the movement plan creation unit creates the movement plan so as to include one or both of a route and an area specified in the re-photographing information and to form the shortest distance. The information processing apparatus according to claim 1 , wherein the geographic information is a three-dimensional point cloud. The information processing apparatus according to claim 1 , wherein the geographic information is an orthoimage. The information processing device according to claim 1 , further comprising a geographic information creating unit that creates the geographic information. Displaying geographic information created based on an image captured by a camera equipped on the mobile object on a display unit;
Corresponding re-photographing information for re-photographing by the camera to the geographical information based on an input from a user;
An information processing method for creating a movement plan for the moving body for re-photographing by the camera based on the re-photographing information. a geographic information creation device that creates geographic information based on images captured by a camera equipped on a moving object;
An information processing system comprising an information processing device including: a display processing unit that displays the geographical information on a display unit; an association unit that associates re-photographing information for re-photographing by the camera with the geographical information based on input from a user; and a movement plan creation unit that creates a movement plan for the moving body for re-photographing by the camera based on the re-photographing information.