WO2021224955A1 - Own-position estimation device, flight control device, image generation device, artificial satellite system, and own-position estimation method - Google Patents

Abstract

An own-position estimation device (41) is provided in a mobile item (3) that moves around an object (O) and that captures images of the object (O) by using a camera (23). The own-position estimation device (41) is provided with: a first image acquisition unit (51) that acquires a first template image (I1') based on a captured image (I1) captured by the camera (23) and that acquires a first reference image (I2') based on a simulation image (I2) corresponding to the captured image (I1); a first image matching unit (52) that calculates a first position deviation amount and a first scale change amount by executing first template matching, in which the first template image (I1') is enlarged or reduced and the first reference image (I2') is scanned by using the first template image (I1'); and an own-position estimation unit (53) that calculates an estimated own-position value by using the first position deviation amount and the first scale change amount.

Description

Self-position estimation device, navigation control device, image generator, artificial satellite system and self-position estimation method

The present disclosure relates to a self-position estimation device, a navigation control device, an image generator, an artificial satellite system, and a self-position estimation method.

Conventionally, a system for capturing an image of a subject using a camera provided on a moving body that moves around the subject has been developed. Specifically, for example, a system for imaging a celestial body using a camera provided on an artificial satellite orbiting the celestial body has been developed. Further, a technique for estimating the position of such a subject has been developed by using an image captured by such a camera.

For example, Patent Document 1 discloses the following techniques. That is, a plurality of images are sequentially captured by the moving camera. Next, a group of feature points in each image is extracted. Next, feature matching using these feature point clouds is performed. Based on the feature amount matching result, the three-dimensional position of the subject and the parameters of the camera are estimated.

JP-A-2019-49457

In a moving body, it may be required to estimate the position of the moving body (hereinafter sometimes referred to as "self-position"). Specifically, for example, it may be required to estimate the self-position of the artificial satellite in the artificial satellite. The technique described in Patent Document 1 is used not only for estimating the position of a subject but also for estimating a self-position (see, for example, paragraph [0077] of Patent Document 1). However, when the self-position is estimated by using the technique described in Patent Document 1, there are the following problems.

That is, the technique described in Patent Document 1 extracts a group of feature points in each image and executes feature amount matching as described above. Due to these processes, there is a problem that the amount of calculation is large. Therefore, there is a problem that it is difficult to realize these processes when the computational resources in the mobile body are limited.

This disclosure is made to solve the above-mentioned problems, and aims to reduce the amount of calculation related to the estimation of the self-position.

The self-position estimation device according to the present disclosure is a moving body that moves around the subject, and is a self-position estimating device provided in the moving body that captures the subject using a camera, and is a first template based on an image captured by the camera. A first image acquisition unit that acquires an image and acquires a first reference image based on a simulation image corresponding to the captured image, and a first reference image is scanned using the first template image while scaling the first template image. Self-position estimated value using the first image matching unit that calculates the first position shift amount and the first scale change amount by executing the first template matching, and the first position shift amount and the first scale change amount. It is provided with a self-position estimation unit for calculating.

According to the present disclosure, since it is configured as described above, the amount of calculation related to the estimation of the self-position can be reduced.

It is a block diagram which shows the main part of the artificial satellite system which concerns on Embodiment 1. FIG. It is a block diagram which shows the main part of the image generation apparatus in the artificial satellite system which concerns on Embodiment 1. FIG. It is a block diagram which shows the main part of the navigation control device in the artificial satellite system which concerns on Embodiment 1. FIG. It is explanatory drawing which shows the example of 1st template matching. It is a block diagram which shows the hardware configuration of the main part of the image generation apparatus in the artificial satellite system which concerns on Embodiment 1. FIG. It is a block diagram which shows the other hardware configuration of the main part of the image generation apparatus in the artificial satellite system which concerns on Embodiment 1. FIG. It is a block diagram which shows the other hardware configuration of the main part of the image generation apparatus in the artificial satellite system which concerns on Embodiment 1. FIG. It is a block diagram which shows the hardware composition of the main part of the navigation control device in the artificial satellite system which concerns on Embodiment 1. FIG. It is a block diagram which shows the other hardware configuration of the main part of the navigation control device in the artificial satellite system which concerns on Embodiment 1. FIG. It is a block diagram which shows the other hardware configuration of the main part of the navigation control device in the artificial satellite system which concerns on Embodiment 1. FIG. It is a flowchart which shows the operation of the image generator in the artificial satellite system which concerns on Embodiment 1. FIG. It is a flowchart which shows the operation of the self-position estimation apparatus in the artificial satellite system which concerns on Embodiment 1. FIG. It is a flowchart which shows the operation of the navigation control device in the artificial satellite system which concerns on Embodiment 1. FIG. It is a block diagram which shows the main part of the artificial satellite system which concerns on Embodiment 2. It is a block diagram which shows the main part of the image generation apparatus in the artificial satellite system which concerns on Embodiment 2. It is a block diagram which shows the main part of the navigation control device in the artificial satellite system which concerns on Embodiment 1. FIG. It is explanatory drawing which shows the example of the 2nd template matching. It is a flowchart which shows the operation of the image generator in the artificial satellite system which concerns on Embodiment 2. It is a flowchart which shows the operation of the self-position estimation apparatus in the artificial satellite system which concerns on Embodiment 2. It is a flowchart which shows the operation of the navigation control device in the artificial satellite system which concerns on Embodiment 2.

Hereinafter, in order to explain this disclosure in more detail, a mode for carrying out this disclosure will be described with reference to the attached drawings.

Embodiment 1.
FIG. 1 is a block diagram showing a main part of the artificial satellite system according to the first embodiment. FIG. 2 is a block diagram showing a main part of an image generation device in the artificial satellite system according to the first embodiment. FIG. 3 is a block diagram showing a main part of the navigation control device in the artificial satellite system according to the first embodiment. The artificial satellite system according to the first embodiment will be described with reference to FIGS. 1 to 3.

As shown in FIG. 1, the artificial satellite system 1 includes a ground station 2 and an artificial satellite 3. The ground station 2 includes an image generation device 11, a storage device 12, and a communication device 13. The artificial satellite 3 includes a communication device 21, an attitude estimation device 22, a camera 23, a storage device 24, and a navigation control device 25.

The communication device 13 and the communication device 21 can communicate with each other. As a result, the ground station 2 and the artificial satellite 3 can communicate with each other.

Ground station 2 is provided on celestial body O (not shown). On the other hand, the artificial satellite 3 orbits around the celestial body O. Here, the optical axis of the camera 23 is directed to the celestial body O. Further, in the artificial satellite system 1, a plurality of times (hereinafter, may be referred to as “imaging time”) for imaging the celestial body O are set. The artificial satellite 3 uses the camera 23 to image the celestial body O at each imaging time. Hereinafter, the image I1 captured by the camera 23 at each imaging time may be referred to as an “captured image”.

Hereinafter, a state in which there is no error in the orbit of the artificial satellite 3, no error in the position of the artificial satellite 3, and no error in the attitude of the artificial satellite 3 is referred to as an "ideal state". Sometimes. The orbit of the artificial satellite 3 in the ideal state (hereinafter, may be referred to as “target orbit”) follows a predetermined orbit. Further, the position of the artificial satellite 3 (hereinafter, may be referred to as “target position”) at each imaging time in the ideal state is a predetermined position in the target orbit. Further, the posture of the artificial satellite 3 (hereinafter, may be referred to as “target posture”) at each imaging time in the ideal state is a predetermined posture. Further, in the ideal state, the optical axis of the camera 23 is directed to the central portion of the celestial body O.

As shown in FIG. 2, the image generation device 11 includes a simulation image generation unit 31 and a reference image generation unit 32.

The simulation image generation unit 31 simulates the image I1 to be captured by the camera 23 at each imaging time in an ideal state. As a result, the simulation image I2 corresponding to each image I1 is generated.

That is, the storage device 12 stores information indicating a target trajectory and information indicating individual imaging times. The simulation image generation unit 31 acquires this information. The simulation image generation unit 31 uses the acquired information to predict the position (x, y, z) of the artificial satellite 3 at each imaging time in the ideal state, and the camera at each imaging time in the ideal state. Predict the orientation (vx, by, vz) of the optical axis of 23.

Further, the storage device 12 stores data indicating a three-dimensional model of the celestial body O. The simulation image generation unit 31 acquires such data. The simulation image generation unit 31 arranges the three-dimensional model of the celestial body O in the virtual space for simulation by using the acquired data.

Further, the storage device 12 stores information indicating a parameter related to the celestial body O (hereinafter referred to as "unique parameter") and information indicating a parameter related to the camera 23 (hereinafter referred to as "camera parameter"). The simulation image generation unit 31 acquires this information. The simulation image generation unit 31 sets parameters for simulation (hereinafter referred to as “simulation parameters”) using the acquired information.

The unique parameters include a value indicating the reflectance (for example, albedo) in the celestial body O and a value indicating the solar altitude in the celestial body O. The camera parameters include a value indicating the focus of the camera 23, a value indicating the optical axis of the camera 23, a value indicating the position of the camera 23, and the like. In addition to this, the camera parameters may include a value indicating the lens distortion coefficient in the camera 23, a value indicating noise in the camera 23, and the like.

The simulation image generation unit 31 sets the simulation image generation unit 31 at each imaging time in an ideal state based on the predicted position (x, y, z), the predicted orientation (vx, vy, vz), and the set simulation parameters. Image I1 is simulated. At this time, a model similar to the projection model in the actual camera 23 is used as the projection model of the lens.

As a result, a simulation image I2 corresponding to each image I1 is generated. The simulation image generation unit 31 stores the data indicating the generated simulation image I2 (hereinafter referred to as “simulation image data”) in the storage device 12.

Here, when the distance between the celestial body O and the artificial satellite 3 is small, the celestial body O may protrude from the angle of view of the camera 23, so that only a part of the celestial body O may be imaged by the camera 23. As a result, a state in which only a part of the celestial body O is included in the image I1 may occur. In this case, the simulation image generation unit 31 may generate the following simulation image I2.

That is, the simulation image generation unit 31 generates a simulation image I2 corresponding to a state in which the entire celestial body O is virtually imaged by the camera 23 by virtually expanding the angle of view of the camera 23. You may. As a result, a simulation image I2 including the entire celestial body O is generated. As a result, even when the distance between the celestial body O and the artificial satellite 3 is small, it is possible to realize template matching (hereinafter referred to as "first template matching") executed by the first image matching unit 52 described later. can.

At this time, the simulation image generation unit 31 generates the simulation image I2 by virtually increasing the number of pixels of the camera 23 without changing the focal length of the camera 23. As a result, in generating the simulation image I2, the value indicating the focal length of the camera 23 included in the information indicating the unique parameter can be used as it is.

The reference image generation unit 32 acquires the simulation image data stored in the storage device 12. The reference image generation unit 32 uses the acquired simulation image data to generate a reference image (hereinafter referred to as “first reference image”) I2 ′ for first template matching.

That is, the reference image generation unit 32 executes the binarization process for each simulation image I2 using the acquired simulation image data. At this time, a predetermined value is used for the parameter for binarization (including the threshold value for binarization; hereinafter referred to as "first binarization parameter"). Alternatively, a value calculated by a statistical method is used as the first binarization parameter. Specifically, for example, a value calculated by a discriminant analysis method is used as the first binarization parameter.

When the binarized simulation image I2 contains noise, the reference image generation unit 32 executes noise removal processing on the binarized simulation image I2. As a result, it is possible to prevent such noise from affecting the correlation value calculated by the first template matching (hereinafter referred to as "first correlation value"). For the noise removal process, for example, a scaling process is used.

Next, the reference image generation unit 32 resizes the binarized simulation image I2 or the noise-removed simulation image I2 to a predetermined size. As a result, the individual simulation images I2 are resized to the size for the first template matching.

In this way, the first reference image I2'corresponding to each simulation image I2 is generated. The reference image generation unit 32 stores the data indicating the generated first reference image I2'(hereinafter referred to as "reference image data") in the storage device 12. The reference image data may include the first binarization parameter.

The communication device 13 acquires the reference image data stored in the storage device 12. The communication device 13 transmits the acquired reference image data to the artificial satellite 3. The communication device 21 receives the transmitted reference image data. The communication device 21 stores the received reference image data in the storage device 24.

The posture estimation device 22 is composed of, for example, a start racker. The attitude estimation device 22 estimates the attitude (φ, θ, ψ) of the artificial satellite 3. Here, the posture (φ, θ, ψ) estimated by the posture estimation device 22 includes the posture (φ, θ, ψ) at each imaging time. The posture estimation device 22 stores a value (hereinafter referred to as “posture estimation value”) indicating the estimated posture (φ, θ, ψ) in the storage device 24.

As described above, the camera 23 captures the celestial body O at each imaging time. The camera 23 stores data indicating the image I1 captured at each imaging time (hereinafter referred to as “captured image data”) in the storage device 24.

As shown in FIG. 3, the navigation control device 25 includes a self-position estimation device 41, a navigation value calculation unit 42, and a navigation control unit 43. The self-position estimation device 41 includes a first image acquisition unit 51, a first image matching unit 52, and a self-position estimation unit 53.

The first image acquisition unit 51 acquires the reference image data stored in the storage device 24. As a result, the first reference image I2'is acquired. In addition, the first image acquisition unit 51 acquires the captured image data stored in the storage device 24. The first image acquisition unit 51 uses the acquired captured image data to generate a template image (hereinafter referred to as “first template image”) I1 ′ for first template matching. As a result, the first template image I1'is acquired.

Specifically, for example, the first image acquisition unit 51 acquires reference image data indicating the first reference image I2'corresponding to the image I1 captured at the latest imaging time. As a result, the first reference image I2'corresponding to the image I1 captured at the latest imaging time is acquired. In addition, the first image acquisition unit 51 acquires captured image data indicating the image I1 captured at the latest imaging time. The first image acquisition unit 51 uses the acquired captured image data to generate a first template image I1'corresponding to the image I1 captured at the latest imaging time. As a result, the first template image I1'corresponding to the image I1 captured at the latest imaging time is acquired.

Here, a method of generating the first template image I1'will be described. The first image acquisition unit 51 executes a binarization process similar to the binarization process executed by the reference image generation unit 32 on each captured image I1. At this time, the first image acquisition unit 51 may use the first binarization parameter included in the acquired reference image data. Next, the first image acquisition unit 51 appropriately executes a noise removal process similar to the noise removal process executed by the reference image generation unit 32 on the binarized captured image I1. In this way, the first template image I1'corresponding to each captured image I1 is generated. That is, the first template image I1'corresponding to each captured image I1 is acquired.

The first image matching unit 52 uses the first template image I1'acquired by the first image acquisition unit 51 and the first reference image I2' acquired by the first image acquisition unit 51 to perform the first template matching. It is what you do. FIG. 4 shows an example of first template matching.

That is, the first image matching unit 52 changes the scale s of the acquired first template image I1'. More specifically, the first image matching unit 52 enlarges or reduces the scale s of the acquired first template image I1'within a predetermined range. As a result, a plurality of first template images I1'having different scales s are generated.

Further, the first image matching unit 52 executes the zero padding process on the acquired first reference image I2'. As a result, the first reference image I2'having a margin region is generated. By adding a margin area to the first reference image I2', even if the celestial body O is located at the end (that is, the upper end, the lower end, the left end or the right end) of the first template image I1', the celestial body O is located. The first template matching can be realized.

Next, the first image matching unit 52 scans the first reference image I2'having the margin area by using each of the plurality of first template images I1'. At this time, the first image matching unit 52 is the first for each position (cx, cy) of the first template image I1'in the first reference image I2'and for each scale s of the first template image I1'. The correlation value (that is, the first correlation value) between the template image I1'and the first reference image I2' is calculated.

In this way, a plurality of first correlation values are calculated. For each first correlation value, for example, SAD (Sum of Absolute Difference) or ZNCC (Zero-mean Normalized Cross-Correlatio) is used.

Hereinafter, the positions (cx_b, cy_b) corresponding to the maximum value among the plurality of calculated first correlation values may be referred to as "first matching positions". Further, the scale s_b corresponding to the maximum value among the plurality of calculated first correlation values may be referred to as a "first matching scale".

The first matching position (cx_b, cy_b) corresponds to the amount of deviation of the position of the celestial body O in the first template image I1'with respect to the position of the celestial body O in the first reference image I2'. Hereinafter, such a deviation amount may be referred to as a “first position deviation amount”. Further, the first matching scale s_b corresponds to the amount of change in the scale of the celestial body O in the first template image I1'with respect to the scale of the celestial body O in the first reference image I2'. Hereinafter, such a change amount may be referred to as a “first scale change amount”.

That is, the first image matching unit 52 calculates the first matching position (cx_b, cy_b) corresponding to the first position deviation amount by executing the template matching, and the first scale corresponding to the first scale change amount. The matching scale s_b is calculated. The first image matching unit 52 outputs the calculated first matching position (cx_b, cy_b) to the self-position estimation unit 53, and outputs the calculated first matching scale s_b to the self-position estimation unit 53. ..

The self-position estimation unit 53 acquires the first matching position (cx_b, cy_b) output by the first image matching unit 52. Further, the self-position estimation unit 53 acquires the first matching scale s_b output by the first image matching unit 52.

Further, the self-position estimation unit 53 acquires the posture estimation values (φ, θ, ψ) stored in the storage device 24. Specifically, for example, the self-position estimation unit 53 acquires the posture estimation values (φ, θ, ψ) at the latest imaging time.

Further, the storage device 24 stores data indicating a digital elevation model of the celestial body O. The self-position estimation unit 53 acquires such data.

The self-position estimation unit 53 acquires the first position deviation amount corresponding to the acquired first matching position (cx_b, cy_b), the first scale change amount corresponding to the acquired first matching scale s_b, and the above-mentioned acquired one. The self-position of the artificial satellite 3 is estimated by using the attitude estimation values (φ, θ, ψ) and the numerical elevation values included in the acquired data. Thereby, for example, the self-position at the latest imaging time is estimated.

Various known techniques can be used to estimate such self-position. Detailed description of these techniques will be omitted. The self-position estimation unit 53 outputs a value indicating the estimated self-position (hereinafter referred to as “self-position estimation value”) to the navigation value calculation unit 42.

The navigation value calculation unit 42 acquires the self-position estimation value output by the self-position estimation unit 53. The navigation value calculation unit 42 calculates the amount of deviation of the self-position indicated by the acquired self-position estimated value with respect to the target position at the corresponding imaging time (for example, the latest imaging time). At this time, the navigation value calculation unit 42 may calculate the amount of deviation in two directions (including the horizontal direction and the vertical direction).

The navigation value calculation unit 42 calculates a control value (hereinafter referred to as “navigation value”) used for controlling a thruster or the like on the artificial satellite 3 according to the calculated deviation amount. The navigation value calculation unit 42 outputs the calculated navigation value to the navigation control unit 43.

The navigation control unit 43 acquires the navigation value output by the navigation value calculation unit 42. The navigation control unit 43 controls the thruster or the like of the artificial satellite 3 by using the acquired navigation value. As a result, when there is an error in the position (x, y, z) of the artificial satellite 3, the position (x, y, z) of the artificial satellite 3 can be brought closer to the target position.

Hereinafter, the processes executed by the simulation image generation unit 31 may be collectively referred to as "simulation image generation processing". Further, the processes executed by the reference image generation unit 32 may be collectively referred to as "reference image generation process". Further, the processes executed by the first image acquisition unit 51 may be collectively referred to as "first image acquisition process". Further, the processes executed by the first image matching unit 52 may be collectively referred to as "first image matching process". Further, the processes executed by the self-position estimation unit 53 may be collectively referred to as "self-position estimation process". Further, the processing executed by the navigation value calculation unit 42 may be collectively referred to as "navigation value calculation processing". Further, the processing and control executed by the navigation control unit 43 may be collectively referred to as "navigation control".

Hereinafter, the processes executed by the self-position estimation device 41 may be collectively referred to as "self-position estimation process, etc." That is, the self-position estimation process and the like include a first image acquisition process, a first image matching process, and a self-position estimation process.

Hereinafter, the functions of the simulation image generation unit 31 are collectively referred to as "simulation image generation function". Further, the functions of the reference image generation unit 32 may be collectively referred to as a "reference image generation function". In addition, the functions of the first image acquisition unit 51 may be collectively referred to as the "first image acquisition function". In addition, the functions of the first image matching unit 52 may be collectively referred to as the "first image matching function". Further, the functions of the self-position estimation unit 53 may be collectively referred to as "self-position estimation function". Further, the functions of the navigation value calculation unit 42 may be collectively referred to as a "navigation value calculation function". Further, the functions of the navigation control unit 43 may be collectively referred to as "navigation control function".

Hereinafter, the code of "F1" may be used for the simulation image generation function. In addition, the reference image generation function may use the reference numeral "F2". Further, the reference numeral "F11" may be used for the first image acquisition function. Further, the reference numeral "F12" may be used for the first image matching function. In addition, the code "F13" may be used for the self-position estimation function. In addition, the code "F21" may be used for the navigation value calculation function. In addition, the code "F22" may be used for the navigation control function.

Next, the hardware configuration of the main part of the image generation device 11 will be described with reference to FIGS. 5 to 7.

As shown in FIG. 5, the image generator 11 has a processor 61 and a memory 62. The memory 62 stores programs corresponding to a plurality of functions (including a simulation image generation function and a reference image generation function) F1 and F2. The processor 61 reads and executes the program stored in the memory 62. As a result, a plurality of functions F1 and F2 are realized.

Alternatively, as shown in FIG. 6, the image generator 11 has a processing circuit 63. The processing circuit 63 executes processing corresponding to the plurality of functions F1 and F2. As a result, a plurality of functions F1 and F2 are realized.

Alternatively, as shown in FIG. 7, the image generator 11 has a processor 61, a memory 62, and a processing circuit 63. A program corresponding to a part of the plurality of functions F1 and F2 is stored in the memory 62. The processor 61 reads and executes the program stored in the memory 62. As a result, some of these functions are realized. Further, the processing circuit 63 executes processing corresponding to the remaining functions of the plurality of functions F1 and F2. As a result, such a residual function is realized.

The processor 61 is composed of one or more processors. As each processor, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microprocessor, or a DSP (Digital Signal Processor) is used.

The memory 62 is composed of one or more non-volatile memories. Alternatively, the memory 62 is composed of one or more non-volatile memories and one or more volatile memories. That is, the memory 62 is composed of one or more memories. The individual memory uses, for example, a semiconductor memory, a magnetic disk, an optical disk, a magneto-optical disk, a magnetic tape, or a magnetic drum. More specifically, each volatile memory uses, for example, a RAM (Random Access Memory). In addition, individual non-volatile memories include, for example, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmory), EEPROM (Electrically Erasable Programmory), flexible disk drive A compact disc, a DVD (Digital Versaille Disc), a Blu-ray disc, or a mini disc is used.

The processing circuit 63 is composed of one or more digital circuits. Alternatively, the processing circuit 63 is composed of one or more digital circuits and one or more analog circuits. That is, the processing circuit 63 is composed of one or more processing circuits. The individual processing circuits are, for example, ASIC (Application Special Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), FPGA (Field Program Is.

Here, when the processor 61 is composed of a plurality of processors, the correspondence between the plurality of functions F1 and F2 and the plurality of processors is arbitrary. That is, each of the plurality of processors may read and execute a program corresponding to one or more corresponding functions among the plurality of functions F1 and F2. The processor 61 may include a dedicated processor corresponding to each function F1 and F2.

Further, when the memory 62 is composed of a plurality of memories, the correspondence between the plurality of functions F1 and F2 and the plurality of memories is arbitrary. That is, each of the plurality of memories may store a program corresponding to one or more corresponding functions among the plurality of functions F1 and F2. The memory 62 may include a dedicated memory corresponding to the individual functions F1 and F2.

Further, when the processing circuit 63 is composed of a plurality of processing circuits, the correspondence between the plurality of functions F1 and F2 and the plurality of processing circuits is arbitrary. That is, each of the plurality of processing circuits may execute processing corresponding to one or more corresponding functions among the plurality of functions F1 and F2. The processing circuit 63 may include a dedicated processing circuit corresponding to each function F1 and F2.

That is, in the image generation device 11, the individual functions F1 and F2 may be realized by a dedicated processing circuit.

Next, the hardware configuration of the main part of the navigation control device 25 will be described with reference to FIGS. 8 to 10.

As shown in FIG. 8, the navigation control device 25 has a processor 71 and a memory 72. The memory 72 is a program corresponding to a plurality of functions (including a first image acquisition function, a first image matching function, a self-position estimation function, a navigation value calculation function, and a navigation control function) F11 to F13, F21, and F22. Is remembered. The processor 71 reads and executes the program stored in the memory 72. As a result, a plurality of functions F11 to F13, F21, and F22 are realized.

Alternatively, as shown in FIG. 9, the navigation control device 25 has a processing circuit 73. The processing circuit 73 executes processing corresponding to a plurality of functions F11 to F13, F21, and F22. As a result, a plurality of functions F11 to F13, F21, and F22 are realized.

Alternatively, as shown in FIG. 10, the navigation control device 25 includes a processor 71, a memory 72, and a processing circuit 73. The memory 72 stores programs corresponding to some of the plurality of functions F11 to F13, F21, and F22. The processor 71 reads and executes the program stored in the memory 72. As a result, some of these functions are realized. Further, the processing circuit 73 executes processing corresponding to the remaining functions of the plurality of functions F11 to F13, F21, and F22. As a result, such a residual function is realized.

The specific example of the processor 71 is the same as the specific example of the processor 61. The specific example of the memory 72 is the same as the specific example of the memory 62. The specific example of the processing circuit 73 is the same as the specific example of the processing circuit 63. Therefore, detailed description thereof will be omitted.

Here, when the processor 71 is composed of a plurality of processors, the correspondence between the plurality of functions F11 to F13, F21, F22 and the plurality of processors is arbitrary. That is, each of the plurality of processors may read and execute a program corresponding to one or more corresponding functions among the plurality of functions F11 to F13, F21, and F22. The processor 71 may include a dedicated processor corresponding to each function F11 to F13, F21, F22.

Further, when the memory 72 is composed of a plurality of memories, the correspondence between the plurality of functions F11 to F13, F21, F22 and the plurality of memories is arbitrary. That is, each of the plurality of memories may store a program corresponding to one or more corresponding functions among the plurality of functions F11 to F13, F21, and F22. The memory 72 may include dedicated memories corresponding to the individual functions F11 to F13, F21, and F22.

Further, when the processing circuit 73 is composed of a plurality of processing circuits, the correspondence between the plurality of functions F11 to F13, F21, F22 and the plurality of processing circuits is arbitrary. That is, each of the plurality of processing circuits may execute processing corresponding to one or more corresponding functions among the plurality of functions F11 to F13, F21, and F22. The processing circuit 73 may include a dedicated processing circuit corresponding to each function F11 to F13, F21, F22.

That is, in the self-position estimation device 41, the individual functions F11 to F13 may be realized by a dedicated processing circuit. Further, in the navigation control device 25, the individual functions F11 to F13, F21, and F22 may be realized by a dedicated processing circuit.

Next, the operation of the image generator 11 will be described with reference to the flowchart shown in FIG.

First, the simulation image generation unit 31 executes the simulation image generation process (step ST1). Next, the reference image generation unit 32 executes the reference image generation process (step ST2).

Next, the operation of the self-position estimation device 41 will be described with reference to the flowchart shown in FIG.

First, the first image acquisition unit 51 executes the first image acquisition process (step ST11). When the captured image I1 indicated by the captured image data acquired in step ST11 contains at least a part of the celestial body O (step ST12 “YES”), then the first image matching unit 52 performs the first image matching process. Is executed (step ST13). Next, the self-position estimation unit 53 executes the self-position estimation process (step ST14).

If the captured image I1 indicated by the captured image data acquired in step ST11 does not include the celestial body O (step ST12 “NO”), it is impossible or difficult to accurately execute the first template matching. be. Therefore, the processes of steps ST13 and ST14 are skipped. As a result, the processing of the self-position estimation device 41 is completed.

The determination in step ST12 is, for example, as follows. That is, the number of white pixels in the first template image I1'generated in step ST11 is calculated. When the calculated number is equal to or greater than a predetermined threshold value, it is determined that the corresponding captured image I1 contains at least a part of the celestial body O (step ST12 “YES”). If not, it is determined that the corresponding captured image I1 does not contain the celestial body O (step ST12 “NO”).

Next, the operation of the navigation control device 25 will be described with reference to the flowchart shown in FIG.

First, the self-position estimation device 41 executes self-position estimation processing and the like (step ST21). As a result, the process described with reference to FIG. 12 is executed. Next, the navigation value calculation unit 42 executes the navigation value calculation process (step ST22). Next, the navigation control unit 43 executes navigation control (step ST23).

Next, the effect of using the self-position estimation device 41 will be described.

As described above, the self-position estimation device 41 estimates the self-position by executing the first template matching using the first template image I1'based on the captured image I1 and the first reference image I2' based on the simulation image I2. Is what you do. In other words, in the self-position estimation device 41, in estimating the self-position, it is not necessary to perform a process of extracting a feature point cloud in each captured image I1. Further, there is no need for a process of performing feature amount matching between the captured images I1.

Therefore, by using the self-position estimation device 41, it is possible to reduce the amount of calculation related to self-position estimation as compared with the case of using the technique described in Patent Document 1. As a result, even when the computational resources of the artificial satellite 3 are limited, it is possible to estimate the self-position of the artificial satellite 3.

Next, a modified example according to the first embodiment will be described.

The reference image generation unit 32 may be provided in the self-position estimation device 41 instead of being provided in the image generation device 11. That is, the reference image generation process may be executed by the self-position estimation device 41 instead of being executed by the image generation device 11.

The navigation control device 25 may be provided on a moving body different from the artificial satellite 3. For example, the navigation control device 25 may be provided in a vehicle, a ship, or an aircraft. Therefore, the subject by the camera 23 is not limited to the celestial body O. The subject by the camera 23 may be any object. Further, the image generation device 11 may be provided at a place different from that of the ground station 2. The image generation device 11 may be provided at any place as long as it can communicate with the moving body having the navigation control device 25.

As described above, the self-position estimation device 41 according to the first embodiment is a moving body (artificial satellite 3) that moves around the subject (celestial body O), and uses the camera 23 to move the subject (celestial body O). In the self-position estimation device 41 provided in the moving body (artificial satellite 3) to be imaged, the first template image I1'based on the image captured by the camera 23 is acquired, and the first template image I1'based on the simulated image I2 corresponding to the captured image I1 is acquired. 1 The first image acquisition unit 51 that acquires the reference image I2'and the first template image I1' are scaled up and down, and the first template matching that scans the first reference image I2'using the first template image I1'is executed. By doing so, the first image matching unit 52 that calculates the first displacement amount and the first scale change amount, and the self-position estimation that calculates the self-position estimation value using the first position deviation amount and the first scale change amount. A unit 53 is provided. As a result, the amount of calculation related to the estimation of the self-position can be reduced.

Further, the artificial satellite system 1 according to the first embodiment is an artificial satellite 3 that orbits around the celestial body O, and is a self-position estimation device 41 provided in the artificial satellite 3 that images the celestial body O by using the camera 23. In the artificial satellite system 1 including the image generation device 11 provided in the ground station 2 capable of communicating with the artificial satellite 3, the self-position estimation device 41 displays the first template image I1'based on the image I1 captured by the camera 23. The first image acquisition unit 51 that acquires the first reference image I2'based on the simulation image I2 corresponding to the captured image I1 and the first template image I1'are scaled and the first template image I1'is used. The first image matching unit 52 that calculates the first position shift amount and the first scale change amount by executing the first template matching that scans the first reference image I2', and the first position shift amount and the first A self-position estimation unit 53 that calculates a self-position estimation value using the amount of scale change is provided, and the image generation device 11 generates a simulation image generation unit 31 that generates a simulation image I2 and a first reference image I2'. A reference image generation unit 32 is provided. As a result, the amount of calculation related to the estimation of the self-position can be reduced.

Further, the self-position estimation method according to the first embodiment is a moving body (artificial satellite 3) that moves around the subject (celestial body O), and the moving body that images the subject (celestial body O) using the camera 23. In the self-position estimation method in (artificial satellite 3), the first image acquisition unit 51 acquires the first template image I1'based on the image I1 captured by the camera 23, and is based on the simulation image I2 corresponding to the captured image I1. Step ST11 to acquire the first reference image I2'and the first image matching unit 52 scale the first template image I1'and scan the first reference image I2' using the first template image I1'. Step ST13 for calculating the first position shift amount and the first scale change amount by executing one template matching, and the self-position estimation unit 53 self-position using the first position shift amount and the first scale change amount. A step ST14 for calculating an estimated value is provided. As a result, the amount of calculation related to the estimation of the self-position can be reduced.

Embodiment 2.
FIG. 14 is a block diagram showing a main part of the artificial satellite system according to the second embodiment. FIG. 15 is a block diagram showing a main part of an image generation device in the artificial satellite system according to the second embodiment. FIG. 16 is a block diagram showing a main part of the navigation control device in the artificial satellite system according to the first embodiment. The artificial satellite system according to the second embodiment will be described with reference to FIGS. 14 to 16.

Note that, in FIG. 14, the same blocks as those shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted. Further, in FIG. 15, the same blocks as those shown in FIG. 2 are designated by the same reference numerals, and the description thereof will be omitted. Further, in FIG. 16, the same blocks as those shown in FIG. 3 are designated by the same reference numerals, and the description thereof will be omitted.

As shown in FIG. 14, the artificial satellite system 1a includes a ground station 2a and an artificial satellite 3a. The ground station 2a includes an image generation device 11a, a storage device 12, and a communication device 13. The artificial satellite 3a includes a communication device 21, an attitude estimation device 22, a camera 23, a storage device 24, and a navigation control device 25a.

As shown in FIG. 15, the image generation device 11a includes a simulation image generation unit 31 and a reference image generation unit 32. In addition to this, the image generation device 11a includes a template image generation unit 33.

The template image generation unit 33 acquires the simulation image data stored in the storage device 12. The template image generation unit 33 uses the acquired simulation image data to perform a template image (hereinafter, referred to as “second template matching”) for template matching (hereinafter referred to as “second template matching”) executed by the second image acquisition unit 54, which will be described later. It is called a "second template image".) I2 "is generated.

That is, the template image generation unit 33 executes edge detection processing for each simulation image I2 using the acquired simulation image data. Any detection method may be used for the edge detection process.

Next, the template image generation unit 33 executes binarization processing for the simulation image I2 after edge detection. The threshold value for binarization may use a predetermined value, or may use a value calculated by a statistical method.

In this way, the second template image I2 "corresponding to each simulation image I2" is generated. The template image generation unit 33 generates data indicating the generated second template image I2 "(hereinafter," template image data ". ”) Is stored in the storage device 12.

Note that the template image data may include information indicating a detection method in the edge detection process (hereinafter referred to as "detection method information"). Further, the template image data may include parameters for edge detection (hereinafter referred to as "edge detection parameters"). Further, the template image data may include a parameter for binarization (including a threshold value for binarization; hereinafter referred to as "second binarization parameter").

Further, when the template image generation unit 33 generates the second template image I2 ", the template image generation unit 33 arranges the central portion of the celestial body O in the region including the central portion of the second template image I2" (hereinafter referred to as "central region"). It may be something to do.

The communication device 13 acquires the reference image data and the template image data stored in the storage device 12. The communication device 13 transmits the acquired reference image data and template image data to the artificial satellite 3a. The communication device 21 receives the transmitted reference image data and template image data. The communication device 21 stores the received reference image data and the template image data in the storage device 24.

As shown in FIG. 16, the navigation control device 25a includes a self-position estimation device 41a, a navigation value calculation unit 42, and a navigation control unit 43. The self-position estimation device 41 includes a first image acquisition unit 51, a first image matching unit 52, and a self-position estimation unit 53a. In addition to this, the self-position estimation device 41 includes a second image acquisition unit 54 and a second image matching unit 55.

The second image acquisition unit 54 acquires the template image data stored in the storage device 24. As a result, the second template image I2 ”is acquired. Further, the second image acquisition unit 54 acquires the captured image data stored in the storage device 24. The second image acquisition unit 54 obtains the captured image data. Using the acquired captured image data, a reference image for second template matching (hereinafter referred to as "second reference image") I1 "is generated. As a result, the second reference image I1" is acquired. Will be done.

Specifically, for example, the second image acquisition unit 54 acquires template image data indicating the second template image I2 ”corresponding to the image I1 captured at the latest imaging time. The second template image I2 ”corresponding to the image I1 captured at the time is acquired. In addition, the second image acquisition unit 54 acquires captured image data indicating the image I1 captured at the latest imaging time. The second image acquisition unit 54 uses the acquired captured image data to generate a second reference image I1 ”corresponding to the image I1 captured at the latest imaging time, thereby generating the latest imaging time. The second reference image I1 ”corresponding to the image I1 captured in is acquired.

Here, a method of generating the second reference image I1 ”will be described. First, the second image acquisition unit 54 is similar to the edge detection process executed by the template image generation unit 33 for each captured image I1. The edge detection process is executed. At this time, the second image acquisition unit 54 may use the detection method information and the edge detection parameters included in the acquired template image data. Next, the second image acquisition. The unit 54 executes a binarization process similar to the binarization process executed by the template image generation unit 33 on the captured image I1 after edge detection. At this time, the second image acquisition unit 54 executes the binarization process. The second binarization parameter included in the acquired template image data may be used. In this way, the second reference image I1 ”corresponding to each captured image I1 is generated. That is, the second reference image I1 ”corresponding to each captured image I1 is acquired.

The second image matching unit 55 uses the second template image I2 "acquired by the second image acquisition unit 54 and the second reference image I1" acquired by the second image acquisition unit 54 to perform the second template matching. It is what you do. FIG. 17 shows an example of second template matching.

Here, the algorithm of the second template matching is the same as the algorithm of the first template matching.

That is, the second image matching unit 55 changes the scale s of the acquired second template image I2 ". More specifically, the second image matching unit 55 changes the scale s of the acquired second template image I2". The scale s of "is enlarged or reduced within a predetermined range. As a result, a plurality of second template images I2 "having different scales s are generated.

Next, the second image matching unit 55 scans the acquired second reference image I1 "using each of the plurality of second template images I2". At this time, the second image matching unit 55 is used for each position (cx, cy) of the second template image I2 "in the second reference image I1" and for each scale s of the second template image I2 ". The correlation value between the template image I2 "and the second reference image I1" (hereinafter referred to as "second correlation value") is calculated.

However, the second image matching unit 55 scans only the area including the position (cx, cy) corresponding to the first matching position (cx_b, cy_b) in the acquired second reference image I1 ”. In other words, the second image matching unit 55 may exclude the remaining region of the acquired second reference image I1 ”from the scanning target.

Further, the second image matching unit 55 selects one or more second template images I2 "having scales s corresponding to the first matching scale s_b among the plurality of second template images I2". There may be. The second image matching unit 55 may use each of the selected one or more second template images I2 ”for scanning. In other words, the second image matching unit 55 may use the plurality of images. The remaining second template image I2 "of the second template image I2" may not be used for scanning.

In this way, a plurality of second correlation values are calculated. For each second correlation value, for example, SAD or ZNCC is used.

Hereinafter, the position (cx_e, cy_e) corresponding to the maximum value among the plurality of calculated second correlation values may be referred to as a "second matching position". Further, the scale s_e corresponding to the maximum value among the plurality of calculated second correlation values may be referred to as a "second matching scale".

The second matching position (cx_e, cy_e) corresponds to the amount of deviation of the position of the celestial body O in the second template image I2 "with respect to the position of the celestial body O in the second reference image I1". Hereinafter, such a deviation amount may be referred to as a “second position deviation amount”. Further, the second matching scale s_e corresponds to the amount of change in the scale of the celestial body O in the second template image I2 "with respect to the scale of the celestial body O in the second reference image I1". Hereinafter, such a change amount may be referred to as a “second scale change amount”.

That is, the second image matching unit 55 calculates the second matching position (cx_e, cy_e) corresponding to the second position deviation amount by executing the second template matching, and corresponds to the second scale change amount. The second matching scale s_e is calculated. The second image matching unit 55 outputs the calculated second matching position (cx_e, cy_e) to the self-position estimation unit 53a, and outputs the calculated second matching scale s_e to the self-position estimation unit 53a. ..

The self-position estimation unit 53a estimates the self-position of the artificial satellite 3a by an estimation method similar to the estimation method by the self-position estimation unit 53. However, when estimating the self-position, the self-position estimation unit 53a may use the second position shift amount and the second scale change amount instead of using the first position shift amount and the first scale change amount. .. In other words, the self-position estimation unit 53a selectively uses the first position shift amount and the first scale change amount or the second position shift amount and the second scale change amount for self-position estimation.

Hereinafter, the processing executed by the template image generation unit 33 may be collectively referred to as "template image generation processing". Further, the processes executed by the second image acquisition unit 54 may be collectively referred to as "second image acquisition process". Further, the processes executed by the second image matching unit 55 may be collectively referred to as "second image matching process". Further, the processes executed by the self-position estimation unit 53a may be collectively referred to as "self-position estimation process".

Hereinafter, the processes executed by the self-position estimation device 41a may be collectively referred to as "self-position estimation process, etc." That is, the self-position estimation process and the like include a first image acquisition process, a first image matching process, a second image acquisition processing process, a second image matching process, and a self-position estimation process.

Hereinafter, the functions of the template image generation unit 33 may be collectively referred to as "template image generation function". In addition, the functions of the second image acquisition unit 54 may be collectively referred to as the "second image acquisition function". In addition, the functions of the second image matching unit 55 may be collectively referred to as the "second image matching function". Further, the functions of the self-position estimation unit 53a may be collectively referred to as "self-position estimation function".

Hereinafter, the code of "F3" may be used for the template image generation function. In addition, the reference numeral "F14" may be used for the second image acquisition function. In addition, the reference numeral "F15" may be used for the second image matching function. Further, the reference numeral "F13a" may be used for the self-position estimation function in the self-position estimation unit 53a.

The hardware configuration of the main part of the image generation device 11a is the same as that described with reference to FIGS. 5 to 7 in the first embodiment. Therefore, detailed description thereof will be omitted.

That is, the image generation device 11a has a plurality of functions (including a simulation image generation function, a reference image generation function, and a template image generation function) F1 to F3. The individual functions F1 to F3 may be realized by the processor 61 and the memory 62, or may be realized by the processing circuit 63.

Here, the processor 61 may include a dedicated processor corresponding to each function F1 to F3. Further, the memory 62 may include a dedicated memory corresponding to the individual functions F1 to F3. Further, the processing circuit 63 may include a dedicated processing circuit corresponding to the individual functions F1 to F3.

That is, in the image generation device 11a, the individual functions F1 to F3 may be realized by a dedicated processing circuit.

The hardware configuration of the main part of the navigation control device 25a is the same as that described with reference to FIGS. 8 to 10 in the first embodiment. Therefore, detailed description thereof will be omitted.

That is, the navigation control device 25a has a plurality of functions (first image acquisition function, first image matching function, second image acquisition function, second image matching function, self-position estimation function, navigation value calculation function, and navigation control function. Includes) F11, F12, F13a, F14, F15, F21, F22. The individual functions F11, F12, F13a, F14, F15, F21, and F22 may be realized by the processor 71 and the memory 72, or may be realized by the processing circuit 73.

Here, the processor 71 may include a dedicated processor corresponding to each function F11, F12, F13a, F14, F15, F21, F22. Further, the memory 72 may include a dedicated memory corresponding to each function F11, F12, F13a, F14, F15, F21, F22. Further, the processing circuit 73 may include a dedicated processing circuit corresponding to each function F11, F12, F13a, F14, F15, F21, F22.

That is, in the self-position estimation device 41a, the individual functions F11, F12, F13a, F14, and F15 may be realized by a dedicated processing circuit. Further, in the navigation control device 25a, the individual functions F11, F12, F13a, F14, F15, F21, and F22 may be realized by a dedicated processing circuit.

Next, the operation of the image generation device 11a will be described with reference to the flowchart shown in FIG. In FIG. 18, the same reference numerals are given to the same steps as those shown in FIG.

First, the simulation image generation unit 31 executes the simulation image generation process (step ST1). Next, the reference image generation unit 32 executes the reference image generation process (step ST2). Next, the template image generation unit 33 executes the template image generation process (step ST3).

Next, the operation of the self-position estimation device 41a will be described with reference to the flowchart shown in FIG. In FIG. 19, the same reference numerals are given to the same steps as those shown in FIG.

First, the first image acquisition unit 51 executes the first image acquisition process (step ST11). When the captured image I1 indicated by the captured image data acquired in step ST11 contains at least a part of the celestial body O (step ST12 “YES”), then the first image matching unit 52 performs the first image matching process. Is executed (step ST13).

If the captured image I1 indicated by the captured image data acquired in step ST11 does not include the celestial body O (step ST12 “NO”), it is impossible or difficult to accurately execute the first template matching. be. It is also impossible or difficult to accurately perform the second template matching. Therefore, the processing after step ST13 is skipped. As a result, the processing of the self-position estimation device 41a is completed.

When the captured image I1 indicated by the captured image data acquired in step ST11 includes the central portion of the celestial body O (step ST15 “YES”), the second image acquisition unit 54 then performs the second image acquisition process. Execute (step ST16). Next, the second image matching unit 55 executes the second image matching process (step ST17).

If the captured image I1 indicated by the captured image data acquired in step ST11 does not include the central portion of the celestial body O, it is impossible or difficult to accurately execute the second template matching. Therefore, the processes of steps ST16 and ST17 are skipped. As a result, the process of the self-position estimation device 41a proceeds to step ST14a.

Next, the self-position estimation unit 53a executes the self-position estimation process (step ST14a). Here, when the processes of steps ST16 and ST17 are skipped, the first position shift amount and the first scale change amount are used for estimating the self-position. On the other hand, when the processes of steps ST16 and ST17 are executed, the second position shift amount and the second scale change amount are used for estimating the self-position.

Next, the operation of the navigation control device 25a will be described with reference to the flowchart shown in FIG. In FIG. 20, the same reference numerals are given to the same steps as those shown in FIG.

First, the self-position estimation device 41a executes self-position estimation processing and the like (step ST21a). As a result, the process described with reference to FIG. 19 is executed. Next, the navigation value calculation unit 42 executes the navigation value calculation process (step ST22). Next, the navigation control unit 43 executes navigation control (step ST23).

Next, the effect of using the self-position estimation device 41a will be described.

When the distance between the celestial body O and the artificial satellite 3 is large, the following states may occur. That is, a state may occur in which the entire celestial body O is included in the captured image I1 and only a part of the region in the captured image I1 corresponds to the celestial body O. Hereinafter, such a state may be referred to as a "first state". On the other hand, when the distance between the celestial body O and the artificial satellite 3 is small, the following states may occur. That is, a state may occur in which the entire image I1 or substantially the entire image I1 is a region corresponding to the celestial body O. Hereinafter, such a state may be referred to as a "second state". In the second state, a part of the celestial body O may protrude from the captured image I1.

In the second state, the image generated by executing the binarization process (that is, the first template image I1') is entirely or substantially entirely composed of white pixels. As a result, the accuracy of matching in the first template matching may decrease. Then, since the first position deviation amount and the first scale change amount are used for the estimation of the self-position, the estimation accuracy of the self-position may decrease.

On the other hand, in the second state, it is highly probable that the captured image I1 contains detailed features of the celestial body O. Therefore, by using the images generated by executing the edge detection process (that is, the second template image I2 "and the second reference image I1"), highly accurate second template matching can be realized. In other words, the accuracy of the second template matching can be improved. Then, by using the second misalignment amount and the second scale change amount, the self-position estimation accuracy is improved as compared with the case where the first misalignment amount and the first scale amount are used for self-position estimation. Can be done.

Also, as described above, the second template matching algorithm is the same as the first template matching algorithm. Therefore, in the second state, it is possible to realize the estimation of the self-position by the same algorithm as the algorithm in the first state.

Further, by arranging the central portion of the celestial body O in the central region of the second template image I2 ", the following effects can be obtained. That is, as described above, one of the celestial bodies O in the second state. The portion may protrude from the captured image I1. Even in such a case, if the central portion of the celestial body O is included in the captured image I1, matching by such a central portion can be realized. The amount of the second misalignment based on the position of the central portion can be calculated.

Next, a modified example according to the second embodiment will be described.

The second image matching unit 55 may execute parabolic fitting using each second correlation value and the corresponding scale s after executing the second template matching. As a result, the second image matching unit 55 may calculate a more detailed second matching scale s_e.

The reference image generation unit 32 may be provided in the self-position estimation device 41a instead of being provided in the image generation device 11a. That is, the reference image generation process may be executed by the self-position estimation device 41a instead of being executed by the image generation device 11a.

The template image generation unit 33 may be provided in the self-position estimation device 41a instead of being provided in the image generation device 11a. That is, the template image generation process may be executed by the self-position estimation device 41a instead of being executed by the image generation device 11a.

The navigation control device 25a may be provided on a moving body different from the artificial satellite 3a. For example, the navigation control device 25a may be provided in a vehicle, a ship, or an aircraft. Further, the image generation device 11a may be provided at a place different from that of the ground station 2a. The image generation device 11a may be provided at any place as long as it can communicate with the moving body having the navigation control device 25a.

As described above, the self-position estimation device 41a according to the second embodiment acquires the second template image I2 "based on the simulation image I2 and the second reference image I1" based on the captured image I1. By executing the second template matching that scans the second reference image I1 "using the image acquisition unit 54 and the second template image I2" while scaling the second template image I2 ", the amount of the second misalignment and the amount of the second position shift are obtained. A second image matching unit 55 for calculating the second scale change amount is provided, and the self-position estimation unit 53a uses the first position shift amount and the first scale change amount, or the second position shift amount and the second scale. The self-position estimation value is calculated using the amount of change. Thereby, the self-position estimation by the same algorithm can be realized in the first state or the second state.

Further, in the artificial satellite system 1a according to the second embodiment, the self-position estimation device 41a acquires the second template image I2 "based on the simulation image I2 and the second reference image I1" based on the captured image I1. The second position is obtained by performing a second template matching in which the second image acquisition unit 54 and the second template image I2 "are scaled and the second reference image I1" is scanned using the second template image I2 ". The image generation device 11a includes a second image matching unit 55 for calculating the deviation amount and the second scale change amount, and the image generation device 11a includes a template image generation unit 33 for generating the second template image I2 ”, and a self-position estimation unit 53a. Calculates its own position estimate using the first misalignment amount and the first scale change amount, or using the second misalignment amount and the second scale change amount. Thereby, the self-position estimation by the same algorithm can be realized in the first state or the second state.

Further, in the self-position estimation method according to the second embodiment, the second image acquisition unit 54 acquires the second template image I2 "based on the simulation image I2 and obtains the second reference image I1" based on the captured image I1. The acquisition step ST16 and the second image matching unit 55 perform the second template matching that scales the second template image I2 "and scans the second reference image I1" using the second template image I2 ". The self-position estimation unit 53a includes the step ST17 for calculating the second position shift amount and the second scale change amount, and the self-position estimation unit 53a uses the first position shift amount and the first scale change amount or the second position shift amount. And the second scale change amount is used to calculate the self-position estimation value. Thereby, the self-position estimation by the same algorithm can be realized in the first state or the second state.

It should be noted that, within the scope of the disclosure of the present application, it is possible to freely combine each embodiment, modify any component of each embodiment, or omit any component in each embodiment. ..

The self-position estimation device, navigation control device, image generation device, artificial satellite system, and self-position estimation method according to the present disclosure can be used, for example, for an orbiting satellite.

1,1a artificial satellite system, 2,2a ground station, 3,3a artificial satellite, 11,11a image generator, 12 storage device, 13 communication device, 21 communication device, 22 attitude estimation device, 23 camera, 24 storage device, 25, 25a Navigation control device, 31 Simulation image generation unit, 32 Reference image generation unit, 33 Template image generation unit, 41, 41a Self-position estimation device, 42 Navigation value calculation unit, 43 Navigation control unit, 51 First image acquisition unit , 52 1st image matching unit, 53, 53a self-position estimation unit, 54 2nd image acquisition unit, 55 2nd image matching unit, 61 processor, 62 memory, 63 processing circuit, 71 processor, 72 memory, 73 processing circuit.

Claims (17)

In a self-position estimation device provided in a moving body that moves around a subject and images the subject using a camera.
A first image acquisition unit that acquires a first template image based on an image captured by the camera and also acquires a first reference image based on a simulation image corresponding to the captured image.
The first template image and the first scale change amount are calculated by performing the first template matching that scans the first reference image using the first template image while scaling the first template image. 1 image matching part and
A self-position estimation unit that calculates a self-position estimation value using the first position shift amount and the first scale change amount, and a self-position estimation unit.
A self-position estimation device comprising. A second image acquisition unit that acquires a second template image based on the simulation image and also acquires a second reference image based on the captured image.
The second template image is scaled and the second template image is used to scan the second reference image to perform second template matching to calculate the second misalignment amount and the second scale change amount. Equipped with 2 image matching parts
The self-position estimation unit calculates the self-position estimation value using the first position shift amount and the first scale change amount, or using the second position shift amount and the second scale change amount. The self-position estimation device according to claim 1, wherein the self-position estimation device is characterized. The first template image is generated by executing a binarization process on the captured image.
The self-position estimation device according to claim 1 or 2, wherein the first reference image is generated by executing a binarization process on the simulation image. The second template image is generated by executing an edge detection process and a binarization process on the simulation image.
The self-position estimation device according to claim 2, wherein the second reference image is generated by executing an edge detection process and a binarization process on the captured image. Claim 1 or claim, wherein the simulation image corresponds to a state in which the entire subject is virtually imaged by the camera by virtually expanding the angle of view of the camera. Item 2. The self-position estimation device according to item 2. The self-position estimation device according to claim 1 or 2, wherein the moving body is an artificial satellite and the subject is a celestial body. The self-position estimation device according to claim 1 or 2,
A navigation value calculation unit that calculates a navigation value according to the self-position estimated value, and a navigation value calculation unit.
A navigation control unit that executes navigation control of the moving body using the navigation value, and
A navigation control device equipped with. In the image generator for the self-position estimation device according to claim 1,
A simulation image generator that generates the simulation image,
A reference image generation unit that generates the first reference image, and
An image generator characterized by comprising. In the image generation device for the self-position estimation device according to claim 2.
A simulation image generator that generates the simulation image,
A reference image generation unit that generates the first reference image, and
A template image generation unit that generates the second template image, and
An image generator characterized by comprising. The simulation image generation unit is characterized in that the simulation image corresponding to a state in which the entire subject is virtually imaged by the camera is generated by virtually expanding the angle of view of the camera. The image generator according to claim 8 or 9. The image generation device according to claim 8 or 9, wherein the reference image generation unit generates the first reference image by executing a binarization process on the simulation image. The image generation device according to claim 9, wherein the template image generation unit generates the second template image by executing an edge detection process and a binarization process on the simulation image. The moving body is an artificial satellite, and the subject is a celestial body.
The image generator according to claim 8 or 9, wherein the image generation device is provided in a ground station capable of communicating with the artificial satellite. An artificial satellite that orbits around a celestial body, a self-position estimation device provided on the artificial satellite that images the celestial body using a camera, and an image generation device provided on a ground station capable of communicating with the artificial satellite. In artificial satellite systems including
The self-position estimation device
A first image acquisition unit that acquires a first template image based on an image captured by the camera and also acquires a first reference image based on a simulation image corresponding to the captured image.
The first template image and the first scale change amount are calculated by performing the first template matching that scans the first reference image using the first template image while scaling the first template image. 1 image matching part and
It is provided with a self-position estimation unit that calculates a self-position estimation value using the first position shift amount and the first scale change amount.
The image generator
A simulation image generator that generates the simulation image,
An artificial satellite system including a reference image generation unit that generates the first reference image. The self-position estimation device
A second image acquisition unit that acquires a second template image based on the simulation image and also acquires a second reference image based on the captured image.
The second template image is scaled and the second template image is used to scan the second reference image to perform second template matching to calculate the second misalignment amount and the second scale change amount. Equipped with 2 image matching parts
The image generation device includes a template image generation unit that generates the second template image.
The self-position estimation unit calculates the self-position estimation value using the first position shift amount and the first scale change amount, or using the second position shift amount and the second scale change amount. The artificial satellite system according to claim 14. In a method of estimating a self-position in a moving body that moves around a subject and captures the subject using a camera.
A step in which the first image acquisition unit acquires a first template image based on an image captured by the camera and also acquires a first reference image based on a simulation image corresponding to the captured image.
The first image matching unit scales the first template image and executes the first template matching that scans the first reference image using the first template image, whereby the first misalignment amount and the first Steps to calculate the amount of scale change and
A step in which the self-position estimation unit calculates a self-position estimation value using the first position deviation amount and the first scale change amount.
A self-position estimation method comprising. A step in which the second image acquisition unit acquires a second template image based on the simulation image and also acquires a second reference image based on the captured image.
The second image matching unit scales the second template image and executes the second template matching that scans the second reference image using the second template image, whereby the second misalignment amount and the second are performed. With a step to calculate the amount of scale change,
The self-position estimation unit calculates the self-position estimation value using the first position shift amount and the first scale change amount, or using the second position shift amount and the second scale change amount. The self-position estimation method according to claim 16.

Images