JP6041641B2 - Dimension estimation apparatus, dimension estimation program, and dimension estimation method - Google Patents

Description

  The present invention relates to a size estimation device, a size estimation program, and a size estimation method for measuring the size of a subject shown in a photograph, for example.

Stereo measurement is known as a method for measuring the size of a subject shown in a photograph. Stereo measurement is a method of photographing a subject from different directions and measuring the size of the subject by the principle of triangulation using the parallax at the time of photographing.
However, when the number or angle of shooting of the monitoring target is limited as in a monitoring operation for monitoring a suspicious ship at sea, the monitoring target cannot be shot from different directions while securing an appropriate parallax. Therefore, it is difficult to measure the exact size of the monitoring target by stereo measurement.

In addition, when a subject including a comparison portion with a known size is shown in a photograph, the size of the measurement portion of the subject can be measured by comparing the size of the measurement portion of the subject with the comparison portion. it can.
However, when the subject is photographed from an oblique direction, the size of the measurement portion of the subject cannot be accurately measured because the size of the subject is different from the size of the near side portion and the far side portion of the subject.

FIG. 36 is a schematic diagram of a conventional dimension measuring method.
For example, it is assumed that the diameter “2 meters” of the circular portion of the subject is known ((a) in FIG. 36). Here, it is assumed that the subject is imaged from the front ((b) of FIG. 36).
In this case, the ratio “2 (= 40/20)” between the number of pixels “20” in the circular portion of the subject and the number of pixels “40” in the height direction of the subject, and the diameter “2 meters” of the circular portion of the subject. Can be used to calculate the height of the subject “4 meters (= 2 × 2 meters)” ((c) and (d) in FIG. 36).
Similarly, the ratio “6 (= 120/20)” between the number of pixels “20” in the circular portion of the subject and the number of pixels “120” in the width direction of the subject, and the diameter “2 meters” of the circular portion of the subject. By using this, the width of the subject “12 meters (= 6 × 2 meters)” can be calculated ((c) and (d) in FIG. 36).
However, when the subject is imaged from an oblique direction ((e) of FIG. 36), the size of the subject is different because the front portion d2 and the back portion d1 of the subject have different sizes. The width cannot be measured accurately.

Non-Patent Document 1 discloses a method of calculating a three-dimensional posture vector representing the direction in which a subject is shown from a single picture showing a subject taken from an oblique direction.
However, there is no disclosure or suggestion of a method for measuring the size of a subject based on a three-dimensional posture vector representing the direction in which the subject is reflected.

  Patent Document 1 discloses a method for recognizing a three-dimensional position and posture based on vision.

JP-A-7-98214

T. T. Shakunaga, H .; Kaneko, 'Perceptive angle transform: Principle for shape from angles', International Journal of Computer Vision, vol. 3, no. 3, pp. 239-254, 1989.

  An object of the present invention is to make it possible to accurately measure the size of an object using an object image obtained by imaging the object from an oblique direction, for example.

The size estimation apparatus of the present invention is
A reference representing three reference line segments along the width direction, the height direction, and the depth direction of the target object shown in the target image obtained by imaging the target object from an oblique direction, using two-dimensional coordinate values of the target image. A reference line segment information generation unit for generating line segment information;
Based on the reference line segment information generated by the reference line segment information generation unit and the imaging parameters of the target image, the width direction, the height direction, and the depth direction of the target object are three-dimensionally displayed. A reference axis vector calculation unit for calculating three reference axis vectors represented using vector values;
Using the reference axis vector of at least one of the three reference axis vectors calculated by the reference axis vector calculation unit, the length of the target object shown in the target image is a known line segment A reference line segment vector calculating unit for calculating a reference line segment vector value as a reference line segment vector;
The length of the object shown in the target image is an unknown line segment using at least one of the three reference axis vectors calculated by the reference axis vector calculation unit. A measurement line vector calculation unit that calculates a vector value of the measurement line segment as a measurement line segment vector;
Using the known length of the reference line segment, the reference line segment vector calculated by the reference line segment vector calculation unit, and the measurement line segment vector calculated by the measurement line segment vector calculation unit, A measurement line segment length calculation unit that calculates the length of the measurement line segment.

The reference line segment vector calculator is
A unit vector V _s having a direction from a coordinate point P _c indicating a position where the target image is captured to one end point P _s of the reference line segment is calculated;
It calculates a unit vector V _e from the coordinate point P _c with the direction of the other end point P _e of the reference line,
The reference line segment vector is calculated using the unit vector V _s , the unit vector V _e, and at least one of the three reference axis vectors.

The reference line segment vector calculator is
The parametric t _s for from the coordinate point P _c to obtain a vector V _s t _s to the end point P _s of the reference line segment, calculated by using one of the reference axis vector of the three reference axis vector And
The parametric t _e for obtaining the vector V _e t _e from the coordinate point P _c to the end point P _e of the reference line, is calculated by using the one of the reference axis vector,
Wherein the vector _V s _{t s} by using the vector _V e _{t e,} calculates the reference line segment vector.

The reference line segment vector calculation unit calculates the reference line segment vector L _r by calculating the following equation (101).

The reference line segment vector calculation unit calculates the following expression (102) using the coordinate value of the origin P ₀ of the three reference axis vectors and using the reference axis vector Oz as one of the reference axis vectors, The parameter t _s and the parameter t _te are calculated.

The measurement line segment vector calculation unit
A unit vector V _ms having a direction from the coordinate point P _c indicating the position where the target image is captured to one end point P _ms of the measurement line segment,
A unit vector V _me having a direction from the coordinate point P _c to the other end point P _me of the reference line segment is calculated;
The measurement line segment vector is calculated using the unit vector V _ms , the unit vector V _me, and at least one of the three reference axis vectors.

The measurement line segment vector calculation unit
The parametric t _ms for from the coordinate point P _c to obtain a vector V _ms t _ms to the end point P _ms of the measurement line segment, calculated by using one of the reference axis vector of the three reference axis vector And
A parametric variable t _me for obtaining a vector V _me t _me from the coordinate point P _c to the end point P _{me of the} measurement line segment is calculated using any of the reference axis vectors;
The reference line segment vector is calculated using the vector V _ms t _ms and the vector V _me t _me .

The measurement line segment vector calculation unit calculates the measuring line segment vector L _m by calculating the following equation (103).

The measurement line segment vector calculation unit calculates the following formula (104) using the coordinate value of the origin P ₀ of the three reference axis vectors and using the reference axis vector Ox as one of the reference axis vectors, The parameter t _ms and the parameter t _me are calculated.

  The measurement line segment length calculation unit calculates, as the length of the measurement line segment, a value obtained by multiplying a ratio of the size of the measurement line segment vector to the reference line segment vector by a known length of the reference line segment. .

The reference axis vector calculation unit includes:
Three unit vectors representing the width direction, the height direction, and the depth direction of the object based on the reference line segment information and the imaging parameter, and an FPM (First Perspective Moving) coordinate system or Calculate three unit vectors in the SPM (Second Perspective Moving) coordinate system,
Using the three unit vectors in the FPM coordinate system or the SPM coordinate system, the three vectors in the world coordinate system are calculated as the three reference axis vectors.

The size estimation program of the present invention is:
A reference representing three reference line segments along the width direction, the height direction, and the depth direction of the target object shown in the target image obtained by imaging the target object from an oblique direction, using two-dimensional coordinate values of the target image. A reference line segment information generation process for generating line segment information;
Based on the reference line segment information generated by the reference line segment information generation process and the imaging parameters of the target image, the width direction, the height direction, and the depth direction of the target object are three-dimensional. A reference axis vector calculation process for calculating three reference axis vectors represented using vector values;
Using the reference axis vector of at least one of the three reference axis vectors calculated by the reference axis vector calculation process, the length of the object shown in the target image is a known line segment A reference line vector calculation process for calculating a reference line vector value as a reference line vector;
It is a line segment whose length is unknown among the objects shown in the target image using at least one of the three reference axis vectors calculated by the reference axis vector calculation process. A measurement line vector calculation process for calculating a vector value of the measurement line segment as a measurement line segment vector;
Using the known length of the reference line segment, the reference line segment vector calculated by the reference line segment vector calculation process, and the measurement line segment vector calculated by the measurement line segment vector calculation process, A computer executes a measurement line segment length calculation process for calculating the length of the measurement line segment.

The reference line segment vector calculation process includes:
A process of calculating a unit vector V _s having a direction from a coordinate point P _c indicating a position where the target image is captured to one end point P _s of the reference line segment;
A process of calculating the unit vector V _e from the coordinate point P _c with the direction of the other end point P _e of the reference line,
Processing for calculating the reference line segment vector using the unit vector V _s , the unit vector V _e, and at least one of the three reference axis vectors.

The reference line segment vector calculation process includes:
The parametric t _s for from the coordinate point P _c to obtain a vector V _s t _s to the end point P _s of the reference line segment, calculated by using one of the reference axis vector of the three reference axis vector Processing to
The parametric t _e for obtaining the vector V _e t _e from the coordinate point P _c to the end point P _e of the reference line, and processing for calculating using said one of the reference axis vector,
Wherein by using the vector V _s t _s and the vector V _e t _e, and a process for calculating the reference line segment vector.

The reference line segment vector calculation process includes a process of calculating the reference line vector L _r by calculating the following equation (101).

The reference line segment vector calculation process uses the coordinate value of the origin P ₀ of the three reference axis vectors and calculates the following formula (102) using the reference axis vector Oz as any of the reference axis vectors, A process for calculating the parameter t _s and the parameter t _te .

The measurement line segment vector calculation process includes:
A process of calculating a unit vector V _ms having a direction from a coordinate point P _c indicating a position where the target image is captured to one end point P _ms of the measurement line segment;
A process of calculating a unit vector V _me having a direction from the coordinate point P _c to the other end point P _me of the reference line segment;
Processing for calculating the measurement line segment vector using the unit vector V _ms , the unit vector V _me, and at least one of the three reference axis vectors.

The measurement line segment vector calculation process includes:
The parametric t _ms for from the coordinate point P _c to obtain a vector V _ms t _ms to the end point P _ms of the measurement line segment, calculated by using one of the reference axis vector of the three reference axis vector Processing to
The parametric t _me to obtain a vector V _me t _me from the coordinate point P _c to the end point P _me of the measurement line segment, the process of calculating using said one of the reference axis vector,
Processing for calculating the reference line segment vector using the vector V _ms t _ms and the vector V _me t _me .

The measurement line segment vector calculation process includes a process of calculating the measurement line segment vector L _m by calculating the following equation (103).

The measurement line segment vector calculation process uses the coordinate value of the origin P ₀ of the three reference axis vectors and calculates the following formula (104) using the reference axis vector Ox as any one of the reference axis vectors, A process of calculating the parameter t _ms and the parameter t _me .

  In the measurement line segment length calculation process, a value obtained by multiplying a ratio of the magnitude of the measurement line segment vector to the reference line segment vector by a known length of the reference line segment is calculated as the length of the measurement line segment. Includes processing.

The reference axis vector calculation process includes:
Three unit vectors representing the width direction, the height direction, and the depth direction of the object based on the reference line segment information and the imaging parameter, and an FPM (First Perspective Moving) coordinate system or A process of calculating three unit vectors in an SPM (Second Perspective Moving) coordinate system;
Processing using the three unit vectors in the FPM coordinate system or the SPM coordinate system to calculate the three vectors in the world coordinate system as the three reference axis vectors.

The size estimation method according to the present invention includes:
A reference line segment information generation unit displays three reference line segments along the width direction, the height direction, and the depth direction of the target object that are reflected in the target image obtained by imaging the target object from an oblique direction. Generate the reference line segment information expressed using the coordinate value of
Based on the reference line segment information generated by the reference line segment information generation unit and the imaging parameters of the target image, a reference axis vector calculation unit is configured to calculate the width direction, the height direction, and the height of the target object. Calculate three reference axis vectors that represent the depth direction using three-dimensional vector values,
The reference line vector calculation unit uses a reference axis vector of at least one of the three reference axis vectors calculated by the reference axis vector calculation unit, and uses the longest of the objects shown in the target image. A vector value of a reference line segment whose length is a known line segment is calculated as a reference line segment vector,
The measurement line segment vector calculation unit uses a reference axis vector of at least one of the three reference axis vectors calculated by the reference axis vector calculation unit, and uses the longest of the objects shown in the target image. The vector value of the measurement line segment whose length is unknown is calculated as the measurement line segment vector,
The measurement line segment length calculation unit includes the known length of the reference line segment, the reference line segment vector calculated by the reference line segment vector calculation unit, and the measurement line calculated by the measurement line segment vector calculation unit. The length of the measurement line segment is calculated using the segment vector.

  According to the present invention, for example, the size of an object can be accurately measured using an object image obtained by imaging the object from an oblique direction.

2 is a functional configuration diagram of an image size estimation apparatus 100 according to Embodiment 1. FIG. 3 is a flowchart illustrating an image size estimation method according to the first embodiment. 3 is a diagram illustrating an example of a photographic image 101 according to Embodiment 1. FIG. 6 is a diagram illustrating an example of a posture reference axis [lx, ly, lz] according to Embodiment 1. FIG. 6 is a diagram illustrating an example of a posture reference axis [lx, ly, lz] according to Embodiment 1. FIG. 6 is a relationship diagram of imaging parameters 139 according to Embodiment 1. FIG. 6 is a flowchart illustrating an example of a shooting parameter setting process (S130) according to the first embodiment. 3 is a configuration diagram of a reference setting unit 150 according to Embodiment 1. FIG. 6 is a flowchart showing reference setting processing (S150) in the first embodiment. FIG. 6 is a schematic diagram of reference axis vector projection processing (S151) in the first embodiment. It is a schematic diagram of the reference axis vector moving process (S153) in the first embodiment. 6 is a schematic diagram of reference line segment setting processing (S154) according to Embodiment 1. FIG. 6 is a schematic diagram of reference line segment setting processing (S155) in the first embodiment. FIG. FIG. 6 is a schematic diagram of reference line vector calculation processing (S156) in the first embodiment. 3 is a configuration diagram of a dimension estimation unit 160 according to Embodiment 1. FIG. 4 is a flowchart showing a dimension estimation process (S160) in the first embodiment. FIG. 3 is a schematic diagram from S161 to S165 in the first embodiment. 6 is a schematic diagram of S166 in the first embodiment. FIG. 6 is a flowchart showing a reference axis vector estimation process (S140) in the first embodiment. 6 is a flowchart showing a reference axis vector calculation process [FPM coordinate system] (S142) in the first embodiment. 6 is a schematic diagram of endpoint unit vector calculation processing (S142A) in Embodiment 1. FIG. 6 is a schematic diagram of intersection unit vector calculation processing (S142B) in Embodiment 1. FIG. It is a schematic diagram of FPM coordinate plane calculation processing (S142C) in the first embodiment. 6 is a schematic diagram of FPM coordinate plane vector calculation processing (S142D) in Embodiment 1. FIG. FIG. 5 is a schematic diagram of FPM coordinate system unit vector calculation processing (S142E) in the first embodiment. FIG. 3 is a relationship diagram of reference axis vectors in a subject coordinate system in the first embodiment. 6 is a flowchart showing a reference axis vector calculation process [SPM coordinate system] (S143) in the first embodiment. 6 is a schematic diagram of intersection unit vector calculation processing (S143A) in Embodiment 1. FIG. 6 is a schematic diagram of SPM coordinate plane calculation processing (S143B) in Embodiment 1. FIG. 6 is a schematic diagram of posture reference axis projection processing (S143C) in Embodiment 1. FIG. 6 is a schematic diagram of unit vector calculation processing (S143D) in the SPM coordinate system according to Embodiment 1. FIG. 3 is a relationship diagram of posture reference axes in an SPM coordinate plane in Embodiment 1. FIG. FIG. 7 is a schematic diagram of reference origin calculation processing (S143F) in the first embodiment. 3 is a diagram illustrating an example of hardware resources of the image size estimation apparatus 100 according to Embodiment 1. FIG. 3 is a diagram illustrating a three-dimensional model of a subject 102 in Embodiment 1. FIG. It is a schematic diagram of the conventional dimension measuring method.

Embodiment 1 FIG.
A mode in which the size of the subject is accurately measured using a single image obtained by imaging the subject from an oblique direction will be described.

FIG. 1 is a functional configuration diagram of an image size estimation apparatus 100 according to the first embodiment.
A functional configuration of the image size estimation apparatus 100 according to Embodiment 1 will be described with reference to FIG.

The image size estimation device 100 (an example of a size estimation device) includes a photographic image display unit 110, an attitude reference axis setting unit 120 (an example of a reference line segment information generation unit), an imaging parameter setting unit 130, and a reference axis vector estimation unit 140 ( An example of a reference axis vector calculation unit), a reference setting unit 150 (an example of a reference line segment vector calculation unit), a dimension estimation unit 160 (an example of a measurement line segment vector calculation unit, a measurement line segment length calculation unit), and an estimation result output unit 170.
The image size estimation apparatus 100 includes a photographic image database 180, an apparatus storage unit 190, and an input device 109.

The photographic image display unit 110 displays a photographic image 101 input using the input device 109 on a display.
The posture reference axis setting unit 120 generates posture reference axis data 129 indicating the posture reference axis of the subject shown in the photographic image 101 using the two-dimensional coordinate values of the photographic image 101.
The shooting parameter setting unit 130 acquires shooting parameters 139 set in the camera when the photographic image 101 is shot.
The reference axis vector estimation unit 140 calculates a three-dimensional reference axis vector 149 that represents the posture of the subject shown in the photographic image 101 using the posture reference axis data 129 and the shooting parameters 139.
The reference setting unit 150 uses the reference axis vector 149 to calculate a three-dimensional reference line segment vector 159a that represents a reference line segment that is a line segment having a known length among the subjects shown in the photographic image 101. Further, the reference setting unit 150 acquires the reference line segment length 159b as the known length of the reference line segment.
The dimension estimation unit 160 uses the reference axis vector 149, the reference line segment vector 159a, and the reference line segment length 159b to measure a line segment whose length is an unknown line segment among the subjects shown in the photographic image 101. A measurement line segment length 169 indicating the length is calculated.
The estimation result output unit 170 outputs the measurement line segment length 169.

  The photographic image database 180 is a database that stores the photographic image 101 and the measurement line segment length 169 in association with each other.

The device storage unit 190 stores data used by the image size estimation device 100.
For example, the photographic image 101, the posture reference axis data 129, the imaging parameter 139, the reference axis vector 149, the reference line segment vector 159 a, the reference line segment length 159 b, and the measurement line segment length 169 are data stored in the device storage unit 190. It is an example.

FIG. 2 is a flowchart illustrating an image size estimation method according to the first embodiment.
The image size estimation method in Embodiment 1 will be described with reference to FIG.

In S110, the user designates the photographic image 101 using the input device 109 such as a keyboard or a mouse.
The photographic image display unit 110 acquires the photographic image 101 from the input device 109 such as a CD drive or a scanner in accordance with the user's specification. For example, the photographic image display unit 110 reads the photographic image 101 from a CD (compact disc) inserted into the CD drive. Alternatively, the photographic image display unit 110 receives the photographic image 101 from the scanner that has read the photographic image 101. However, the photographic image display unit 110 may acquire the photographic image 101 from the input device 109 other than the CD drive or the scanner.
The photographic image display unit 110 displays the acquired photographic image 101 on the display.
FIG. 3 is a diagram showing an example of the photographic image 101 in the first embodiment.
As shown in FIG. 3, the photographic image 101 is an image obtained by imaging the subject 102 from an oblique direction.
After S110, the process proceeds to S120.

In S <b> 120, the user uses the input device 109 to specify three line segments representing the posture of the subject 102 shown in the photographic image 101 as the posture reference axis of the subject 102.
The three line segments representing the posture of the subject 102 are three lines: a line segment representing the width direction of the subject 102, a line segment representing the depth direction of the subject 102, and a line segment representing the height direction of the subject 102. It's a minute.
The posture reference axis setting unit 120 specifies the designated posture reference axis using the two-dimensional coordinate value of the photographic image 101, and generates posture reference axis data 129 representing the posture reference axis. The two-dimensional coordinate values of the photographic image 101 are a horizontal coordinate value of the photographic image 101 and a vertical coordinate value of the photographic image 101.
4 and 5 are diagrams illustrating an example of the posture reference axis [lx, ly, lz] in the first embodiment.
As shown in FIG. 4, the user moves the mouse cursor 109a by operating the mouse, and moves along the line segment lx along the height side of the subject 102 and the side of the subject 102 in the depth direction. A line segment ly and a line segment lz along the side in the width direction of the subject 102 are designated. However, the user may designate the three line segments lx, ly, and lz using the input device 109 (for example, a keyboard) other than the mouse.
For example, the posture reference axis setting unit 120 specifies the two-dimensional coordinate values at both ends of the line segment lx, the two-dimensional coordinate values at both ends of the line segment ly, and the two-dimensional coordinate values at both ends of the line segment lz. Then, data including each identified coordinate value is generated as posture reference axis data 129.
The user may specify three line segments lx, ly, and lz that intersect at one point as shown in FIG. 4, or three line segments lx that do not intersect at one point as shown in FIG. , Ly, and lz may be specified.
After S120, the process proceeds to S130.
However, S120 may be executed after S130 or in parallel with S130.

In S <b> 130, the shooting parameter setting unit 130 acquires, as the shooting parameters 139, the focal length of the camera that captured the photographic image 101, the vertical size of the CCD (Charge Coupled Device), or the viewing angle in the vertical direction of the lens. A CCD is an example of an image sensor.
Hereinafter, the vertical size of the CCD is referred to as the vertical CCD size. The vertical viewing angle of the lens is referred to as the vertical viewing angle.
FIG. 6 is a relationship diagram of the imaging parameters 139 according to the first embodiment. The focal length, the vertical CCD size, and the vertical viewing angle are in a relationship as shown in FIG.
The shooting parameter setting unit 130 may acquire other shooting parameters 139 corresponding to the focal length, the vertical CCD size, or the vertical viewing angle.

FIG. 7 is a flowchart illustrating an example of the shooting parameter setting process (S130) according to the first embodiment.
The shooting parameter setting process (S130) in the first embodiment will be described with reference to FIG.

In S <b> 131, the shooting parameter setting unit 130 determines whether additional information regarding the shooting parameter 139 is added to the photographic image 101.
For example, Exif (Exchangeable image file format) is an example of additional information added to the photographic image 101.
If additional information is added to the photographic image 101 (YES), the process proceeds to S132.
When additional information is not added to the photographic image 101 (NO), the process proceeds to S136a.

In S <b> 132, the shooting parameter setting unit 130 determines whether or not the additional information of the photographic image 101 includes a focal length.
When the focal length is included in the additional information of the photographic image 101 (YES), the process proceeds to S133a.
If the additional information of the photographic image 101 does not include the focal length (NO), the process proceeds to S133b.

In S133a, the shooting parameter setting unit 130 acquires the focal length from the additional information of the photographic image 101.
After S133a, the process proceeds to S134.

In S133b, the imaging parameter setting unit 130 displays a message requesting input of the focal length on the display.
Then, the user inputs the focal length using the input device 109, and the imaging parameter setting unit 130 acquires the input focal length.
After S133b, the process proceeds to S134.

In S <b> 134, the shooting parameter setting unit 130 determines whether the additional information of the photographic image 101 includes the vertical CCD size.
If the additional information of the photographic image 101 includes the vertical CCD size (YES), the process proceeds to S135a.
If the additional information of the photographic image 101 does not include the vertical CCD size (NO), the process proceeds to S135b.

In S <b> 135 a, the shooting parameter setting unit 130 acquires the vertical CCD size from the additional information of the photographic image 101.
After S135a, the process proceeds to S137.

In S135b, the shooting parameter setting unit 130 displays a message requesting input of the vertical CCD size on the display.
Then, the user inputs the vertical CCD size using the input device 109, and the shooting parameter setting unit 130 acquires the input vertical CCD size.
However, when the additional information of the photographic image 101 includes the focal length, the shooting parameter setting unit 130 may calculate the vertical CCD size based on the focal length. For example, when the focal length is a value converted to 35 mm, the vertical CCD size is 35 mm (mm: millimeter).
After S135b, the process proceeds to S137.

In S136a, the imaging parameter setting unit 130 acquires the vertical CCD size input by the user, as in S135b.
After S136a, the process proceeds to S136b.

In S136b, the imaging parameter setting unit 130 acquires the focal length input by the user as in S133b.
After S136b, the process proceeds to S137.

In S137, the shooting parameter setting unit 130 calculates the vertical viewing angle using the vertical CCD size and the focal length.
For example, the imaging parameter setting unit 130 calculates the vertical viewing angle by calculating the following equation (1).

However, when the additional information of the photographic image 101 includes the vertical viewing angle, the shooting parameter setting unit 130 may acquire the vertical viewing angle from the additional information of the photographic image 101. Further, the imaging parameter setting unit 130 may display a message requesting input of the vertical viewing angle and acquire the vertical viewing angle input by the user.
By S137, the shooting parameter setting process (S130) ends.

Returning to FIG. 2, the description of the image size estimation method will be continued.
After S130, the process proceeds to S140.

In S140, the reference axis vector estimation unit 140 uses the posture reference axis data 129 generated in S120 and the shooting parameter 139 acquired in S130 to represent a reference representing the posture of the subject 102 shown in the photographic image 101. An axis vector 149 is calculated. The reference axis vector 149 is a three-dimensional vector value that represents the posture of the subject 102.
For example, the reference axis vector estimation unit 140 calculates the reference axis vector 149 by applying the technique disclosed in Non-Patent Document 1.
Reference axis vector estimation processing (S140) using the technique of Non-Patent Document 1 will be described separately.
After S140, the process proceeds to S150.

In S150, the reference setting unit 150 uses the reference axis vector 149 to calculate a reference line segment vector 159a representing the reference line segment of the subject 102 shown in the photographic image 101. The reference line segment is a line segment of the subject 102 whose length is known. For example, the reference line segment is designated by the user.
Further, the reference setting unit 150 acquires the length of the reference line segment (reference line segment length 159b).

  Details of the reference setting unit 150 and the reference setting process (S150) will be described below.

FIG. 8 is a configuration diagram of the reference setting unit 150 according to the first embodiment.
The configuration of the reference setting unit 150 in the first embodiment will be described with reference to FIG.

The reference setting unit 150 includes a reference axis vector projection unit 151, a reference line segment setting unit 152, a reference line segment vector calculation unit 153 (an example of a reference line segment vector calculation unit), and a reference line segment length acquisition unit 154.
The reference axis vector projection unit 151 projects and displays the reference axis vector 149 on the subject 102 shown in the photographic image 101.
The reference line segment setting unit 152 acquires reference line segment information and draws the reference line segment on the subject 102 shown in the photographic image 101.
The reference line segment vector calculation unit 153 calculates a reference line segment vector 159 a using the reference axis vector 149 projected onto the subject 102.
The reference line segment length acquisition unit 154 acquires a known reference line segment length 159b.

FIG. 9 is a flowchart showing the reference setting process (S150) in the first embodiment.
The reference setting process (S150) in the first embodiment will be described with reference to FIG.

In S151, the reference axis vector projection unit 151 projects and displays the reference axis vector 149 on the subject 102 shown in the photographic image 101 (see FIG. 10).
Projection onto the photographic image 101 is to convert the three-dimensional reference axis vector 149 into a two-dimensional vector on the photographic image plane. The photographic image plane is a plane that includes a coordinate point that is separated from the camera by a focal length and is orthogonal to the viewing direction of the camera. An image obtained by projecting the subject 102 onto the photographic image plane corresponds to the photographic image 101.
For example, the reference axis vector projection unit 151 projects the reference axis vector 149 at the intersection of three line segments (posture reference axes) of the subject 102 or the end point of any of the line segments.
Hereinafter, a plane formed by two vectors (for example, Oy and Oz) among the three vectors [Ox, Oy, Oz] representing the reference axis vector 149 is referred to as a “reference plane”.
A portion of the subject 102 whose size is known is referred to as “reference”.
FIG. 10 is a schematic diagram of the reference axis vector projection process (S151) in the first embodiment. “P ₀ ” in FIG. 10 indicates the origin of the reference axis vector 149.
After S151, the process proceeds to S152.

In S152, the user determines whether or not the reference axis vector 149 is projected on a reference whose size is known in the subject 102.
If the reference axis vector 149 is projected on the reference (YES), the process proceeds to S156.
If the reference axis vector 149 is not projected onto the reference, that is, if the reference axis vector 149 is projected onto a portion other than the reference (NO), the process proceeds to S153.

In step S153, the user operates the mouse to move the reference axis vector 149 to the reference of the subject 102 using the mouse cursor 109a. However, the user may move the reference axis vector 149 using an input device 109 (for example, a keyboard) other than a mouse.
The reference axis vector projection unit 151 projects and displays the reference axis vector 149 on the reference of the subject 102 in accordance with the movement of the reference axis vector 149 by the user.

FIG. 11 is a schematic diagram of the reference axis vector moving process (S153) in the first embodiment.
For example, the user moves the reference axis vector 149 to the reference of the subject 102 by the procedure shown in FIGS. The reference axis vector 149 before movement is indicated by a dotted line, and the reference axis vector 149 after movement is indicated by a solid line.
(1) The user moves the reference axis vector 149 in the reference plane (Oy-Oz plane).
The reference axis vector projection unit 151 projects the reference axis vector 149 on the movement destination in the reference plane.
(2) The user moves the reference axis vector 149 in the direction of the normal vector (Ox) orthogonal to the reference plane.
The reference axis vector projection unit 151 projects the reference axis vector 149 to the reference of the movement destination.
Hereinafter, the reference plane (Oy-Oz plane) represented by the reference axis vector 149 after moving to the reference is referred to as a “reference plane”.
After S153 (FIG. 9), the process proceeds to S154.

In step S154, the user operates the mouse and designates a reference line segment as a reference of the subject 102 using the mouse cursor 109a. The reference line segment is a line segment having a known length among the references. The user may specify the reference line segment using the input device 109 other than the mouse.
The reference line segment setting unit 152 draws a reference line segment according to the user's specification (see FIG. 12).
FIG. 12 is a schematic diagram of the reference line segment setting process (S154) in the first embodiment.
After S154, the process proceeds to S156.

In S155, the user designates the reference line segment as the reference of the subject 102, as in S154.
The reference line segment setting unit 152 draws a reference line segment according to the user's specification (see FIG. 13).
FIG. 13 is a schematic diagram of the reference line segment setting process (S155) in the first embodiment.
After S155, the process proceeds to S156.

In step S156, the reference line segment vector calculation unit 153 calculates a reference line segment vector 159a using the reference axis vector 149.
The reference line segment vector 159a is a three-dimensional vector value representing the reference line segment.

FIG. 14 is a schematic diagram of reference line vector calculation processing (S156) in the first embodiment.
For example, the reference line segment vector calculation unit 153 calculates the reference line segment vector Lr by the procedure shown in (1) and (2) of FIG.
(1) The reference line segment vector calculation unit 153 acquires the coordinate values of the end point Ps and the end point Pe of the reference line segment in the photographic image plane lv (Ez).
Then, the reference line vector calculation unit 153 calculates a unit vector Vs and a unit vector Ve indicating the direction from the center point Pc of the camera to the end point Ps or the end point Pe. The center point Pc of the camera is a coordinate point indicating the position of the camera when the photographic image 101 is captured, that is, a coordinate point indicating the position where the photographic image 101 is captured.
(2) The reference line segment vector calculation unit 153 calculates a reference line segment vector Lr using the unit vector Vs and the unit vector Ve.
The reference line segment vector Lr can be calculated using the following equations.

  The reference plane Γr formed by the reference can be expressed by the following equation (2).

A straight line ls passing through the camera coordinate point Pc and the end point Ps of the reference line segment can be expressed by the following expression (3s) using the unit vector Vs.
Similarly, a straight line le that passes through the coordinate point Pc of the camera and the end point Pe of the reference line segment can be expressed by the following expression (3e) using the unit vector Ve.

When the formula (3s) of the straight line ls is decomposed into xyz components, the following formula (4s) is obtained.
Similarly, when the above equation (3e) of the straight line le is decomposed into xyz components, the following equation (4e) is obtained.

When the decomposition formula (4s) of the straight line ls is substituted into the formula (2) of the reference plane Γr and expanded, the following formula (5s) is obtained.
Similarly, when the decomposition equation (4e) of the straight line le is substituted into the equation (2) of the reference plane Γr and expanded, the following equation (5e) is obtained.

However, in the above formulas (5s) and (5e), when the reference plane Γr is not the Ox-Oy plane but the Ox-Oz plane, the reference axis vector Oz (x _oz , y _oz , z _oz ) is the reference axis vector. Replaced by Oy (x _oy , y _oy , z _oy ).
Similarly, when the reference plane is the Oy-Oz plane, the reference axis vector Oz (x _oz , y _oz , z _oz ) is replaced with the reference axis vector Ox (x _ox , y _ox , z _ox ).

Reference line vector calculation unit 153 calculates the above equation (5s) calculates the parametric t _s, is calculated the parametric t _e by calculating the above equation (5e).

The parameter t _s is a variable value for obtaining the end point Ps. In other words, the parametric _{t s} is a variable value to obtain a vector _V s _{t s} from the center point Pc of the camera to the end point Ps.
Similarly, the parametric t _e is a variable value to obtain the end point Pe. That is, the parametric variable t _e is a variable value for obtaining a vector V _e t _e from the camera center point Pc to the end point Pe.

Reference line vector calculation unit 153, the a parametric value t _s and the parametric value t _e into Equation (6) below, to calculate a reference line segment vector Lr.

Through S156 described above, the reference line segment vector 159a (Lr) is calculated.
After S156 (FIG. 9), the process proceeds to S157.

In S157, the user inputs a known reference segment length 159b using the input device 109.
The reference line segment length acquisition unit 154 acquires the input reference line segment length 159b.
After S157, the reference setting process (S150) ends.

Returning to FIG. 2, the description of the image size estimation method will be continued.
After the reference setting process (S150), the process proceeds to S160.

In S160, the dimension estimation unit 160 calculates a measurement line segment vector using the reference axis vector 149, similarly to the reference setting process (S150).
Then, the dimension estimating unit 160 calculates the measurement line segment length 169 using the measurement line segment vector, the reference line segment vector 159a, and the reference line segment length 159b.
The measurement line segment vector is a three-dimensional vector value representing the measurement line segment. The measurement line segment is a line segment of the subject 102 whose length is unknown, that is, a line segment of a target portion to be measured. For example, the measurement line segment is specified by the user.

  Details of the dimension estimation unit 160 and the dimension estimation process (S160) will be described below.

FIG. 15 is a configuration diagram of the dimension estimation unit 160 in the first embodiment.
The configuration of the dimension estimation unit 160 in the first embodiment will be described with reference to FIG.

The dimension estimation unit 160 includes a reference axis vector projection unit 161, a measurement line segment setting unit 162, a measurement line segment vector calculation unit 163 (an example of a measurement line segment vector calculation unit), and a measurement line segment length calculation unit 164 (measurement line segment length). An example of a calculation unit).
The reference axis vector projection unit 161 projects and displays the reference axis vector 149 on the subject 102 shown in the photographic image 101.
The measurement line segment setting unit 162 acquires measurement line segment information, and draws the measurement line segment on the subject 102 shown in the photographic image 101.
The measurement line vector calculation unit 163 calculates a measurement line segment vector using the reference axis vector 149 projected onto the subject 102.
The measurement line segment length calculation unit 164 calculates the measurement line segment length 169 using the measurement line segment vector, the reference line segment vector 159a, and the reference line segment length 159b.

FIG. 16 is a flowchart showing the dimension estimation process (S160) in the first embodiment.
The dimension estimation process (S160) in Embodiment 1 is demonstrated based on FIG.

Steps S161 to S166 in the dimension estimation process (S160) are the same as steps S151 to S156 in the reference setting process (S150) (see FIG. 9).
That is, the processing in which “reference” in S151 to S156 is replaced with “measurement part”, “reference plane” is replaced with “measurement plane”, and “reference line segment” is replaced with “measurement line segment” is S161 to S166. .

FIG. 17 is a schematic diagram of S161 to S165 in the first embodiment.
FIG. 17 shows an outline of processing from S161 to S165. FIG. 17 corresponds to a schematic diagram (FIGS. 11 to 13) of S151 to S155.
The reference axis vector projection unit 161 projects and displays the reference axis vector 149 on the subject 102 shown in the photographic image 101 (S161).
When the reference axis vector 149 is not projected on the measurement part of the subject 102 (S162: YES), the reference axis vector projection unit 161 moves the reference axis vector 149 to the measurement part of the subject 102 (S163).
The measurement line segment setting unit 162 draws the measurement line segment according to the user's designation (S164, S165).

FIG. 18 is a schematic diagram of S166 in the first embodiment.
FIG. 18 shows an outline of the processing in S166. FIG. 18 corresponds to a schematic diagram (FIG. 14) of S156.
The measurement line segment vector calculation unit 163 calculates the measurement line segment vector Lm using the reference axis vector 149 projected onto the measurement portion (S166).

That is, the measurement line vector calculation unit 163 calculates a unit vector Vms from the camera center point Pc to the end point Pms of the measurement line segment and a unit vector Vme from the camera center point Pc to the end point Pme of the measurement line segment. To do.
Also, measurement line segment vector calculation unit 163 calculates the following equation (7s) to calculate the parametric _{t ms,} calculates the parametric _{t me} by calculating the following formula (7e).
Then, the measurement line vector calculation unit 163 calculates the following equation (7m) using the parameter t _ms and the parameter t _me, and calculates the measurement line vector Lm.

However, in the above formulas (7s) and (7e), when the measurement plane Γm is not the Oy-Oz plane but the Ox-Oz plane, the reference axis vector Ox (x _ox , y _ox , z _ox ) is the reference axis vector. Replaced by Oy (x _oy , y _oy , z _oy ).
Similarly, when the reference plane is the Ox-Oy plane, the reference axis vector Ox (x _ox , y _ox , z _ox ) is replaced with the reference axis vector Oz (x _oz , y _oz , z _oz ).

The parameter t _ms is a variable value for obtaining the end point Pms. That is, the parameter t _ms is a variable value for obtaining a vector V _ms t _ms from the camera center point Pc to the end point Pms.
Similarly, the parameter t _me is a variable value for obtaining the end point Pme. That is, the parameter t _me is a variable value for obtaining a vector V _me t _me from the camera center point Pc to the end point Pme.

  In S167 (see FIG. 16), the measurement line segment length calculation unit 164 is obtained by the measurement line segment vector calculated in S166, the reference line segment vector 159a calculated by the reference setting unit 150, and the reference setting unit 150. The measurement line segment length 169 is calculated using the reference line segment length 159b.

In the following, equation (8) for calculating the measurement line segment length l _{m_true} is shown.
In the following equation (8), “Lm” is a measurement line segment vector, “Lr” is a reference line segment vector 159a, and “l _{r_true} ” is a reference line segment length 159b.

That is, the measurement line segment length calculation unit 164 calculates, as the measurement line segment length 169, a value obtained by multiplying the reference line segment length by the ratio of the size of the measurement line segment vector to the reference line segment vector.
After S167, the dimension estimation process (S160) ends.

Returning to FIG. 2, the description of the image size estimation method will be continued.
After the dimension estimation process (S160), the process proceeds to S170.

In S170, the estimation result output unit 170 outputs the measurement line segment length 169 calculated in S160.
For example, the estimation result output unit 170 displays the measurement line segment length 169 on the display 911.
Further, the estimation result output unit 170 associates the measurement line segment length 169 and the photographic image 101 (including information such as a measurement line segment or a measurement line segment vector) and stores them in the photographic image database 180.
The measurement part can be used as a new reference, and the measurement line segment length 169 can be used as a new known reference line length.

  Next, reference axis vector estimation processing (S140) using the technique of Non-Patent Document 1 will be described.

FIG. 19 is a flowchart showing the reference axis vector estimation process (S140) in the first embodiment.
The reference axis vector estimation process (S140) in the first embodiment will be described with reference to FIG.

In S141, the reference axis vector estimation unit 140 determines that the posture reference axes [lx, ly, lz] of the subject 102 intersect at one point as shown in FIG. 4 based on the posture reference axis data 129, or as shown in FIG. It is judged whether it intersects with one point.
If the posture reference axes [lx, ly, lz] intersect at one point (YES), the process proceeds to S142.
If the posture reference axes [lx, ly, lz] do not intersect at one point (NO), the process proceeds to S143.

In S142, the reference axis vector estimation unit 140 calculates a reference axis vector 149 using the FPM coordinate system defined in Non-Patent Document 1.
FPM is an abbreviation for First Perspective Moving.
Details of the reference axis vector calculation process [FPM coordinate system] (S142) will be described separately.
After S142, the reference axis vector estimation process (S140) ends.

In S143, the reference axis vector estimation unit 140 calculates the reference axis vector 149 using the SPM coordinate system defined in Non-Patent Document 1.
SPM is an abbreviation for Second Perspective Moving.
Details of the reference axis vector estimation process [SPM coordinate system] (S143) will be described separately.
After S143, the reference axis vector estimation process (S140) ends.

FIG. 20 is a flowchart showing the reference axis vector calculation process [FPM coordinate system] (S142) in the first embodiment.
The reference axis vector calculation process (S142) using the FPM coordinate system will be described with reference to FIG.

In S142A, the reference axis vector estimation unit 140 obtains a unit vector [Ex, Ey, Ez] from the camera center point Pc to the end point of the attitude reference axis [lx, ly, lz] in the photographic image plane lv (Ez). Calculate (see FIG. 21).
Hereinafter, the unit vector [Ex, Ey, Ez] is referred to as an “endpoint unit vector”.
FIG. 21 is a schematic diagram of endpoint unit vector calculation processing (S142A) in the first embodiment.
After S142A, the process proceeds to S142B.

In S142b, the reference axis vector estimation unit 140 calculates the unit vector Ec to the intersection _{l 0} of attitude reference axis in the image plane lv (Ez) from the center point Pc of the camera (see Figure 22).
Hereinafter, the unit vector Ec is referred to as “intersection unit vector”.
FIG. 22 is a schematic diagram of the intersection unit vector calculation process (S142B) in the first embodiment.
After S142B, the process proceeds to S142C.

In S142C, the reference axis vector estimation unit 140 calculates a coordinate point Pf that is separated from the camera center point Pc by the focal length f in the direction of the intersection unit vector Ec (see (1) in FIG. 23).
The reference axis vector estimation unit 140 calculates a plane lv (Ec) that includes the coordinate point Pf and is orthogonal to the intersection unit vector Ec (see (2) in FIG. 23). That is, the intersection unit vector Ec is a normal vector of the plane lv (Ec).
Hereinafter, the plane lv (Ec) is referred to as “FPM coordinate plane”. Further, the coordinate point Pf is referred to as “FPM coordinate origin”.
FIG. 23 is a schematic diagram of FPM coordinate plane calculation processing (S142C) in the first embodiment.
After S142C, the process proceeds to S142D.

In S142D, the reference axis vector estimation unit 140 calculates the intersection [Px ′, Py ′, Pz ′] between the straight line including the end point unit vector [Ex, Ey, Ez] and the FPM coordinate plane lv (Ec) ( (See (1) in FIG. 24).
The reference axis vector estimation unit 140 calculates a vector [Gx, Gy, Gz] from the FPM coordinate origin Pf to the intersection [Px ′, Py ′, Pz ′] (see (2) in FIG. 24).
Hereinafter, the vector [Gx, Gy, Gz] is referred to as “FPM coordinate plane vector”.
FIG. 24 is a schematic diagram of FPM coordinate plane vector calculation processing (S142D) in the first embodiment.
After S142D, the process proceeds to S142E.

In S142E, the reference axis vector estimation unit 140 selects one of the FPM coordinate plane vectors [Gx, Gy, Gz] (for example, Gx).
The reference axis vector estimation unit 140 calculates a unit vector Ea (= Gx / | Gx |) of the selected FPM coordinate plane vector (Gx) (see FIG. 25).
The reference axis vector estimation unit 140 calculates an outer product vector Eb (= Ec × Ea) of the intersection unit vector Ec and the unit vector Ea (see FIG. 25).
The unit vector Ea is the A-axis unit vector of the FPM coordinate system, the outer product vector Eb is the B-axis unit vector of the FPM coordinate system, and the intersection unit vector Ec is the C-axis unit vector of the FPM coordinate system. The unit vector [Ea, Eb, Ec] is also called “orthogonal basis”.
FIG. 25 is a schematic diagram of unit vector calculation processing (S142E) of the FPM coordinate system in the first embodiment.
After S142E, the process proceeds to S142F.

In S142F, the reference axis vector estimation unit 140 calculates a reference axis vector [E1, E2, E3] of the subject coordinate system using the FPM coordinate plane vector [Gx, Gy, Gz].
The subject coordinate system is a local coordinate system set for the subject 102. The reference axis vector [E1, E2, E3] is a unit vector (normal orthogonal basis) of the subject coordinate system.
FIG. 26 is a relationship diagram of reference axis vectors of the subject coordinate system in the first embodiment.
In Figure 26, the angle Δβ apparent formed by the Gz and Gy, the angle beta ₁ apparent formed by the angle beta ₂ and Gx and Gz apparent formed by the Gx and Gy are be represented by the following formula (11) it can.

  Furthermore, tan Δα of the actual angle Δα formed by E1 and E2 can be expressed by the following formula (12) using the angle ψ formed by E3 and the C axis of the FPM coordinate system.

  Here, since the angle Δα formed by the reference axis vector [E1, E2] of the subject 102 is 90 degrees, the above equation (12) can be expanded into the following equation (13).

The reference axis vector [E1, E2, E3] of the subject coordinate system can be expressed by the following equation (14) based on the relationship between the above equations (11) and (13).
In the following formula (14), α ₁ is an actual angle formed by E3 and E1, and α ₂ is an actual angle formed by E3 and E2.

  After S142F (FIG. 20), the process proceeds to S142G.

In S142G, the reference axis vector estimation unit 140 converts the reference axis vector [E1, E2, E3] of the subject coordinate system into the reference axis vector [O _x , O _y , O _z ] of the world coordinate system. M _h is calculated. The transformation matrix M _h can be expressed using the following equation (15).

The meanings of the symbols used in the above equation (15) are as follows.
( _XEa , _yEa , _zEa ) is a vector value representing the A-axis unit vector Ea (see FIG. 25) of the FPM coordinate system in the world coordinate system.
(X _Eb , y _Eb , z _Eb ) is a vector value representing the B-axis unit vector Eb (see FIG. 25) of the FPM coordinate system in the world coordinate system.
(X _Ec , y _Ec , z _Ec ) is a vector value representing the C-axis unit vector Ec (see FIG. 25) of the FPM coordinate system in the world coordinate system.
(A _E1 , b _E1 , c _E1 ) are vector values representing the reference axis vector E1 (see FIG. 26) of the subject coordinate system in the FPM coordinate system.
(A _E2 , b _E2 , c _E2 ) are vector values representing the reference axis vector E2 (see FIG. 26) of the subject coordinate system in the FPM coordinate system.
(A _E3 , b _E3 , c _E3 ) are vector values representing the reference axis vector E3 (see FIG. 26) of the subject coordinate system in the FPM coordinate system.

Then, the reference axis vector estimation unit 140 converts the reference axis vector [E1, E2, E3] of the subject coordinate system into the reference axis vector [O _x , O _y , O of the world coordinate system using the transformation matrix H _h described above. _z ], and outputs the reference axis vector [O _x , O _y , O _z ] of the world coordinate system as the reference axis vector 149 of the subject 102.
Equation (16) for converting the reference axis vector [E1, E2, E3] of the subject coordinate system into the reference axis vector [O _x , O _y , O _z ] of the world coordinate system is shown below.

  With S142G, the reference axis vector calculation process [FPM coordinate system] (S142) ends.

FIG. 27 is a flowchart showing a reference axis vector calculation process [SPM coordinate system] (S143) in the first embodiment.
The reference axis vector calculation process (S143) using the SPM coordinate system will be described with reference to FIG.

In S143A, the reference axis vector estimation unit 140 selects two attitude reference axes (for example, lx and ly) from the attitude reference axes [lx, ly, lz] in the photographic image plane lv (Ez).
The reference axis vector estimation unit 140 calculates an intersection Pxy on the extension of the two selected posture reference axes (lx, ly) (see (1) in FIG. 28).
The reference axis vector estimation unit 140 calculates a unit vector Exy from the camera center point Pc to the intersection point Pxy (see (2) in FIG. 28). Hereinafter, the unit vector Exy is referred to as “intersection unit vector”.
FIG. 28 is a schematic diagram of the intersection unit vector calculation process (S143A) in the first embodiment.
After S143A, the process proceeds to S143B.

In S143B, the reference axis vector estimation unit 140 calculates a coordinate point Pf that is separated from the center point Pc of the camera by the focal length f in the direction of the intersection unit vector Exy (see (1) in FIG. 29).
The reference axis vector estimation unit 140 calculates a plane lv (Ec) including the coordinate point Pf and orthogonal to the intersection unit vector Exy (see (2) in FIG. 29). That is, the intersection unit vector Exy is a normal vector of the plane lv (Ec).
Hereinafter, the plane lv (Ec) is referred to as “SPM coordinate plane”. The coordinate point Pf is referred to as “SPM coordinate origin”.
FIG. 29 is a schematic diagram of the SPM coordinate plane calculation process (S143B) in the first embodiment.
After S143B, the process proceeds to S143C.

In S143C, the reference axis vector estimation unit 140 projects the attitude reference axis [lx, ly, lz] from the photographic image plane lv (Ez) to the SPM coordinate plane lv (Ec) (see FIG. 30).
Hereinafter, the attitude reference axis projected onto the SPM coordinate plane lv (Ec) is denoted as [lx ′, ly ′, lz ′].
FIG. 30 is a schematic diagram of posture reference axis projection processing (S143C) in the first embodiment.
After S143C, the process proceeds to S143D.

In S143D, the reference axis vector estimation unit 140 selects the posture reference axis (lx, ly) selected in S143A from the posture reference axes [lx ′, ly ′, lz ′] projected on the SPM coordinate plane lv (Ec). The posture reference axes (lx ′, ly ′) after projection are selected.
The reference axis vector estimation unit 140 determines the bisector of the angle formed by the selected posture reference axis (lx ′, ly ′) as the S axis of the SPM coordinate system (see (1) in FIG. 31).
Further, the reference axis vector estimation unit 140 determines the normal vector Exy of the SPM coordinate plane lv (Ec) as the C axis of the SPM coordinate plane, and sets the axis orthogonal to the S axis and the C axis as the T axis of the SPM coordinate system. (See (1) in FIG. 31).
Then, the reference axis vector estimation unit 140 calculates an S-axis unit vector Es, a T-axis unit vector Et, and a C-axis unit vector Ec in the SPM coordinate system (see (2) in FIG. 31).
FIG. 31 is a schematic diagram of unit vector calculation processing (S143D) of the SPM coordinate system in the first embodiment.

Below, the equation (21) for calculating the unit vector [Es, Et, Ec] of the SPM coordinate system is shown.
“Gx ′” used in the following equation (21) is a unit vector having the same direction as the posture reference axis lx ′. “Gy ′” is a unit vector having the same direction as the orientation reference axis ly ′.

  After S143D, the process proceeds to S143E.

In S143E, the reference axis vector estimation unit 140 uses the posture reference axes [lx ′, ly ′, lz ′] in the SPM coordinate plane as follows, and the reference axis vectors [E1, E2, E3 of the subject coordinate system are used as follows. ] Is calculated.
The subject coordinate system is a local coordinate system set for the subject 102. The reference axis vector [E1, E2, E3] is a unit vector (normal orthogonal basis) of the subject coordinate system.

FIG. 32 is a relationship diagram of attitude reference axes in the SPM coordinate plane according to the first embodiment.
The PAT curve shown in FIG. 32 is a curve defined in Non-Patent Document 1. PAT is an abbreviation for Perspective Angle Transform.
The PAT curve can be expressed by the following equation (22) using the angle Δβ formed by lx ′ and ly ′ and the focal length f.

  A straight line expression including lz ′ can be expressed by the following expression (23) using the variable γ and the variable δ.

  The polar coordinate value of the intersection P1 between the PAT curve and the straight line lz ′ can be calculated by the following formula (24) using the above formula (22) and the above formula (23).

  The polar coordinate value of the intersection point P1 between the PAT curve and the straight line lz ′ is expressed as (cos η) using the length a of the straight line from Pf to P1 and the angle η formed by the straight line from Pf to P1 and the S axis. , Asin η).

  Here, the reference axis vector [E1, E2, E3] of the subject coordinate system can be expressed by the following equation (25).

The reference axis vector estimation unit 140 calculates the reference axis vector [E1, E2, E3] of the subject coordinate system by calculating the above equation (25).
After S143E (FIG. 27), the process proceeds to S143F.

In S143F, the reference axis vector estimation unit 140 selects a reference point _{l 0} from the image plane lv (Ez). For example, the reference axis vector estimation unit 140 selects the end point of the posture reference axis lx as the reference point ₁₀ (see (1) in FIG. 33).
The reference axis vector estimation unit 140 calculates a unit vector E ₀ from the camera center point Pc to the reference point ₁₀ (see (2) in FIG. 33).
Reference axis vector estimation unit 140 uses a unit vector E _0, to calculate the coordinate point P ₀ apart by the focal length f in the direction from the center point Pc of the camera to the reference point l ₀ (in FIG. 33 (2) reference).
Hereinafter, the coordinate point P _{0 is referred} to as “reference origin”.
FIG. 33 is a schematic diagram of the reference origin calculation process (S143F) in the first embodiment.
After S143F, the process proceeds to S143G.

In S143G, the reference axis vector estimation unit 140 converts the reference axis vector [E1, E2, E3] of the subject coordinate system into the reference axis vector [O _x , O _y , O _z ] of the world coordinate system. M _h is calculated. The transformation matrix M _h can be expressed by the following equation (26).

The meanings of the symbols used in the above equation (26) are as follows.
(X _p0 , y _p0 , z _p0 ) is a coordinate value representing the reference origin P ₀ (see (2) in FIG. 33) in the world coordinate system (x, y, z).
(X _Es , y _Es , z _Es ) is a vector value representing the S-axis unit vector Es (see (2) in FIG. 31) of the SPM coordinate system in the world coordinate system.
(X _Et , y _Et , z _Et ) is a vector value representing the T-axis unit vector Et (see (2) in FIG. 31) of the FPM coordinate system in the world coordinate system.
(X _Ec , y _Ec , z _Ec ) is a vector value representing the C-axis unit vector Ec (= Exy) in the FPM coordinate system in the world coordinate system.
(A _E1 , b _E1 , c _E1 ) is a vector value representing the reference axis vector E1 (see Expression (25)) of the subject coordinate system in the FPM coordinate system.
(A _E2 , b _E2 , c _E2 ) is a vector value representing the reference axis vector E2 (see Expression (25)) of the subject coordinate system in the FPM coordinate system.
(A _E3 , b _E3 , c _E3 ) is a vector value representing the reference axis vector E3 (see Expression (25)) of the subject coordinate system in the FPM coordinate system.

Then, the reference axis vector estimation unit 140 converts the reference axis vector [E1, E2, E3] of the subject coordinate system into the reference axis vector [O _x , O _y , O of the world coordinate system using the transformation matrix H _h described above. _z ], and outputs the reference axis vector [O _x , O _y , O _z ] of the world coordinate system as the reference axis vector 149 of the subject 102.
Equation (27) for converting the reference axis vector [E1, E2, E3] of the subject coordinate system into the reference axis vector [O _x , O _y , O _z ] of the world coordinate system is shown below.

  With S143G, the reference axis vector calculation process [SPM coordinate system] (S143) ends.

FIG. 34 is a diagram illustrating an example of hardware resources of the image size estimation apparatus 100 according to the first embodiment.
In FIG. 34, an image size estimation apparatus 100 (an example of a computer) includes a CPU 901 (Central Processing Unit). The CPU 901 is connected to hardware devices such as a ROM 903, a RAM 904, a communication board 905 (communication device), a display 911 (display device), a keyboard 912, a mouse 913, a drive 914, and a magnetic disk device 920 via a bus 902. Control hardware devices. The drive 914 is a device that reads and writes storage media such as an FD (Flexible Disk), a CD (Compact Disc), and a DVD (Digital Versatile Disc).
The ROM 903, the RAM 904, the magnetic disk device 920, and the drive 914 are examples of storage devices. A keyboard 912, a mouse 913, and a communication board 905 are examples of input devices. The display 911 and the communication board 905 are examples of output devices.

  The communication board 905 is wired or wirelessly connected to a communication network such as a LAN (Local Area Network), the Internet, or a telephone line.

  The magnetic disk device 920 stores an OS 921 (operating system), a program group 922, and a file group 923.

  The program group 922 includes programs that execute the functions described as “units” in the embodiments. A program (for example, a size estimation program) is read and executed by the CPU 901. That is, the program causes the computer to function as “to part”, and causes the computer to execute the procedures and methods of “to part”.

  The file group 923 includes various data (input, output, determination result, calculation result, processing result, etc.) used in “˜part” described in the embodiment.

In the embodiment, arrows included in the configuration diagrams and flowcharts mainly indicate input and output of data and signals.
The processing of the embodiment described based on the flowchart and the like is executed using hardware such as the CPU 901, a storage device, an input device, and an output device.

  In the embodiment, what is described as “to part” may be “to circuit”, “to apparatus”, and “to device”, and “to step”, “to procedure”, and “to processing”. May be. That is, what is described as “to part” may be implemented by any of firmware, software, hardware, or a combination thereof.

According to the first embodiment, the correct size of the subject 102 can be measured using the single photographic image 101 that reflects the subject 102 from an oblique direction.
For example, when a suspicious ship is reflected as a subject in a photographic image 101 obtained by imaging the sea, the dimensions of the suspicious ship can be measured according to the first embodiment. In this case, the floating ring attached to the suspicious ship can be used as a reference. This is because the float mounted on the ship is generally standardized and has a predetermined size.
In addition, a database containing the dimensions of the ships used in each country is prepared, and by comparing the measured dimensions of the suspicious ship with the dimensions recorded in the database, the suspicious ship is in which country. Can also be specified.
However, the ship is an example of the subject 102, and the float is an example of a reference. That is, the dimensions of the subject 102 other than the ship may be measured according to the first embodiment, or a reference other than the floating ring may be used.

In addition, a three-dimensional model of the subject 102 can be generated by calculating the size of each part of the subject 102 (for example, a book) (see (1) in FIG. 35).
Furthermore, the subject 102 viewed from different directions can be displayed using a three-dimensional model of the subject 102 (see (2) to (5) in FIG. 35).
Note that the three-dimensional model of the subject 102 can be drawn more clearly by pasting the pixels of the photographic image 101 as a texture on the three-dimensional model of the subject 102.
FIG. 35 is a diagram illustrating a three-dimensional model of the subject 102 in the first embodiment.

In the first embodiment, for example, the following dimension estimation apparatus (100) has been described. The code | symbol or name of a corresponding structure is described in a parenthesis.
The dimension estimation apparatus includes a reference line segment information generation unit (120), a reference axis vector calculation unit (140), a reference line segment vector calculation unit (153), a measurement line segment vector calculation unit (163), and a measurement A line segment length calculation unit (164).
The reference line segment information generation unit includes three reference line segments along a width direction, a height direction, and a depth direction of the object shown in a target image (101) obtained by imaging the object (102) from an oblique direction. Is generated using the two-dimensional coordinate values of the target image.
The reference axis vector calculation unit, based on the reference line segment information generated by the reference line segment information generation unit and an imaging parameter (imaging parameter) of the target image, and the width direction of the object and the Three reference axis vectors (149) representing the height direction and the depth direction using three-dimensional vector values are calculated.
The reference line segment vector calculation unit uses at least one reference axis vector among the three reference axis vectors calculated by the reference axis vector calculation unit, among the objects reflected in the target image. A vector value of a reference line segment having a known length is calculated as a reference line segment vector (159a).
The measurement line segment vector calculation unit uses at least one reference axis vector among the three reference axis vectors calculated by the reference axis vector calculation unit, among the objects reflected in the target image. A vector value of a measurement line segment whose length is unknown is calculated as a measurement line segment vector.
The measurement line segment length calculation unit is calculated by the known length (159b) of the reference line segment, the reference line segment vector calculated by the reference line segment vector calculation unit, and the measurement line segment vector calculation unit. The measured line segment length (169) is calculated using the measured line segment vector.

  100 Image Size Estimation Device, 101 Photo Image, 102 Subject, 109 Input Device, 109a Mouse Cursor, 110 Photo Image Display Unit, 120 Posture Reference Axis Setting Unit, 129 Posture Reference Axis Data, 130 Shooting Parameter Setting Unit, 139 Shooting Parameter, 140 reference axis vector estimation unit, 149 reference axis vector, 150 reference setting unit, 151 reference axis vector projection unit, 152 reference line segment setting unit, 153 reference line segment vector calculation unit, 154 reference line segment length acquisition unit, 159a reference line Segment vector, 159b Reference line segment length, 160 Dimension estimation unit, 161 Reference axis vector projection unit, 162 Measurement line segment setting unit, 163 Measurement line segment vector calculation unit, 164 Measurement line segment length calculation unit, 169 Measurement line segment length, 170 Estimated results Fruit output unit, 180 photo image database, 190 device storage unit, 901 CPU, 902 bus, 903 ROM, 904 RAM, 905 communication board, 911 display, 912 keyboard, 913 mouse, 914 drive, 920 magnetic disk device, 921 OS, 922 programs, 923 files.

Claims (23)

A reference representing three reference line segments along the width direction, the height direction, and the depth direction of the target object shown in the target image obtained by imaging the target object from an oblique direction, using two-dimensional coordinate values of the target image. A reference line segment information generation unit for generating line segment information;
Based on the reference line segment information generated by the reference line segment information generation unit and the imaging parameters of the target image, the width direction, the height direction, and the depth direction of the target object are three-dimensionally displayed. A reference axis vector calculation unit for calculating three reference axis vectors represented using vector values;
Using the reference axis vector of at least one of the three reference axis vectors calculated by the reference axis vector calculation unit, the length of the target object shown in the target image is a known line segment A reference line segment vector calculating unit for calculating a reference line segment vector value as a reference line segment vector;
The length of the object shown in the target image is an unknown line segment using at least one of the three reference axis vectors calculated by the reference axis vector calculation unit. A measurement line vector calculation unit that calculates a vector value of the measurement line segment as a measurement line segment vector;
Using the known length of the reference line segment, the reference line segment vector calculated by the reference line segment vector calculation unit, and the measurement line segment vector calculated by the measurement line segment vector calculation unit, A dimension estimation apparatus comprising: a measurement line segment length calculation unit that calculates a length of the measurement line segment. The reference line segment vector calculator is
A unit vector V _s having a direction from a coordinate point P _c indicating a position where the target image is captured to one end point P _s of the reference line segment is calculated;
It calculates a unit vector V _e from the coordinate point P _c with the direction of the other end point P _e of the reference line,
The reference line segment vector is calculated using the unit vector V _s , the unit vector V _e, and at least one of the three reference axis vectors. The dimension estimation apparatus of description. The reference line segment vector calculator is
The parametric t _s for from the coordinate point P _c to obtain a vector V _s t _s to the end point P _s of the reference line segment, calculated by using one of the reference axis vector of the three reference axis vector And
The parametric t _e for obtaining the vector V _e t _e from the coordinate point P _c to the end point P _e of the reference line, is calculated by using the one of the reference axis vector,
Wherein the vector V _s t _s by using the vector V _e t _e, the size estimating apparatus according to claim 2, wherein the calculating the reference line segment vector. The dimension estimation apparatus according to claim 3, wherein the reference line segment vector calculation unit calculates the reference line segment vector L _r by calculating the following equation (101).

The reference line segment vector calculation unit calculates the following expression (102) using the coordinate value of the origin P ₀ of the three reference axis vectors and using the reference axis vector Oz as one of the reference axis vectors, The size estimation apparatus according to claim 4, wherein the parameter t _s and the parameter t _te are calculated.

The measurement line segment vector calculation unit
A unit vector V _ms having a direction from the coordinate point P _c indicating the position where the target image is captured to one end point P _ms of the measurement line segment,
A unit vector V _me having a direction from the coordinate point P _c to the other end point P _me of the reference line segment is calculated;
2. The measurement line segment vector is calculated using the unit vector V _ms , the unit vector V _me, and at least one of the three reference axis vectors. The dimension estimation apparatus according to claim 5. The measurement line segment vector calculation unit
The parametric t _ms for from the coordinate point P _c to obtain a vector V _ms t _ms to the end point P _ms of the measurement line segment, calculated by using one of the reference axis vector of the three reference axis vector And
A parametric variable t _me for obtaining a vector V _me t _me from the coordinate point P _c to the end point P _{me of the} measurement line segment is calculated using any of the reference axis vectors;
The dimension estimation apparatus according to claim 6, wherein the reference line segment vector is calculated using the vector V _ms t _ms and the vector V _me t _me . The measurement line segment vector calculation unit has the following formula (103) was calculated to calculate the measurement line segment vector L _m, it dimension estimating apparatus according to claim 7, wherein the.

The measurement line segment vector calculation unit calculates the following formula (104) using the coordinate value of the origin P ₀ of the three reference axis vectors and using the reference axis vector Ox as one of the reference axis vectors, the parametric t _ms and calculates and said parametric t _me, it dimension estimating apparatus according to claim 8, wherein.

The measurement line segment length calculation unit calculates, as the length of the measurement line segment, a value obtained by multiplying a ratio of the size of the measurement line segment vector to the reference line segment vector by a known length of the reference line segment. The size estimation apparatus according to any one of claims 1 to 9, wherein The reference axis vector calculation unit includes:
Three unit vectors representing the width direction, the height direction, and the depth direction of the object based on the reference line segment information and the imaging parameter, and an FPM (First Perspective Moving) coordinate system or Calculate three unit vectors in the SPM (Second Perspective Moving) coordinate system,
11. The three vectors in the world coordinate system are calculated as the three reference axis vectors using the three unit vectors in the FPM coordinate system or the SPM coordinate system. The dimension estimation apparatus of description. A reference representing three reference line segments along the width direction, the height direction, and the depth direction of the target object shown in the target image obtained by imaging the target object from an oblique direction, using two-dimensional coordinate values of the target image. A reference line segment information generation process for generating line segment information;
Based on the reference line segment information generated by the reference line segment information generation process and the imaging parameters of the target image, the width direction, the height direction, and the depth direction of the target object are three-dimensional. A reference axis vector calculation process for calculating three reference axis vectors represented using vector values;
Using the reference axis vector of at least one of the three reference axis vectors calculated by the reference axis vector calculation process, the length of the object shown in the target image is a known line segment A reference line vector calculation process for calculating a reference line vector value as a reference line vector;
It is a line segment whose length is unknown among the objects shown in the target image using at least one of the three reference axis vectors calculated by the reference axis vector calculation process. A measurement line vector calculation process for calculating a vector value of the measurement line segment as a measurement line segment vector;
Using the known length of the reference line segment, the reference line segment vector calculated by the reference line segment vector calculation process, and the measurement line segment vector calculated by the measurement line segment vector calculation process, A dimension estimation program for causing a computer to execute a measurement line segment length calculation process for calculating the length of a measurement line segment. The reference line segment vector calculation process includes:
A process of calculating a unit vector V _s having a direction from a coordinate point P _c indicating a position where the target image is captured to one end point P _s of the reference line segment;
A process of calculating the unit vector V _e from the coordinate point P _c with the direction of the other end point P _e of the reference line,
And a process of calculating the reference line segment vector using the unit vector V _s , the unit vector _Ve, and at least one of the three reference axis vectors. The dimension estimation program according to claim 12. The reference line segment vector calculation process includes:
The parametric t _s for from the coordinate point P _c to obtain a vector V _s t _s to the end point P _s of the reference line segment, calculated by using one of the reference axis vector of the three reference axis vector Processing to
The parametric t _e for obtaining the vector V _e t _e from the coordinate point P _c to the end point P _e of the reference line, and processing for calculating using said one of the reference axis vector,
Wherein the vector V _s t _s by using the vector V _e t _e, according to claim 13, wherein the dimension estimating program, characterized in that it comprises a process for calculating the reference line segment vector. The reference line segment vector calculation process according to claim 14, wherein the size estimation program which calculates the following equation (101), characterized in that it comprises a process for calculating the reference line segment vector L _r.

The reference line segment vector calculation process uses the coordinate value of the origin P ₀ of the three reference axis vectors and calculates the following formula (102) using the reference axis vector Oz as any of the reference axis vectors, The size estimation program according to claim 15, further comprising a process of calculating the parameter t _s and the parameter t _te .

The measurement line segment vector calculation process includes:
A process of calculating a unit vector V _ms having a direction from a coordinate point P _c indicating a position where the target image is captured to one end point P _ms of the measurement line segment;
A process of calculating a unit vector V _me having a direction from the coordinate point P _c to the other end point P _me of the reference line segment;
And a process of calculating the measurement line segment vector using the unit vector V _ms , the unit vector V _me, and at least one of the three reference axis vectors. The size estimation program according to any one of claims 12 to 16. The measurement line segment vector calculation process includes:
The parametric t _ms for from the coordinate point P _c to obtain a vector V _ms t _ms to the end point P _ms of the measurement line segment, calculated by using one of the reference axis vector of the three reference axis vector Processing to
The parametric t _me to obtain a vector V _me t _me from the coordinate point P _c to the end point P _me of the measurement line segment, the process of calculating using said one of the reference axis vector,
18. The size estimation program according to claim 17, further comprising: calculating the reference line segment vector using the vector V _ms t _ms and the vector V _me t _me . The measurement line segment vector calculation process according to claim 18, wherein the dimension estimating program, characterized in that it comprises a process for calculating the measurement line segment vector L _m by calculating the following equation (103).

The measurement line segment vector calculation process uses the coordinate value of the origin P ₀ of the three reference axis vectors and calculates the following formula (104) using the reference axis vector Ox as any one of the reference axis vectors, the parametric t _ms and the parametric t _me including a process of calculating the claim 19, wherein the dimension estimating program, characterized in that.

In the measurement line segment length calculation process, a value obtained by multiplying a ratio of the magnitude of the measurement line segment vector to the reference line segment vector by a known length of the reference line segment is calculated as the length of the measurement line segment. The size estimation program according to any one of claims 12 to 20, further comprising a process. The reference axis vector calculation process includes:
Three unit vectors representing the width direction, the height direction, and the depth direction of the object based on the reference line segment information and the imaging parameter, and an FPM (First Perspective Moving) coordinate system or A process of calculating three unit vectors in an SPM (Second Perspective Moving) coordinate system;
22. The method of calculating three vectors of the world coordinate system as the three reference axis vectors using the three unit vectors of the FPM coordinate system or the SPM coordinate system. The dimension estimation program in any one of. A reference line segment information generation unit displays three reference line segments along the width direction, the height direction, and the depth direction of the target object that are reflected in the target image obtained by imaging the target object from an oblique direction. Generate the reference line segment information expressed using the coordinate value of
Based on the reference line segment information generated by the reference line segment information generation unit and the imaging parameters of the target image, a reference axis vector calculation unit is configured to calculate the width direction, the height direction, and the height of the target object. Calculate three reference axis vectors that represent the depth direction using three-dimensional vector values,
The reference line vector calculation unit uses a reference axis vector of at least one of the three reference axis vectors calculated by the reference axis vector calculation unit, and uses the longest of the objects shown in the target image. A vector value of a reference line segment whose length is a known line segment is calculated as a reference line segment vector,
The measurement line segment vector calculation unit uses a reference axis vector of at least one of the three reference axis vectors calculated by the reference axis vector calculation unit, and uses the longest of the objects shown in the target image. The vector value of the measurement line segment whose length is unknown is calculated as the measurement line segment vector,
The measurement line segment length calculation unit has a known length of the reference line segment, the reference line segment vector calculated by the reference line segment vector calculation unit, and the measurement line calculated by the measurement line segment vector calculation unit. A dimension estimation method, wherein a length of the measurement line segment is calculated using a segment vector.

Images