JP2005227929A - Processing method of captured image of object, image display system, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To process an image such as reading information after determining the position of the marker, estimating camera parameters, obtaining a normalized image corrected for distortion and the like in processing of an image having a positioning marker.
An image processing means 12 searches for a position of a positioning marker, detects a temporary reading area, calculates a temporary camera parameter matrix, generates a temporary data cell read image, a shape and size of the positioning marker, and a temporary data cell read image Determination of the position to be specified by positioning markers using and, calculation of the area including the entire object, creation of a data cell read image using the temporary camera parameter matrix, detection of the data cell area in the data cell read image Then, the size information of the object is extracted from the detected data cell area, and the camera parameter matrix for associating the pixels included in the object area in the captured video with the points on the actual object is calculated.
[Selection] Figure 2

Description

  The present invention relates to a technique for reading a code from an image obtained by capturing an object and processing the code.

  A technique for estimating the position, orientation, and coordinate system of an object in an image using a two-dimensional code has been made so far. A system for displaying in a superimposed manner is disclosed as a prior art. In general, a two-dimensional code is composed of a data cell area for holding information and a code for specifying the position of a two-dimensional code called a positioning marker and for ensuring reading of information from the data cell area. . The method for reading the two-dimensional code when estimating the position, orientation, and coordinate system is as follows: (1) search for a positioning marker from the captured image, (2) estimate the coordinate system, (3) data cell area It consists of three steps of reading data from.

Here, a specific procedure in the system in which the object is a paper medium will be described. First, a positioning marker is searched in the image, and the position of the positioning marker in the image is estimated. Next, the geometric relationship between the points on the screen and the coordinates on the actual printed matter is calculated. Using the fact that the actual printed matter is a plane as shown in FIG. 16, the four corner points of the two-dimensional code in the image are associated with the four corner points of the actual two-dimensional code, and the camera parameter matrix is calculate. Finally, using this camera parameter matrix, it is possible to calculate which point on the actual printed material corresponds to the pixel included in the printed material in the image, and return the data area to the original plane shape and read the data (for example, (See Patent Document 1 and Non-Patent Document 1).
JP 2000-82107 A Jun Rekimoto and Yuji Ayatsuka, CyberCode: Designing Augmented Reality Environments with Visual Tags, Designing Augmented Reality 2000,

  In the above-described conventional method, in the step of searching for a two-dimensional code from an image picked up by a digital camera or the like, an error or a search error due to region fullness due to quantization and binarization processing may occur. This error naturally affects the step of associating the four corner points of the two-dimensional code with the coordinate points on the actual printed matter.

  That is, as shown in FIG. 17, the coordinate system virtually installed on the printed matter in the image has a two-dimensional code as the origin, and the X axis 7 and the Y axis 8 are calculated by the projective transformation matrix. The X and Y coordinates in the coordinate system are expressed by multiplying the basis vectors representing the X and Y axes by scalar multiplication. At this time, the error affects the position of the origin and the directions of the X axis and the Y axis. The deviation of the origin position does not expand more than the deviation on the first image. However, in proportion to the distance between the point where the real world coordinates are to be calculated and the origin of the virtual coordinate system, the influence of the error in the orientation of the X-axis and the Y-axis, which is the basis vector for indicating the coordinates, is shown in FIG. In the image, even an error of several pixels cannot be ignored in the image. In particular, this error becomes fatal in the process of distinguishing fine points in the area where the printed matter in the image is shown and letting the user specify it.

  In order to reduce the influence of this error, there is a method of attaching a two-dimensional code to several places of printed matter or a table, but a lot of two-dimensional codes are pasted on daily use such as pamphlets and posters. Is unrealistic because the proportion of the two-dimensional code in the printed matter becomes too large.

  This invention is made | formed in view of this situation, The objective is to provide the processing technique of the code provided to the target object which can solve the said subject.

  Accordingly, in order to solve the above-mentioned problem, claim 1 is a method for processing a captured image of an object provided with a positioning marker and a data cell region including size information of the object, and images the object Searching the position of the positioning marker of the object in the captured image from the captured image, detecting a temporary reading area surrounded by the center of the positioning marker in the captured image, and each center of the positioning marker And a point corresponding to a positioning marker of an arbitrary area having the same area as the temporary reading area, and a pixel included in the temporary reading area and an area inside the arbitrary area having the same area as the temporary reading area Calculating a temporary camera parameter matrix for associating points, and correcting the object in the captured image using the temporary camera parameter matrix Generating a temporary data cell read image, and using a predetermined positioning marker shape and size and the temporary data cell read image, to be specified by the positioning marker of the object in the captured image A step of determining a position, a step of calculating a region including the entire object using a position to be specified by a positioning marker of the object in the determined captured image, and the entire object Using the temporary camera parameter matrix for points inside the area, calculating points on the object and creating a data cell read image; and detecting a data cell area in the data cell read image Extracting the size information of the object from the detected data cell region; and the actual object of the acquired object Calculating a camera parameter matrix that associates the pixels included in the region of the target object in the captured video with the points on the actual target object using the image and the specific position of the target object in the captured video; It is characterized by having.

  The second aspect of the present invention is a method for processing a captured image of a target object provided with positioning markers arranged at four corners and a data cell region including size information of the target object. The position of the positioning marker arranged at the four corners of the object in the captured image is searched from the captured image, and the temporary reading area surrounded by the centers of the four positioning markers in the captured video is detected. Using the step, the position of each center of the four positioning markers, and four square points having the same area as the temporary reading area, the pixels included in the temporary reading area are temporarily associated with the points inside the square. A step of calculating a camera parameter matrix, and using the temporary camera parameter matrix, the object in the captured image is corrected to a square and the temporary data cell read image Using the predetermined shape and size of the positioning marker and the temporary data cell read image, specifying the positions of the four corners of the object in the captured image, and the identified imaging Using the positions of the four corners of the object in the image to calculate a circumscribed rectangle including the entire object, and using the temporary camera parameter matrix for the points inside the circumscribed rectangle, Calculating a point above, creating a data cell read image, detecting a data cell region in the data cell read image, and extracting size information of an object from the detected data cell region Pixels included in the area of the object in the captured image using the step, the actual size of the acquired object and the positions of the four corner points of the object in the captured image And having a step of calculating a camera parameter matrix for associating the actual point on the object, the.

  The image processing means for processing the input captured image of the target object to calculate the camera parameter matrix and acquiring the target object content information, the captured image and the region of the target object specified by the user Of the object in the image display system, comprising: a display unit that displays an image obtained by superimposing the object content information on the captured image; and an input unit that inputs a position coordinate of the region of the object specified in the captured image A method for processing a photographed image, wherein the image processing means searches for a position in the photographed image of the positioning marker of the object from a photographed image obtained by photographing the object, and a positioning marker in the photographed video A step of detecting a temporary reading area surrounded by each center of the positioning marker, the position of each center of the positioning marker, and the same area as the temporary reading area. Calculating a temporary camera parameter matrix that associates pixels included in the temporary reading area with points in the arbitrary area having the same area as the temporary reading area, using points corresponding to positioning markers in the arbitrary area; , Using the temporary camera parameter matrix to correct an object in the captured image to generate a temporary data cell read image, a predetermined shape and size of the positioning marker, and the temporary data cell read image. Using the step of determining the position to be specified by the positioning marker of the object in the captured image, and using the position to be specified by the positioning marker of the object in the determined captured image, Calculating a region including the entire object, and the temporary point with respect to a point inside the region including the entire object. A point on the object is calculated using a mellar parameter matrix, a step of creating a data cell read image, a step of detecting a data cell region in the data cell read image, and the detected data cell region Extracting object size information and object content information, and using the actual size of the acquired object and a specific position of the object in the captured image, the region of the object in the captured image Calculating a camera parameter matrix that associates pixels included in the image with points on the actual object, and calculating a position of an area on the actual object designated by the user using the camera parameter matrix; The object content information corresponding to the position of the area on the actual object designated by the user is superimposed on the captured image of the object and transmitted to the display means. It is characterized by having.

  The image processing means for processing the input captured image to calculate the camera parameter matrix and acquiring the object content information, and the object related to the captured image and the region of the object specified by the user A display unit that displays an image in which content information is superimposed on a captured image, and an input unit that inputs the position coordinates of the region of the target specified by the user in the captured image. A processing method, wherein the image processing means searches for a position in a captured image of positioning markers arranged at four corners of the target object from a captured image obtained by capturing the target object; Detecting a temporary reading area surrounded by the centers of the four positioning markers, positions of the centers of the four positioning markers, and the temporary reading A step of calculating a temporary camera parameter matrix for associating pixels included in the temporary reading region with the points inside the square using the four points of the square having the same area as the region, and using the temporary camera parameter matrix Using the step of generating a temporary data cell read image by correcting the object in the captured image to a square, and using the predetermined shape and size of the positioning marker and the temporary data cell read image, Identifying the positions of the four corners of the object at the step, calculating the circumscribed rectangle including the entire object using the positions of the four corners of the object in the identified captured image, and the circumscribing Using the temporary camera parameter matrix for the points inside the rectangle, the points on the object are calculated to create a data cell read image. Detecting a data cell region in the data cell read image; extracting size information and target content information of the target object from the detected data cell region; and actual data of the acquired target object. A step of calculating a camera parameter matrix for associating pixels included in the region of the object in the photographed image with points on the actual object using the size and the positions of the four corner points of the object in the photographed image Calculating the position of the area on the actual object specified by the user using the camera parameter matrix; and object content information corresponding to the position of the area on the actual object specified by the user Is superimposed on the captured image of the object, and is transmitted to the display means.

  Further, a fifth aspect of the present invention provides a database for accumulating information on the size of an object and information including object content information, a search means for searching the database in response to a request from an image processing means, and an input captured image Image processing means for calculating the camera parameter matrix and acquiring the object content information stored in the database, and the captured image and the object content information relating to the region of the object specified by the user in the captured image A method for processing a captured image of an object in an image display system, comprising: a display means for displaying a superimposed image; and an input means for inputting a position coordinate of a region of the object specified by the user in the captured image, The image processing means detects a position of the positioning marker of the target object in the captured image from a captured image of the target object. A step of detecting a temporary reading area surrounded by each center of the positioning marker in the captured video, a position of each center of the positioning marker, and a positioning marker of an arbitrary area having the same area as the temporary reading area A temporary camera parameter matrix that associates a pixel included in the temporary reading area with a point in an arbitrary area having the same area as the temporary reading area, using the points corresponding to the temporary reading area; and And correcting the object in the captured image to generate a temporary data cell read image, and using the predetermined positioning marker shape and size and the temporary data cell read image, the captured image. Determining the position to be specified by the positioning marker of the object in the said, Calculating a region including the entire target object using a position to be specified by the positioning marker of the target object in the captured image, and a point inside the region including the entire target object Calculating a point on the object using a temporary camera parameter matrix, creating a data cell read image, detecting a data cell region in the data cell read image, and the detected data cell region Detecting object identification information from the object, transmitting the object identification information to the search means, size information of the object searched by the search means from the database based on the object identification information, and the object Using the step of acquiring the content information, the actual size of the acquired object and the specific position of the object in the captured image, the object in the captured image A step of calculating a camera parameter matrix for associating pixels included in the region of the object with a point on the actual object, and calculating the position of the region on the actual object specified by the user using the camera parameter matrix And the step of superimposing the object content information corresponding to the position of the area on the actual object designated by the user on the captured image of the object and transmitting it to the display means. It is characterized by that.

  A sixth aspect of the present invention relates to a database for storing information on the size of an object and information including object content information, a search means for searching the database in response to a request from an image processing means, and an input captured image. Image processing means for processing to calculate a camera parameter matrix and acquiring object content information stored in the database, and superimpose on the captured image the object content information related to the captured image and the region of the object specified by the user A method for processing a captured image of an object in an image display system, comprising: display means for displaying the image obtained; and input means for inputting a position coordinate of a region of the object specified by the user in the captured image, The image processing means is configured to display a captured image of positioning markers arranged at four corners of the target object from a captured image of the target object. A step of searching for a position, a step of detecting a temporary reading area surrounded by the centers of the four positioning markers in the captured image, a position of each center of the four positioning markers, and the same area as the temporary reading area A step of calculating a temporary camera parameter matrix that associates pixels included in the temporary reading area with the points inside the square using the four points of the square having, and using the temporary camera parameter matrix, The object in the captured image is generated using the step of generating a temporary data cell read image by correcting the target object to a square, the shape and size of a predetermined positioning marker, and the temporary data cell read image. Using the four corner positions of the object and the positions of the four corners of the object in the identified captured image Calculating a circumscribed rectangle including a body, calculating points on the object using the temporary camera parameter matrix for points inside the circumscribed rectangle, and creating a data cell read image; Detecting a data cell area in the data cell read image; detecting object identification information from the detected data cell area; transmitting the object identification information to the search means; The step of acquiring the object size information and the object content information searched by the search means from the database based on the object identification information, the actual size of the acquired object and the four corners of the object in the captured video The camera parameter matrix that correlates the pixels included in the area of the object in the captured image with the points on the actual object is calculated using the position of the point Using the camera parameter matrix, calculating a position of an area on the actual object specified by the user, and an object corresponding to the position of the area on the actual object specified by the user A step of superimposing the content information on a captured image of the object and transmitting it to the display means.

  The image processing means for processing the input photographed image to calculate the camera parameter matrix and acquiring the object content information, and the object related to the photographed image and the region of the object designated by the user In the image display system, comprising: a display unit that displays an image in which content information is superimposed on a captured image; and an input unit that inputs a position coordinate of a region of an object specified by the user in the captured image. Means for searching for a position of the positioning marker of the object in the captured image from the captured image of the object; and means for detecting a temporary reading area surrounded by each center of the positioning marker in the captured image; Using the position of each center of the positioning marker and the point corresponding to the positioning marker in an arbitrary area having the same area as the temporary reading area Means for calculating a temporary camera parameter matrix for associating a pixel included in the temporary reading area with a point in an arbitrary area having the same area as the temporary reading area; and using the temporary camera parameter matrix, an object in the captured image Using a means for correcting an object to generate a temporary data cell read image, a predetermined shape and size of a positioning marker, and the temporary data cell read image, a positioning marker for an object in the captured image Means for determining a position to be specified; means for calculating a region including the entire object using a position to be specified by a positioning marker of the object in the determined captured image; and Using the temporary camera parameter matrix for the points inside the entire area, calculate the points on the object and read the data cells. Means for creating a captured image; means for detecting a data cell area in the data cell read image; means for extracting object size information and object content information from the detected data cell area; Using the actual size of the target object and the specific position of the target object in the captured video, a camera parameter matrix that associates the pixels included in the target area in the captured video with the points on the actual target object. Means for calculating, using the camera parameter matrix, means for calculating a position of an area on the actual object specified by the user, and an object corresponding to the position of the area on the actual object specified by the user And means for superimposing the object content information on the captured image of the object and transmitting it to the display means.

  Further, an image processing unit that processes an input captured image to calculate a camera parameter matrix and acquire object content information, and an object related to the captured image and an area of the object specified by the user In the image display system, comprising: a display unit that displays an image in which content information is superimposed on a captured image; and an input unit that inputs a position coordinate of a region of an object specified by the user in the captured image. Means for searching the position of the positioning markers arranged at the four corners of the target object in the captured image from the captured image of the target object, and each center of the four positioning markers in the captured image. Means for detecting a temporary reading area, positions of the centers of the four positioning markers, and four square points having the same area as the temporary reading area Means for calculating a temporary camera parameter matrix for associating the pixels included in the temporary reading area with the points inside the square, and using the temporary camera parameter matrix, the object in the captured image is corrected to a square and temporarily Means for generating a data cell read image; means for specifying the positions of four corners of the object in the captured image using a predetermined shape and size of a positioning marker and the temporary data cell read image; Means for calculating a circumscribed rectangle including the entire object using the positions of the four corners of the object in the identified captured image, and the temporary camera parameter matrix for the points inside the circumscribed rectangle. And a means for calculating a point on the object and creating a data cell read image, and a method for detecting a data cell region in the data cell read image. Means for extracting object size information and object content information from the detected data cell area, the actual size of the acquired object, and the positions of the four corner points of the object in the captured image; Means for calculating a camera parameter matrix for associating pixels included in the region of the object in the captured image with a point on the actual object, and using the camera parameter matrix, the actual parameter specified by the user The means for calculating the position of the area on the object and the object content information corresponding to the position of the area on the actual object specified by the user are superimposed on the captured image of the object and transmitted to the display means. And means for performing.

  Further, a ninth aspect of the present invention provides a database for accumulating information on the size of an object and information including object content information, a search means for searching the database in response to a request from an image processing means, and an input captured image Image processing means for calculating the camera parameter matrix and acquiring the object content information stored in the database, and the captured image and the object content information relating to the region of the object specified by the user in the captured image In the image display system having display means for displaying the superimposed image and input means for inputting the position coordinates of the region of the object designated by the user in the photographed image, the image processing means photographs the object. Means for retrieving a position of the positioning marker of the object in the captured image from the captured image, and a position in the captured video Using a means for detecting a temporary reading region surrounded by each center of the marker, a position of each center of the positioning marker, and a point corresponding to the positioning marker of an arbitrary region having the same area as the temporary reading region, Means for calculating a temporary camera parameter matrix for associating a pixel included in the temporary reading area with a point in an arbitrary area having the same area as the temporary reading area; and using the temporary camera parameter matrix, an object in the captured image Using a means for correcting an object to generate a temporary data cell read image, a predetermined shape and size of a positioning marker, and the temporary data cell read image, a positioning marker for an object in the captured image It is specified by means for determining a position to be specified and a positioning marker of the object in the determined captured image. Means for calculating an area including the entire object using the power position, and using the temporary camera parameter matrix for a point inside the area including the entire object, and calculating a point on the object Means for calculating and creating a data cell read image; means for detecting a data cell area in the data cell read image; means for detecting object identification information from the detected data cell area; and Means for transmitting information to the search means, means for acquiring size information and object content information of the object searched by the search means from the database based on the object identification information, and actuality of the acquired object A camera parameter matrix that associates pixels included in the region of the object in the captured image with points on the actual object using the size of the object and a specific position of the object in the captured image Means for calculating, using the camera parameter matrix, means for calculating a position of an area on the actual object specified by the user, and an object corresponding to the position of the area on the actual object specified by the user And means for superimposing the object content information on the captured image of the object and transmitting it to the display means.

  Further, the present invention relates to a database for accumulating information on the size of an object and information including object content information, a search means for searching the database in response to a request from an image processing means, and an input captured image Image processing means for calculating the camera parameter matrix and acquiring the object content information stored in the database, and the captured image and the object content information relating to the region of the object specified by the user in the captured image In the image display system having display means for displaying the superimposed image and input means for inputting the position coordinates of the region of the object designated by the user in the photographed image, the image processing means photographs the object. Means for retrieving positions in the captured image of positioning markers arranged at the four corners of the object from the captured image, Means for detecting a temporary reading area surrounded by the centers of the four positioning markers in the captured image, positions of the centers of the four positioning markers, and four square points having the same area as the temporary reading area A means for calculating a temporary camera parameter matrix for associating the pixels included in the temporary reading area with the points in the square, and using the temporary camera parameter matrix to correct the object in the captured image to a square. Means for generating a temporary data cell read image, and means for specifying the positions of the four corners of the object in the captured image using a predetermined shape and size of a positioning marker and the temporary data cell read image And means for calculating a circumscribed rectangle including the entire object using the positions of the four corners of the object in the specified captured image, and the circumscribed length Means for calculating a point on the object by using the temporary camera parameter matrix for a point inside the shape, and generating a data cell read image; and detecting a data cell region in the data cell read image Means for detecting object identification information from the detected data cell area, means for transmitting the object identification information to the search means, and search means searched from the database based on the object identification information Using the means for acquiring the size information and the object content information of the object, the actual size of the acquired object and the positions of the four corner points of the object in the captured image, the object in the captured image Means for calculating a camera parameter matrix for associating pixels contained in the object region with points on the actual object, and the actual object specified by the user using the camera parameter matrix Means for calculating the position of the upper area, and means for superimposing the object content information corresponding to the position of the area on the actual object designated by the user on the captured image of the object, and transmitting this to the display means It is characterized by having.

  The eleventh aspect is characterized in that the method or system according to any one of the first to tenth aspects described above is described in a computer program and can be executed.

  A twelfth aspect is characterized by recording a program that can execute the method or system according to any one of the first to tenth aspects of the present invention by a computer.

  The present invention provides a method for processing a code printed on an object that enables estimation of coordinates with less error on the object while maintaining the area ratio of the one conventional two-dimensional code in the object. It is. The present invention uses a two-dimensional code as an object as shown in FIG. 1 in order to reduce the influence of an error that occurs when searching for a data cell area in which a positioning marker or information is embedded in an image. For example, four positioning markers 1 and one data cell area 2 are divided into a total of five areas. Using these four positioning markers, a projective transformation matrix is calculated in the same manner as the two-dimensional code, and a coordinate system on the object in the image is estimated. Since the object is surrounded by four positioning markers, the X axis and the Y axis of the coordinate system estimated on the object are sides constituting the object. That is, all the arbitrary coordinates on the estimated object are present inside the four corners of the positioning marker. In the conventional method of printing one positioning marker at one location, the error on the image is slightly increased. However, in the configuration of the positioning marker in the present invention, the influence of the error that increases as the distance from the positioning marker increases. Can be reduced. Note that the number of positioning markers is not limited to four, and may be arranged in a polygonal shape other than a quadrangular shape, or may be arranged in four or more points.

  An error when searching for a positioning marker from an image obtained by photographing an object on which a two-dimensional code is printed affects the virtual coordinate estimation of the object in the image. In the conventional method, this error is enlarged in proportion to the distance between the points to be associated with the two-dimensional code in the step of corresponding any point included in the object on the screen to the point on the actual object. There's a problem.

   Therefore, according to the present invention, by positioning the positioning markers at, for example, the four corners of the object, it is possible to reduce the influence of an error that increases as the distance from the positioning marker increases, and the pixels included in the object in the image The point on the actual object corresponding to can be calculated more accurately

  Embodiments in the present invention will be described below.

(Configuration example of information linkage device)
FIG. 2 shows a schematic diagram of the information reading device 10 and the printed material database device 20 from the printed material in the present embodiment. This embodiment is a system that links information with a printed material, estimates a coordinate system of the printed material from a code, and superimposes and displays information related to an appropriate position of the printed material in an image.

(Information reading device 10)
The information reading device 10 includes a photographing unit 11 for photographing a printed matter, an input unit 14 for operating the information reading device 10, a display unit 15 for displaying a photographed image and a reading result, a photographed image, and an image for processing linkage information. The processing unit 12 and the network unit 13 for communicating with the printed material database device 20 are configured.

(Printed database device 20)
The printed material database device 20 includes a network unit 22, a search unit 21, and a database unit 23 for communicating with the information reading device 10.

  The database means 23 is composed of a data table 30 as shown in FIG. The data table 30 includes a printed matter identification number 31 for identifying each printed matter, data 32 of actual size of the printed matter, and printed matter content information 33.

  The printed matter content information 33 includes an area identification number 34, area data 35, and area cooperation information 36.

  The area identification number 34 is information for identifying the contents on the printed material.

  The area data 35 represents the area of the content on the printed material with a geometrical shape such as a dot array, a circle, or a rectangle.

  The area cooperation information 36 is configured by information superimposed on each area, URL information, and the like.

(Usage of this device)
The user photographs the printed matter with the photographing means 11. An image photographed by the photographing means is sent to the image processing means 12 as a photographed image.

  The image processing means 12 processes the captured image, calculates a camera parameter matrix, and obtains printed matter content information. This process will be described later.

  Next, the user designates the pixel of the printed region in the image using the input unit 14 while viewing the captured image displayed on the display unit 15.

  The image processing means 12 calculates actual coordinates on the actual printed matter corresponding to the input pixel coordinates using the camera parameter matrix. In this camera parameter calculation method, it is determined which area included in the printed content information corresponds to the real coordinate, and the corresponding area-related information is acquired. This area-related information is superimposed on the captured image and displayed on the display unit 15. The user can view this result from the display means 15 as shown in FIG.

(Printed matter)
The characteristics of the code printed on the printed material will be described. In the present embodiment, as shown in FIG. 1, the four positioning markers 1 are printed at the four corners of the printed material, and the data cell area 2 is printed directly below the upper left positioning marker.

(Positioning marker)
In this embodiment, a monochrome QR code positioning marker is used as the positioning marker. As shown in FIG. 1, a margin 3 having a width 1/7 times the size of the positioning marker is printed around the positioning marker (see, for example, Japanese Patent Laid-Open No. 10-208001).

(Data cell area)
The data cell area 2 is composed of cells 41 representing bits. As shown in FIG. 5, it is composed of a two-dimensional array of white (bright) and black (dark) square cells. The size of this cell is 2/7 times the size of the positioning force.

  The cell array 42 of the present embodiment is composed of 11 × 4 cells. The leftmost column is a pattern bit for reading a bit pattern, and white and black cells are alternately arranged. The uppermost cell 44 in the second column from the left end is a pattern bit for measuring the width. The cells other than the topmost cell in the second column from the left end, the right end column, and the second column from the right end represent data. In this embodiment, 32-bit data can be held. This data cell area can be enlarged or reduced according to the amount of data.

(Processing method for images taken of printed materials)
Processing until the camera parameter matrix, the printed material content information, and the actual size information of the printed material are acquired for the captured image obtained by capturing the printed material in the image processing unit 12 will be described with reference to FIGS.

(Image preprocessing and gray scale)
First, the photographed image transmitted from the photographing means 12 is converted into a grayscale image composed of grayscale (S101-1). In the conversion, the luminance value is extracted in 256 steps using the YUV color conversion method, and converted and stored as a gray scale image.

  Next, in order to make it easy to detect the positioning marker for the gray scale image, processing by a median filter is performed to remove noise in the gray scale image.

(Image preprocessing and binarization)
Since the grayscale image is not suitable for image processing as it is, binarization is performed (S101-2). First, the grayscale image is divided into appropriate areas, and a threshold value that maximizes the variance of the binarized areas 0 and 1 is calculated for each area. Each area is binarized using the threshold value. Thereafter, in order to eliminate small noise appearing in the binarized image, noise removal by expansion / contraction is performed to obtain a binarized image.

(Search for position marker)
Next, a positioning marker is searched from the binarized image (S102).

  A run length scan is performed on the lengths of white and black pixels from the upper left to the lower right of the binarized image. From the obtained run length scan results for each line, a corresponding portion where the length of black: white: black: white: black is 1: 1: 3: 1: 1 is searched, and this ratio becomes 3. The coordinates of the image of the center part are stored as the marker corresponding coordinates. When the positioning marker is rotating in the binarized image, the corresponding portion of 1: 1: 3: 1: 1 may not be detected. Considering this case, in order to ensure that an image that has been affected by the rotation is also read, a run-length scan is performed while rotating the binarized image. Next, using the stored marker corresponding coordinates, as shown in FIG. 8, a marker corresponding image for recording only the search result of the positioning marker is created. After the search is completed, the image corresponding to the marker is accidentally 1: 1: 3: 1: 1, and noise removal is performed using expansion / contraction to delete a portion appearing as noise (S102-1). ~ 5).

(Determination of marker coordinate values)
Labeling is performed on the marker corresponding image obtained as a result of the previous processing (S102-6). At this time, if there are five or more regions that are considered to be positioning markers, which should originally have only four, using the result of labeling, two conditions, the circumference of the region and the size of the circumscribed rectangle of the region, Narrow the area. If the circumference is too small or too large, it is determined that it is not a positioning marker, and if the circumscribed rectangle is too thin, it is determined that it is not a positioning marker (S102-7).

  Finally, the centroids of the four areas 150 estimated to be the respective positioning markers are obtained, and the coordinates are determined to be the coordinates of the center of the positioning marker in the captured image (S102-8).

(Image correction to square)
The four points obtained in the preceding process are rearranged so as to correspond to the four points of the square. Based on the camera parameter matrix derivation method shown in FIG. 16, the positions of the four positioning markers in the captured image are associated with four points of arbitrary size squares as shown in FIG. 9 (S103-1), A temporary camera parameter matrix ignoring the aspect ratio of the printed material is obtained (SS103-2).

  Next, using this temporary camera parameter matrix, the printed material in the captured image is corrected to a square (S103-3 to 4). This temporary camera parameter matrix is used to read the data cell area.

  The correction is performed on all coordinate points in the circumscribed rectangle 200 of the four points 201 obtained by the previous processing as shown in FIG. 10 using the temporary camera parameter matrix. Here, a temporary data cell read image is generated, and the corrected result is input to the temporary data cell read image. At this time, since the points of the circumscribed rectangle 200 of the original photographed image are discretely arranged, it is not possible to cover all the coordinates on the printed matter in the real world. Therefore, the temporary data cell read image created here is an image in which quantization noise is mixed. Here, enlargement / contraction processing is performed to remove this small quantization noise.

  As shown in FIG. 11, the completed temporary data cell read image 210 treats the four centroids (centers) of the respective positioning markers as four corner points. Therefore, this image is used to actually print on the printed matter. Cannot read existing data cells or process print contents. Therefore, the size of the square 211 inside the positioning marker is estimated from the temporary data cell read image using the fact that the positioning marker 211 has a known shape (square).

  The center of the positioning marker is translated by the size of the square in the vertical and horizontal directions, and as shown in FIG. 12, the coordinates 221 of the corners of the true printed matter on the corrected image are calculated.

  The coordinates 221 are converted again into coordinates on the photographing screen by the temporary camera parameter matrix, and the circumscribed rectangle 220 including the entire printed matter is calculated. With respect to the points inside the circumscribed rectangle 220, the points on the printed material are calculated using the temporary camera parameter matrix, and the data cell read image 230 of the printed material is obtained again as shown in FIG. Of course, this image 230 includes the entire printed matter.

(Pattern bit reading and orientation)
Next, the cell forming the pattern bit of the data cell area under the positioning marker at the upper left of the data cell read image 230 shown in FIG. 13 is checked (S105-2). If the pattern bit is not detected at this time, it is considered that the data cell read image is rotated and output when the four positioning markers are rearranged. Therefore, the image is rotated by 90 degrees (S105-4), and the pattern is read. Check bits. The order of positions in the image of the positioning markers at the four corners is simultaneously rotated by the angle at which the data cell read image is rotated. This is performed four times, and if no pattern bit is detected once, it is assumed that there is no data cell area in the data cell reading firm image, and reading ends.

(Reading information from the data cell area)
The array of cells in the data cell area is searched using the pattern bits of the data cell read image. The middle point of the boundary between the white and black cells of the pattern bit is used as the Y coordinate when reading in the vertical direction. In addition, since the cell at the top of the second column from the left end is always white and the cell at the top left included in the pattern bit is always black, the horizontal width of the cell at the top left is measured from the image. be able to. Furthermore, since the X coordinate of the center of gravity of the rightmost cell is configured to be equal to the X coordinate of the outer frame of the positioning marker, the X and Y coordinates of the center of gravity of each cell can be calculated. The ratio of white or black pixels contained in the pixels around the center of gravity is examined to determine whether the cell is 0 or 1. As a result, a total of 32 bits of printed matter identification information is acquired from the data cell area (S105-6).

(Acquisition of information about printed matter)
Next, the printed matter identification information acquired by a series of processes in the image processing means 12 is sent to the network means 13. The network means 13 sends the printed matter identification information to the network means 22 in the separately provided printed matter information database apparatus 20. Next, the printed matter identification information is transmitted from the network means 22 to the search means 21. The search means 21 acquires the printed matter content information and the actual size information of the printed matter from the database means 23 using the printed matter identification information. Next, the search unit 21 transmits the printed matter content information and the actual size information of the printed matter to the information reading device 10 through the network unit 22, and the image processing unit 12 acquires the information (S106-1).

(Camera parameter matrix)
The image processing means 12 calculates a temporary camera parameter matrix that takes into account the aspect ratio of the printed material, using information on the actual size of the printed material obtained by the preceding process. In the temporary data cell read image creation step, the camera parameter matrix is calculated assuming that the printed material is temporarily square, but here the camera parameter matrix is calculated using the actual size of the printed material with an appropriate aspect ratio. Is calculated (S106-3). As a result, it is possible to calculate the coordinates on the actual printed matter corresponding to any pixel included in the printed matter region in the image.

(Modification)
A modification of the present embodiment will be described below.

(Modification 1)
In the above embodiment, the actual size information and the printed content information of the printed material are acquired by using the network means 13 and the printed material database device 20 separately provided outside. However, the data cell area of the printed material is obtained. It is also possible to provide information on the actual size of printed matter and printed matter content information without the network means 13 and the printed matter database device 20 provided in the information reading device.

(Modification 2)
In this embodiment, the QR code is used for the positioning marker, but a general positioning marker having a specific geometric shape such as a square or a circle can also be used instead of the QR code. Further, the color of the data cell area and the positioning marker can be configured by using not only white and black colors but also other colors.

(Modification 3)
As shown in FIG. 2, the reading device 10 in the present embodiment has a built-in means 11 for picking up an image of a printed matter. It is also possible to construct the image as shown in FIG. 14 through storage medium means or the like.

(Modification 4)
The printed material positioning markers of the present embodiment can be arranged not only at the four corners but also in a shape other than a quadrangle on the printed material, and can also be arranged at four or more points.

(Modification 5)
The data cell area of this embodiment can be arranged not only at one place but also at two or more places on the printed material.

(Modification 6)
The object in the present embodiment is not limited to a paper medium in which a predetermined code is printed. In addition, a target code is displayed on a general display means such as a CRT or a liquid crystal display as shown in FIG. Application is also possible.

  The present invention can be realized by using a computer by configuring a part or all of the processing functions of the system shown in FIGS. 2 and 14 as a program, or the processing procedures shown in FIGS. 1 and 3 to 15. Can be configured as a program and executed by a computer. In addition, a computer-readable recording medium such as a flexible disk, MO, ROM, or memory card can be used to store a program for realizing the processing function of each unit by the computer or a program for causing the computer to execute the processing procedure. It can be recorded on a CD, a DVD, a removable disk, etc., stored, provided, and distributed via a communication network such as the Internet.

The figure which shows an example which attaches the positioning marker to a target object. The schematic block diagram of the system which consists of an information reading apparatus and a printed matter database apparatus. The schematic block diagram of a data table. Explanatory drawing which shows an example of the image which superimposed the information regarding a target object in a display means. Explanatory drawing which shows an example of a structure of a data cell area | region. The processing flowchart in an image processing means. The processing flowchart in an image processing means. Explanatory drawing which shows an example of a marker applicable image. Explanatory drawing which shows an example of matching with the position in the picked-up image of four positioning markers, and four points of squares of arbitrary sizes. Explanatory drawing which shows the circumscribed rectangle of the center of four positioning markers. Explanatory drawing which shows a temporary data cell reading image. Explanatory drawing which shows the circumscribed rectangle of the coordinate of the corner of a true printed matter. Explanatory drawing which shows a data cell read image. The schematic block diagram of the modification of a reader. Explanatory drawing which shows the example which applied this invention to CRT which is a general display means. Explanatory drawing which calculates the geometric relationship of the point on a screen, and the coordinate on actual printed matter. Explanatory drawing which shows the coordinate system virtually installed on the printed matter in an image. Explanatory drawing which shows the deviation | shift between the coordinate of a real world and the coordinate of a virtual coordinate system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Positioning marker 2 ... Data cell area 3 ... Margin 10 ... Cooperation information reading apparatus 11 ... Imaging | photography means 12 ... Image processing means 13 ... Network means 14 ... Input means 15 ... Display means 20 ... Printed material database apparatus 21 ... Search means 22 ... Network means 23 ... Database means 30 ... Data table 31 ... Printed material identification number 32 ... Actual data size 33 ... Printed material content information 34 ... Area identification number 35 ... Area data 36 ... Area linkage information 41 ... Cell representing bit 42 ... Cell array 44 ... Pattern bits for measuring width

Claims (12)

A method for processing a captured image of an object provided with a positioning marker and a data cell region including size information of the object,
Searching for a position in a captured image of a positioning marker of the target object from a captured image obtained by capturing the target object;
Detecting a temporary reading area surrounded by each center of the positioning marker in the captured video;
Using the position of each center of the positioning marker and the point corresponding to the positioning marker of an arbitrary area having the same area as the temporary reading area, the pixels included in the temporary reading area and the same area as the temporary reading area Calculating a temporary camera parameter matrix that correlates points within an arbitrary region;
Using the temporary camera parameter matrix to correct an object in the captured image to generate a temporary data cell read image;
Determining a position to be specified by the positioning marker of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Calculating a region including the entire object using a position to be specified by the positioning marker of the object in the determined captured image;
Using the temporary camera parameter matrix for points inside the region including the entire object, calculating points on the object, and creating a data cell read image;
Detecting a data cell region in the data cell read image;
Extracting size information of an object from the detected data cell region;
Camera parameters for associating pixels included in the region of the object in the captured image with points on the actual object using the actual size of the acquired object and a specific position of the object in the captured image Calculating a matrix, and a method of processing a captured image of an object. A method for processing a captured image of an object provided with positioning markers arranged at four corners and a data cell region including size information of the object,
Searching for positions in a captured image of positioning markers arranged at four corners of the target object from among captured images of the target object; and
Detecting a temporary reading area surrounded by each of the four positioning markers in the captured video;
Temporary camera parameter matrix for associating the pixels included in the temporary reading area with the points inside the square using the positions of the centers of the four positioning markers and four square points having the same area as the temporary reading area Calculating steps,
Using the temporary camera parameter matrix, correcting the object in the captured image to a square to generate a temporary data cell read image;
Identifying positions of four corners of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Calculating a circumscribed rectangle including the entire object using the positions of the four corners of the object in the identified captured image;
Using the temporary camera parameter matrix for the points inside the circumscribed rectangle, calculating points on the object, creating a data cell read image;
Detecting a data cell region in the data cell read image;
Extracting size information of an object from the detected data cell region;
Using the actual size of the acquired object and the positions of the four corner points of the object in the captured image, the pixels included in the area of the object in the captured image and the points on the actual object are determined. And a step of calculating a camera parameter matrix to be associated. Image processing means for processing a captured image of an input object to calculate a camera parameter matrix and acquire object content information;
Display means for displaying a captured image and an image obtained by superimposing the object content information on the object area designated by the user on the captured image;
A method for processing a captured image of an object in an image display system, comprising: input means for inputting a position coordinate of a region of the object specified in a captured image by a user;
The image processing means searches for a position in the captured image of the positioning marker of the object from a captured image obtained by capturing the object;
Detecting a temporary reading area surrounded by each center of the positioning marker in the captured video;
Using the position of each center of the positioning marker and the point corresponding to the positioning marker of an arbitrary area having the same area as the temporary reading area, the pixels included in the temporary reading area and the same area as the temporary reading area Calculating a temporary camera parameter matrix that correlates points within an arbitrary region;
Using the temporary camera parameter matrix to correct an object in the captured image to generate a temporary data cell read image;
Determining a position to be specified by the positioning marker of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Calculating a region including the entire object using a position to be specified by the positioning marker of the object in the determined captured image;
Using the temporary camera parameter matrix for points inside the region including the entire object, calculating points on the object, and creating a data cell read image;
Detecting a data cell region in the data cell read image;
Extracting object size information and object content information from the detected data cell region;
Camera parameters for associating pixels included in the region of the object in the captured image with points on the actual object using the actual size of the acquired object and a specific position of the object in the captured image Calculating a matrix;
Using the camera parameter matrix to calculate the position of the area on the actual object specified by the user;
Superimposing the object content information corresponding to the position of the area on the actual object designated by the user on the captured image of the object, and transmitting the information to the display means. Of processing a captured image of a target object. Image processing means for processing an input captured image to calculate a camera parameter matrix and acquire object content information;
Display means for displaying a captured image and an image obtained by superimposing the object content information on the object area designated by the user on the captured image;
A method for processing a captured image of an object in an image display system, comprising: input means for inputting a position coordinate of a region of the object specified in a captured image by a user;
The image processing means searches for positions in a captured image of positioning markers arranged at four corners of the target object from among captured images of the target object;
Detecting a temporary reading area surrounded by each of the four positioning markers in the captured video;
Temporary camera parameter matrix for associating the pixels included in the temporary reading area with the points inside the square using the positions of the centers of the four positioning markers and four square points having the same area as the temporary reading area Calculating steps,
Using the temporary camera parameter matrix, correcting the object in the captured image to a square to generate a temporary data cell read image;
Identifying positions of four corners of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Calculating a circumscribed rectangle including the entire object using the positions of the four corners of the object in the identified captured image;
Using the temporary camera parameter matrix for the points inside the circumscribed rectangle, calculating points on the object, creating a data cell read image;
Detecting a data cell region in the data cell read image;
Extracting object size information and object content information from the detected data cell region;
Using the actual size of the acquired object and the positions of the four corner points of the object in the captured image, the pixels included in the area of the object in the captured image and the points on the actual object are determined. Calculating a camera parameter matrix to be associated;
Using the camera parameter matrix to calculate the position of the area on the actual object specified by the user;
The target content information corresponding to the position of the area on the actual target specified by the user is superimposed on the captured image of the target and transmitted to the display means. Method for processing captured images. A database for storing information about the size of the object and information about the object content;
Search means for searching the database in response to a request from the image processing means;
Image processing means for processing an input captured image to calculate a camera parameter matrix and acquiring object content information stored in the database;
Display means for displaying a captured image and an image obtained by superimposing the object content information on the object area designated by the user on the captured image;
A method for processing a captured image of an object in an image display system, comprising: input means for inputting a position coordinate of a region of the object specified in a captured image by a user;
The image processing means searches for a position in the captured image of the positioning marker of the object from a captured image obtained by capturing the object;
Detecting a temporary reading area surrounded by each center of the positioning marker in the captured video;
Using the position of each center of the positioning marker and the point corresponding to the positioning marker of an arbitrary area having the same area as the temporary reading area, the pixels included in the temporary reading area and the same area as the temporary reading area Calculating a temporary camera parameter matrix that correlates points within an arbitrary region;
Using the temporary camera parameter matrix to correct an object in the captured image to generate a temporary data cell read image;
Determining a position to be specified by the positioning marker of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Calculating a region including the entire object using a position to be specified by the positioning marker of the object in the determined captured image;
Using the temporary camera parameter matrix for points inside the region including the entire object, calculating points on the object, and creating a data cell read image;
Detecting a data cell region in the data cell read image;
Detecting object identification information from the detected data cell region;
Transmitting the object identification information to the search means;
Obtaining size information and object content information of the object searched by the search means from the database based on the object identification information;
Camera parameters for associating pixels included in the region of the object in the captured image with points on the actual object using the actual size of the acquired object and a specific position of the object in the captured image Calculating a matrix;
Using the camera parameter matrix to calculate the position of the area on the actual object specified by the user;
Superimposing the object content information corresponding to the position of the area on the actual object designated by the user on the captured image of the object, and transmitting the information to the display means. Of processing a captured image of a target object. A database for storing information about the size of the object and information about the object content;
Search means for searching the database in response to a request from the image processing means;
Image processing means for processing an input captured image to calculate a camera parameter matrix and acquiring object content information stored in the database;
Display means for displaying a captured image and an image obtained by superimposing the object content information on the object area designated by the user on the captured image;
A method for processing a captured image of an object in an image display system, comprising: input means for inputting a position coordinate of a region of the object specified in a captured image by a user;
The image processing means searches for positions in a captured image of positioning markers arranged at four corners of the target object from among captured images of the target object;
Detecting a temporary reading area surrounded by each of the four positioning markers in the captured video;
Temporary camera parameter matrix for associating the pixels included in the temporary reading area with the points inside the square using the positions of the centers of the four positioning markers and four square points having the same area as the temporary reading area Calculating steps,
Using the temporary camera parameter matrix, correcting the object in the captured image to a square to generate a temporary data cell read image;
Identifying positions of four corners of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Calculating a circumscribed rectangle including the entire object using the positions of the four corners of the object in the identified captured image;
Using the temporary camera parameter matrix for the points inside the circumscribed rectangle, calculating points on the object, creating a data cell read image;
Detecting a data cell region in the data cell read image;
Detecting object identification information from the detected data cell region;
Transmitting the object identification information to the search means;
Obtaining size information and object content information of the object searched by the search means from the database based on the object identification information;
Using the actual size of the acquired object and the positions of the four corner points of the object in the captured image, the pixels included in the area of the object in the captured image and the points on the actual object are determined. Calculating a camera parameter matrix to be associated;
Using the camera parameter matrix to calculate the position of the area on the actual object specified by the user;
Superimposing the object content information corresponding to the position of the area on the actual object designated by the user on the captured image of the object, and transmitting the information to the display means. Of processing a captured image of a target object. Image processing means for processing an input captured image to calculate a camera parameter matrix and acquire object content information;
Display means for displaying a captured image and an image obtained by superimposing the object content information on the object area designated by the user on the captured image;
In an image display system having an input means for inputting a position coordinate of a region of an object specified by a user in a captured image,
The image processing means is means for searching for a position of a positioning marker of the object in a photographed image from a photographed image obtained by photographing the object;
Means for detecting a temporary reading area surrounded by each center of the positioning marker in the captured video;
Using the position of each center of the positioning marker and the point corresponding to the positioning marker of an arbitrary area having the same area as the temporary reading area, the pixels included in the temporary reading area and the same area as the temporary reading area Means for calculating a temporary camera parameter matrix that correlates points within an arbitrary region;
Means for correcting a target in a captured image and generating a temporary data cell read image using the temporary camera parameter matrix;
Means for determining a position to be specified by the positioning marker of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Means for calculating a region including the entire object using a position to be specified by the positioning marker of the object in the determined captured image;
Means for calculating a point on the object using the temporary camera parameter matrix for a point inside the region including the entire object, and creating a data cell read image;
Means for detecting a data cell region in the data cell read image;
Means for extracting object size information and object content information from the detected data cell region;
Camera parameters for associating pixels included in the region of the object in the captured image with points on the actual object using the actual size of the acquired object and a specific position of the object in the captured image Means for calculating a matrix;
Means for calculating the position of the region on the actual object specified by the user using the camera parameter matrix;
And means for superimposing the object content information corresponding to the position of the area on the actual object designated by the user on the photographed image of the object and transmitting it to the display means. Image display system. Image processing means for processing an input captured image to calculate a camera parameter matrix and acquire object content information;
Display means for displaying a captured image and an image obtained by superimposing the object content information on the object area designated by the user on the captured image;
In an image display system having an input means for inputting a position coordinate of a region of an object specified by a user in a captured image,
The image processing means is means for searching positions in the captured image of positioning markers arranged at four corners of the object from among captured images obtained by capturing the object;
Means for detecting a temporary reading area surrounded by each center of the four positioning markers in the captured video;
Temporary camera parameter matrix for associating the pixels included in the temporary reading area with the points inside the square using the positions of the centers of the four positioning markers and four square points having the same area as the temporary reading area Means for calculating
Means for correcting the object in the captured image to a square using the temporary camera parameter matrix to generate a temporary data cell read image;
Means for specifying the positions of the four corners of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Means for calculating a circumscribed rectangle including the entire object using the positions of the four corners of the object in the identified captured image;
Means for calculating a point on the object using the temporary camera parameter matrix for the point inside the circumscribed rectangle, and creating a data cell read image;
Means for detecting a data cell region in the data cell read image;
Means for extracting object size information and object content information from the detected data cell region;
Using the actual size of the acquired object and the positions of the four corner points of the object in the captured image, the pixels included in the area of the object in the captured image and the points on the actual object are determined. Means for calculating a camera parameter matrix to be associated;
Means for calculating the position of the region on the actual object specified by the user using the camera parameter matrix;
An image display comprising: means for superimposing object content information corresponding to a position of an area on an actual object designated by the user on a captured image of the object and transmitting the superimposed image to a display means; system. A database for storing information about the size of the object and information about the object content;
Search means for searching the database in response to a request from the image processing means;
Image processing means for processing an input captured image to calculate a camera parameter matrix and acquiring object content information stored in the database;
Display means for displaying a captured image and an image obtained by superimposing the object content information on the object area designated by the user on the captured image;
In an image display system having an input means for inputting a position coordinate of a region of an object specified by a user in a captured image,
The image processing means is means for searching for a position of a positioning marker of the object in a photographed image from a photographed image obtained by photographing the object;
Means for detecting a temporary reading area surrounded by each center of the positioning marker in the captured video;
Using the position of each center of the positioning marker and the point corresponding to the positioning marker of an arbitrary area having the same area as the temporary reading area, the pixels included in the temporary reading area and the same area as the temporary reading area Means for calculating a temporary camera parameter matrix that correlates points within an arbitrary region;
Means for correcting a target in a captured image and generating a temporary data cell read image using the temporary camera parameter matrix;
Means for determining a position to be specified by the positioning marker of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Means for calculating a region including the entire object using a position to be specified by the positioning marker of the object in the determined captured image;
Means for calculating a point on the object using the temporary camera parameter matrix for a point inside the region including the entire object, and creating a data cell read image;
Means for detecting a data cell region in the data cell read image;
Means for detecting object identification information from the detected data cell region;
Means for transmitting the object identification information to the search means;
Means for obtaining size information and object content information of the object searched from the database by the search means based on the object identification information;
Camera parameters for associating pixels included in the region of the object in the captured image with points on the actual object using the actual size of the acquired object and a specific position of the object in the captured image Means for calculating a matrix;
Means for calculating the position of the region on the actual object specified by the user using the camera parameter matrix;
And means for superimposing the object content information corresponding to the position of the area on the actual object designated by the user on the photographed image of the object and transmitting it to the display means. Image display system. A database for storing information about the size of the object and information about the object content;
Search means for searching the database in response to a request from the image processing means;
Image processing means for processing an input captured image to calculate a camera parameter matrix and acquiring object content information stored in the database;
Display means for displaying a captured image and an image obtained by superimposing the object content information on the object area designated by the user on the captured image;
In an image display system having an input means for inputting a position coordinate of a region of an object specified by a user in a captured image,
The image processing means is means for searching positions in the captured image of positioning markers arranged at four corners of the object from among captured images obtained by capturing the object;
Means for detecting a temporary reading area surrounded by each center of the four positioning markers in the captured video;
Temporary camera parameter matrix for associating the pixels included in the temporary reading area with the points inside the square using the positions of the centers of the four positioning markers and four square points having the same area as the temporary reading area Means for calculating
Means for correcting the object in the captured image to a square using the temporary camera parameter matrix to generate a temporary data cell read image;
Means for specifying the positions of the four corners of the object in the captured image using a predetermined shape and size of the positioning marker and the temporary data cell read image;
Means for calculating a circumscribed rectangle including the entire object using the positions of the four corners of the object in the identified captured image;
Means for calculating a point on the object using the temporary camera parameter matrix for the point inside the circumscribed rectangle, and creating a data cell read image;
Means for detecting a data cell region in the data cell read image;
Means for detecting object identification information from the detected data cell region;
Means for transmitting the object identification information to the search means;
Means for obtaining size information and object content information of the object searched from the database by the search means based on the object identification information;
Using the actual size of the acquired object and the positions of the four corner points of the object in the captured image, the pixels included in the area of the object in the captured image and the points on the actual object are determined. Means for calculating a camera parameter matrix to be associated;
Means for calculating the position of the region on the actual object specified by the user using the camera parameter matrix;
And means for superimposing the object content information corresponding to the position of the area on the actual object designated by the user on the photographed image of the object and transmitting it to the display means. Image display system.   A program according to any one of claims 1 to 10, wherein the method or system according to any one of claims 1 to 10 is written in a computer program so as to be executable.   A recording medium having recorded thereon a program that can execute the method or system according to any one of claims 1 to 10 by a computer.

Images