JP2003168116A - Method for extracting universal quantity of dynamic visualization information, animation processing method and animation processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a new method for extracting universal quantity for dynamic visualization information having features such as flexible adaptability even to the dynamic visualization information. <P>SOLUTION: The method for extracting the universal value having a process for acquiring pieces of image data by every frame regarding a plurality of frames to constitute the dynamic visualization information, a process for acquiring appearance frequency distribution regarding information values prescribed for a plurality of pixels to constitute pieces of the image data by every frame and a process for extracting the universal value of the dynamic visualization information by performing orthogonal transformation of the appearance frequency distribution in the frame direction and a device by which the same method can be performed are provided. Pieces of the image data by every frame are acquired for a plurality of the frames to constitute the dynamic visualization information, the appearance frequency distribution is calculated for the information values prescribed for a plurality of the pixels to constitute pieces of the image data by every frame and the universal value of the dynamic visualization information is extracted by performing the orthogonal transformation of the appearance frequency distribution in the frame direction. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for extracting a universal quantity that can be processed by a computer from dynamic visualization information recognized by human vision, and further, utilizing the universal quantity, the dynamic visualization information. The present invention also relates to a moving image processing technique for identifying images.

[0002]

2. Description of the Related Art In this specification, visualization information refers to information represented in a form recognizable by human vision.
An image of an object captured by the human eye is a typical example of visualization information, and an image taken by a general camera or a digital camera corresponds to this. In addition, visualization of distributions or shapes that cannot normally be visually perceived, such as infrared images by a meteorological satellite, images by an electron microscope, magnetic domain images by the Kerr effect, etc., is one type of visualization information. In addition, even if information such as a sound wave of nature or a sound of nature is displayed on a device such as an oscilloscope, information that is originally recognized as a sense other than visual can be visually recognized as an image by a predetermined method. Is visualization information.

Further, in the present invention, the dynamic visualization information is movement of a person photographed by a photographing camera or a digital video, infrared image at each time by a meteorological satellite, migration image of virus by an electron microscope, and visualization information. I will say something that changes over time. That is,
An image taken by a general camera or a digital camera shows a section of dynamic visualization information that is obtained by cutting dynamic visualization information that changes with time at a specific time. In that sense, the dynamic visualization information is shown. Can be said to be a more generalized version of static visualization information.

More generally, the dynamic visualization information in this specification does not necessarily have to change along the time axis, and may be, for example, another axis that is not limited to the time axis. Visualization information that changes along with it is called dynamic visualization information. These pieces of dynamic visualization information differ depending on the conditions of the natural world when they are visualized, and even if the objects are the same, the dynamic visualization information that completely matches is not reproduced.

On the other hand, the human brain recognizes the dynamic information of the outside world by processing the audiovisual information, and the audiovisual information as the target includes a language or a score coded according to a certain configuration rule. There are coded audiovisual information and non-coded dynamic visualization information, that is, non-coded audiovisual information such as movies. The dynamic visualization information of such movies and animations is data based on the human visual information processing ability of processing the latter non-coded audiovisual dynamic information. It is considered that the universal amount of is sensed and recognized. Therefore, in order to realize human visual information processing ability in artificial intelligence by a computer, it is necessary to extract some universal amount of computer-processible dynamic visualization information from the dynamic visualization information.

Further, for example, in the case of performing person identification by image identification, it is easily expected that dynamic visualization information having a large amount of information can perform identification with higher accuracy than static visualization information. But in this regard, rather than a static photo, a so-called still photo,
It is empirically known that a movie can recognize a person much more accurately.

By the way, as a method for extracting a universal amount from a static image, pixel color components R (red), G (green) and B are used.
There is a method in which an image on a computer screen is projected onto a three-dimensional RGB orthogonal coordinate system having (blue) as a coordinate axis and the projected pattern is used as a universal amount of the image. FIG. 8 conceptually shows a projection operation performed by the universal quantity extraction method. The image displayed on the computer screen is a set of pixels arranged in a two-dimensional plane. Therefore, as shown on the left side of the figure, the position on the screen is represented by xy orthogonal coordinates, and the i-th pixel in the x direction. The position is xi, and the j-th pixel position in the y direction is yj. Further, the functions that give R, G, and B color components to the position are fr, fg, and fb, respectively,
The color component values of R, G, and B that form the pixel at the position (xi, yj) are fr (xi, yj) and fg (xi, y, respectively).
j) and fb (xi, yj).

Next, the pixel at the position (xi, yj) is projected onto the point on the RGB orthogonal coordinates to which the color component value corresponds. That is, as shown on the right side of the figure, in RGB rectangular coordinates, R = fr (xi, yj) and G = fg (xi, y
j) and the point (ro, gp, B = fb (xi, yj))
bq) corresponds to the pixel at the position (xi, yj), and the function fRGB (ro, gp, bq) at that point
The value of is incremented by one. Where o, p, q
Is an integer corresponding to each gradation of R, G, and B on the screen, ro is a color component value corresponding to the o-th gradation of R, and gp is a p-th gradation of G. The corresponding color component value, bq, represents the color component value corresponding to the q-th gradation of B. The function fRGB (ro, gp, b
The initial value of q) is 0 at all points.

Such a projection operation is performed for each pixel, and all the pixels at the position (xi, yj) corresponding to the pixel number I from 1 to x and the pixel number J from 1 to y are projected on the RGB orthogonal coordinates. . This gives the function fRGB (r
The value of each point of (o, gp, bq) represents the number of pixels having the corresponding color component value in the image on the screen, and forms a pattern unique to the image. In the above operation, the information about the geometrical coordinates of the static image is removed, and only the intensity distribution information of each RGB component is extracted. Normalize this pattern as needed,
By expressing it in a vector format, it is used as a universal quantity of the image and is used for image processing by a computer.

[0010]

However, according to human visual information processing ability, not only a color image having R, G and B color components but also one or two of these color components is used. It is also possible to recognize dynamic visualization information such as a displayed image or a gray image without color. On the other hand, in the universal amount extraction method for the image described above, each pixel is set to R, G
And B are projected onto one point on the RGB orthogonal coordinates according to the color component values. That is, the projection point is determined according to how the three primary colors are combined in each pixel, and the set of three color component values of R, G, and B that configure each pixel is used as one set of information. is doing. Therefore, the above-mentioned universal quantity extraction method basically targets universal quantity extraction from color images, and is suitable for handling visualization information of achromatic colors (grayscale) and visualization information represented as binary data. Absent.

In other words, the universal quantity obtained by the above universal quantity extraction method is the function fR projected on the RGB orthogonal coordinates.
Since it is composed of the values of each point of GB (ro, gp, bq), the number of elements of the universal quantity is a number obtained by multiplying the number of gradations of R, G and B, for example, R, G and B. If the number of gradations of all is 256, it becomes 256 ³ . Therefore, there is a large amount of universal data to be handled, and it may be difficult for the computer to process it quickly and accurately. On the other hand, if the universal amount of the visualization information can be made into the light intensity information consisting of a single component, the number of elements of the universal amount is overwhelmingly small, and in the case of the above example, only 256 is used. You can

Furthermore, since the above method is intended for static visualization information, it cannot be used to extract a universal quantity from dynamic visualization information which is a set of a plurality of static visualization information. Furthermore, considering that the amount of information that dynamic visualization information has is equivalent to an extremely large amount of static visualization information, a huge amount of information processing is required unless it is a method unique to universal quantity extraction from dynamic visualization information. Therefore, in reality, it may be impossible for the computer to perform the calculation.

The present invention has been made in view of such circumstances, and can be flexibly applied to a wider range of dynamic visualization information, and the amount of data handled by a computer can be reduced to speed up the operation. To provide a new method for extracting a universal amount of dynamic visualization information that enables accurate and accurate processing, and makes it possible to bring the processing ability of dynamic visualization information by a computer closer to that of human visual information processing. It is an object.

Further, the present invention can be applied to moving images of various forms of dynamic visualization information by utilizing the universal quantity obtained by such a universal quantity extraction method, and
It is an object of the present invention to provide a moving image processing technique capable of realizing appropriate dynamic image recognition.

[0015]

In order to achieve the above object, a method for extracting a universal quantity of dynamic visualization information according to the present invention is
The appearance frequency for each information value from the information value distribution of pixels throughout the frame image data forming the dynamic visualization information, or the appearance frequency for each information value of each frame image data forming the dynamic visualization information A Fourier frequency spectrum obtained by performing Fourier transform in the frame moving direction is extracted as a universal amount of the dynamic visualization information.

More specifically, according to the universal quantity extraction method for dynamic visualization information based on the present invention, image data for each frame of a plurality of frames forming the dynamic visualization information is acquired, and for each frame. An appearance frequency distribution is obtained for information values defined for a plurality of pixels forming image data, and the appearance frequency distribution is orthogonally transformed in the frame direction to extract a universal amount of dynamic visualization information.

According to one embodiment of the present invention, in the process of acquiring image data in the above method, moving image data including component values of three primary colors forming each pixel of the frame is acquired.

According to another embodiment of the present invention, the information value is at least one of a light intensity value, a hue and a color component value.

According to still another embodiment of the present invention, the appearance frequency distribution is normalized before orthogonally transforming the appearance frequency distribution in the frame direction. The appearance frequency distribution is normalized so that the appearance frequency distribution for each frame has a value in a certain range. Alternatively, the normalization of the appearance frequency distribution is
It is performed so that the maximum value of the appearance frequency in all frames becomes a predetermined value.

According to one embodiment of the present invention, a discrete Fourier transform is used as the orthogonal transform in the frame direction. Further, the contents shown in the above-mentioned embodiments are not alternatives, and it is possible to combine them and carry out at the same time.

Further, according to another aspect of the present invention, a method for evaluating the similarity of objects based on the universal quantity extraction of dynamic visualization information is proposed. That is, the first universal quantity related to the dynamic visualization information of the reference object is extracted by the method described in any one of the above, and the second universal quantity related to the dynamic visualization information of the unknown object is extracted by the method described in any one of the above.
Is extracted, and the similarity between the reference object and the unknown object is evaluated based on the first universal quantity and the second universal quantity. Here, the method for extracting the first universal quantity and the method for extracting the second universal quantity may be the same or different.

According to still another aspect of the present invention,
A method to identify an object based on universal quantity extraction of dynamic visualization information is proposed. That is, the first universal quantity relating to the dynamic visualization information of a plurality of reference objects is extracted by the method described in any one of the above, and the first universal quantity regarding the dynamic visualization information of the unknown object is extracted by the method described in any one of the above. This is a method of identifying a target object and an unknown object by extracting 2 universal quantities and comparing the 1st universal quantity and the 2nd universal quantity. Here, as in the above, the method for extracting the first universal quantity and the method for extracting the second universal quantity do not necessarily have to be the same.

Furthermore, according to yet another aspect of the present invention, an apparatus for implementing the above method is proposed. That is, for a plurality of frames forming the dynamic visualization information, a means for acquiring image data for each frame, and an appearance frequency distribution for information values defined for a plurality of pixels forming the image data for each frame are displayed. A universal quantity extraction device for dynamic visualization information, comprising: a means for acquiring and a means for orthogonally transforming the appearance frequency distribution in a frame direction to extract a universal quantity of dynamic visualization information. The apparatus includes means for extracting a first universal quantity relating to dynamic visualization information of a reference object by any one of the methods described above, and dynamic visualization of an unknown object by any one of the methods described above. Similarity of objects including means for extracting a second universal quantity related to information, and means for evaluating the similarity between the reference object and the unknown object based on the first universal quantity and the second universal quantity It can be developed into an evaluation device. Further, the apparatus includes means for extracting a first universal quantity relating to the dynamic visualization information of a plurality of reference objects by the method according to any one of the above, and similarly, a means for extracting an unknown object by the method according to any one of the above. An object including means for extracting a second universal quantity related to dynamic visualization information, and means for identifying a target object and an unknown object by comparing the first universal quantity and the second universal quantity The identification device can be developed.

According to one specific embodiment of the present invention, first, the dynamic visualization information is taken in to obtain the moving image data of the dynamic visualization information, and the pixels are displayed in the display image by the moving image data. The distribution of information values included in the information that defines visible light is normalized to a distribution that takes a value within a certain range. Then, the appearance frequency of each normalized information value is obtained from the normalized distribution, and the obtained appearance frequency distribution or the appearance frequency of each information value of the frame image data that constitutes the dynamic visualization information is obtained. A Fourier frequency spectrum obtained by performing Fourier transform in the frame moving direction is extracted as a universal amount of the dynamic visualization information.

Here, as the moving image data, the component values of the three primary colors forming each pixel in the combined image obtained by combining all the frames forming the dynamic visualization information of an arbitrary number of frames so as not to overlap each other. You may use what contains. In this case, a plurality of information value distributions that determine the visible light to be displayed are obtained based on those component values, each of the plurality of distributions is normalized, and there is a normalized appearance frequency distribution of each information value. Hereinafter, this method is referred to as a synthetic image method.

Further, as the moving image data, data including component values of the three primary colors forming each pixel in each frame image forming the dynamic visualization information is used, and the display visible light is displayed based on these component values. Obtain multiple distributions of information values to be defined, normalize each of these multiple distributions, obtain the normalized appearance frequency distribution of each information value, and obtain the universal amount of the dynamic visualization information, that is, the information of each frame. As the value, the light intensity value, the hue, and the color component value of the pixel in each frame are combined and obtained. Next, a Fourier frequency spectrum in which the light intensity value, the hue, and the color component value of the pixels of all the frames are Fourier-transformed in the frame movement direction and the frame movement information phase is deleted may be used. In this case, the information value for which the distribution can be obtained is the Fourier frequency spectrum in which the light intensity value, the hue, and the color component value of the pixels through all the frames are mixed.

As the moving image data, data containing component values of the three primary colors forming each pixel in each frame image forming the dynamic visualization information is used, and the visible light is displayed based on those component values. Obtain multiple distributions of information values to be defined, normalize each of these multiple distributions, obtain the normalized appearance frequency distribution of each information value, and obtain the universal amount of the dynamic visualization information, that is, the information of each frame. As the value, the light intensity value, the hue and the color component value of the pixel of each frame are obtained. Next, the light intensity value of the pixel for all frames,
By using the hue and color component values, the light intensity value of the pixel, the hue and the color component value are Fourier-transformed in the frame moving direction, respectively, through the entire frame, which is obtained by combining the light intensity value of the pixel and the hue and the color component value. Alternatively, a Fourier frequency spectrum in which the frame movement information phase is deleted may be used. The information values that can be obtained in this case include the light intensity value of the pixel through all frames, the hue and the color component value, and the respective Fourier frequency spectra.

On the other hand, in the image processing method according to the present invention, the universal quantity of each of the plurality of dynamic visualization information is extracted in advance by utilizing the universal quantity extraction method. Then, the universal quantity of the dynamic visualization information of the unknown target is extracted by a similar universal quantity extraction method, and the matching with each universal quantity extracted in advance is evaluated to obtain an image of the dynamic visualization information of the unknown object. Is identified as an image of any of the plurality of dynamic visualization information.

The image processing apparatus according to the present invention further comprises first extracting means for extracting in advance the universal amount of each of a plurality of pieces of dynamic visualization information using the universal amount extracting method, and the first extracting means.
Of the dynamic visualization information of the unknown object by the second extracting means for executing the same universal quantity extracting method as the first extracting means. Extract the amount. Then, as the identification means for identifying the image, the dynamic visualization information of the unknown target is evaluated by evaluating the matching between the universal quantity extracted by the second extraction means and the respective universal quantities stored in the storage means. Of the dynamic visualization information is identified as an image of any of the plurality of dynamic visualization information.

[0030]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. The present embodiment is an example in which part of human visual information processing capability is reproduced by a computer. According to human visual information processing ability, when an image captured by the naked eye is given as dynamic visualization information, the storage of the image information, the perception of the significance of the image information, and the recognition of the image information (other image information And the like) are performed. In order to realize these operations by a computer, in this embodiment described below, the video information is stored by extracting and recording a universal amount from a given video, and a linguistic description showing the significance of the video information is provided. The meaning of the video information is perceived by giving it, and the video information is recognized by evaluating the matching with the universal amount of the video information stored in advance.

<Structure> FIG. 1 is a diagram showing the structure of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus is composed of a video capture device 1, a computer 2, a storage device 3, an input device 4 and an output device 5 as shown in the figure.
An image representing a video of the object VO given as a processing target is handled.

The image capturing device 1 is a means for acquiring image information for capturing the image of the object VO, and is constituted by a digital video camera or the like, and color moving image data representing the captured image is used as the image information of the object VO. Electronic calculator 2
Supply to. The object VO may be any object that gives dynamic visualization information recognized by human vision, such as a human being, a car, etc., and the image capturing device 1 uses reflected light obtained by illuminating the object VO with the light source LS. Capture video by.
When capturing the respective images of the plurality of target objects VO, the same light source is always used as the light source LS so that the image generation conditions of the plurality of target objects VO are always constant (reflection depending on the light source). Make sure there is no information change in the optical image.)

Here, the color moving image data from the image capturing device 1 is data that forms an image (visualization information) by the image displayed on the computer screen, and is regularly displayed on the computer screen. It has information of visible light to be displayed in each pixel arranged in. That is, the video capturing device 1 displays the object V on the computer screen.
It is composed of a plurality of frames that form a moving image of O.
Color components R (red) and G of each pixel forming each frame
Information of (green) and B (blue) is supplied to the electronic computer 2 as the color moving image data. In such color moving image data, the video information in the display image depends on the resolution (total number of pixels) of the computer screen and the number of frames, and the information amount of each pixel of the image forming each frame is displayed on the computer screen. Although it depends on the dynamic range of displayable red component, green component, and blue component (range of gradation values that each color component can take), in the present embodiment, these are determined in accordance with the specifications of the electronic computer 2, Video capture device 1
Always supplies color moving image data having a resolution and a dynamic range according to a certain standard conforming to the specifications to the electronic computer 2.

The electronic computer 2 is a computer having predetermined arithmetic means, storage means, control means, etc. necessary for moving image information processing, and each processing operation by the moving image processing apparatus (described later).
It is configured by a computer or the like that has loaded a program for executing. The electronic computer 2 includes a unique pattern extraction unit 2 which is realized by an arithmetic means etc. by the program.
a, identification calculation unit 2b, identification determination unit 2c, and result output unit 2
d is provided as a configuration peculiar to the moving image processing apparatus. A storage device 3, an input device 4, and an output device 5 are connected to the electronic computer 2 as peripheral devices.

The unique pattern extraction unit 2a receives the color moving image data supplied from the image capturing device 1 and extracts the unique pattern of the image represented by the color moving image data.
The unique pattern extracted here corresponds to a universal amount of video (dynamic visualization information), and the unique pattern extraction unit 2a outputs the extracted unique pattern to the storage device 3,
Alternatively, it is supplied to the identification calculation unit 2b. The specific content of the unique pattern will be clarified in the operation description to be described later together with the extraction processing.

The identification calculation unit 2b includes a unique pattern extraction unit 2
An operation for identifying the unique pattern supplied from a as one of the previously stored unique patterns is performed, and the identification determination unit 2c determines the specific unique pattern identified based on the operation result. . Result output unit 2d
Receives the calculation result of the identification calculation unit 2b and the determination result of the identification determination unit 2c, and outputs a predetermined result to the output device 5. Note that the specific contents of the processing by the identification calculation unit 2b, the identification determination unit 2c, and the result output unit 2d will be described in detail later in the operation description.

The storage device 3 records and registers (holds) the unique pattern output from the unique pattern extraction unit 2a. By performing this registration for images of a plurality of objects VO, an image database in which the unique patterns of those images are accumulated is built in the storage device 3. When the identification calculation unit 2b refers to the unique pattern stored in advance, the image database in the storage device 3 is used.

The input device 4 is an input device of the electronic computer 2 which is composed of a keyboard, a pointing device and the like. The input device 4 is used to give a linguistic description representing the meaning of the original video information to the unique pattern registered in the storage device 3 (for example, the object VO is a human being, an animal, or the like). Enter a predetermined linguistic description such as). As a result, the unique pattern extraction unit 2
The significance of the original video information is output from the a together with the unique pattern to the storage device 3 so that the significance of the video information can be perceived. However, inputting this linguistic description is an optional operation,
The operator may perform it as needed.

The output device 5 is an external output device of the electronic computer 2 constituted by a display device, a printer, etc., and the resolution of the display device depends on the specifications of the electronic computer 2 on which the video information in the above-mentioned display image depends. It corresponds to.
The dynamic range on which the information amount of each pixel depends depends on the specification (display setting) of the electronic computer 2 according to the standard such as 24-bit full color so that it can be displayed on the display device of the output device 5. The amount is predetermined.

<Operation> (1) Registration of Unique Pattern of Video Next, the operation of the moving image processing apparatus having the above configuration will be described. FIG. 2 is a flowchart showing a processing procedure for registering a unique pattern of an image. This process is performed on the object V
This is a process of extracting a unique pattern from the O image and registering it in the storage device 3 described above.

When registering the unique pattern of the image, first, the image of the object VO to be made into a database image to be stored in advance is captured by the image capturing device 1 (step S). As a result, the color image data representing the image of the object VO is supplied from the image capturing device 1 to the electronic computer 2.

Here, the color moving image data from the image capturing device 1 is composed of a plurality of frame images forming a moving image of the object VO, each frame image is composed of pixels, and the color component R of each pixel is It has information of (red), G (green), and B (blue). In a display image based on such color moving image data, each pixel displays visible light determined by the combination of reflected light intensity, hue, and color component, and video information is expressed by their geometrical arrangement. Therefore,
In the present moving image processing apparatus, three elements of reflected light intensity, hue, and color component for one frame image in such video information are used as constituent elements of the basic unique pattern, and the three elements are histogram-formed. The light intensity eigenvector, the hue eigenvector, and the color component eigenvector are calculated by arithmetic processing in the eigenpattern extraction unit 2a based on the color image data from the video capturing device 1 (steps S 1 to S 3).

The reflected light intensity eigenvectors, hue eigenvectors, and color component eigenvectors of one frame image of a moving image or a composite image (composite image method described in paragraph 0013) in which all frames are composited so as not to overlap are described below. The calculation processing in the unique pattern extraction unit 2a will be sequentially described. Although the order of description is in accordance with the step progress of FIG. 2, the actual arithmetic processing in the computer does not necessarily have to be performed in this order, and can be changed as appropriate.

Calculation of reflected light intensity eigenvector (step S) The reflected light intensity eigenvector is a value corresponding to the frequency of each class when a histogram is created from the distribution of the reflected light intensity in the display image with each value of the reflected light intensity as a class. It is a vector that is an element. The reflected light intensity value of the i-th pixel in one frame of color image data is Iint, i, and the color component values of R, G, and B of the same pixel are Ri and G, respectively.
Given i and Bi, the reflected light intensity value Iint, i is given by the scalar sum or vector sum of the color component values Ri, Gi and Bi. That is,

[Equation 1] Or

[Equation 2] (Whether the reflected light intensity value is obtained by any of the equations 1 and 2 is arbitrary and may be appropriately selected).

Therefore, if the reflected light intensity distribution in the display image based on the color image data of one frame is Iint, the reflected light intensity distribution Iint is

[Equation 3] Given in. T is the total number of pixels in one display image, and the unique pattern extraction unit 2a performs the calculation of Formula 1 or Formula 2 for each pixel from the first to the Tth, and the reflected light intensity of Formula 3 is obtained. Obtain the distribution Iint.

Next, the reflected light intensity distribution Iint is a distribution of the dynamic range D (a distribution in which the reflected light intensity value takes a value in the range from 0 to a predetermined integer D. Hereinafter, this distribution is represented by the symbol "IintD", "Normalized reflected light intensity distribution Iin
tD ”. ). Here, "Round [a]
"Indicates the operation of converting the value" a "in parentheses into an integer, and" Ma
Suppose that x [A] ”represents an operation for finding the maximum value element in the set“ A ”in parentheses, the unique pattern extraction unit 2
a transforms the reflected light intensity distribution Iint into the normalized reflected light intensity distribution IintD by the following equation 4.

[Equation 4] The dynamic range D is appropriately selected in consideration of the hue and the resolution of color components described later. For example, when the gradation of the color component has the lowest resolution and the number of gradations is 256, the value of D is set to 256, for example.

Thereafter, the obtained normalized reflected light intensity distribution I
At intD, the unique pattern extraction unit 2a counts the number of pixels that take each (normalized) reflected light intensity value from 0 to D. Thereby, the reflected light intensity distribution normalized by the dynamic range D is histogrammed with the reflected light intensity value as a class, and the appearance frequency of each (normalized) reflected light intensity value corresponding to the frequency of each class is obtained. The peculiar pattern extraction unit 2a sets the appearance frequency of each reflected light intensity value thus obtained as array data in a vector format to obtain a reflected light intensity eigenvector Eint. The reflected light intensity eigenvector Eint is an eigenvector obtained for one image.

Calculation of Hue Eigenvector (Step S) The hue eigenvector is a vector whose elements are values corresponding to the frequency of each class when a histogram is created from the distribution of the hue in each pixel in the display image as the class of each hue value. Is. Hue here means R, G, and B in each pixel.
Is the composition ratio of the color component values of R, G, or B and is represented by the ratio of the R, G, or B color component values to the reflected light intensity value. Since the sum of these three color component values is given by the reflected light intensity value, it is the hue for any two of the color components R, G and B that has independent information for the reflected light intensity value. is there.

The hue distribution in the display image for each color component represents the color component composition ratio in each pixel in the display image. For example, when considering the hue distribution of the R component, the hue distribution of the R component in the display image is ITo
Let ne and R be the reflected light intensity values Iint,
By the color component values Ri of i and R,

[Equation 5] Given in. In this case, the unique pattern extraction unit 2a
The right side of the equation (5) is divided for each of the first to Tth pixels to obtain the R component hue distribution Itone, R. Note that the R component hue in each pixel corresponds to the division formula for each pixel on the right side of Equation 5, and "Ri / Iint, i" described is the R component hue of the i-th pixel (hereinafter, this The R component hue of the i-th pixel is represented by the symbol "ITone, R, i".)

Next, the R component hue distribution Itone, R is normalized to the distribution of the dynamic range D. This normalization calculation is similar to the above-described transformation of the reflected light intensity distribution Iint into the normalized reflected light intensity distribution IintD. That is, if the normalized R component hue distribution Itone, R is normalized to a distribution in which the R component hue of each pixel takes a value in the range of 0 to D, Itone, ^RD is defined as the unique pattern extraction unit. 2a is the R component hue distribution ITone, R normalized by the following equation 6 R component hue distribution ITon
e, transformed into R ^D.

[Equation 6]

Then, the obtained normalized R component hue distribution I
In Tone, R ^D , the unique pattern extraction unit 2a
Count the number of pixels taking each (normalized) R component hue from 0 to D. Thereby, the hue distribution of the R component normalized by the dynamic range D is histogrammed with the R component hue as a class, and the appearance frequency of each R component hue corresponding to the frequency of each class (normalized) is obtained. The peculiar pattern extraction unit 2a sets the appearance frequency of each R component hue thus obtained as array data in a vector format to be the R component hue eigenvector ETone, R.

The G component hue eigenvectors ETone, G and the B component hue eigenvectors ETone, B can also be obtained by the same calculation processing as described above (in the calculation processing for obtaining the R component hue eigenvectors ETone, R described above,
If the arithmetic processing indicated by replacing "R" with "G" and "R" with "B" is performed, G component hue eigenvectors ETone, G, B component hue eigenvectors E respectively.
Tone, B is obtained. ). Unique pattern extraction unit 2a
Are R component hue eigenvectors ETone, R, G component hue eigenvectors ETone,
Two sets of eigenvectors of the G and B component hue eigenvectors ETone, B are calculated. This is because the reflected light intensity eigenvector Eint includes information about the sum of color component values, and the degree of freedom is reduced by one. 2
Which combination is used as the eigenvector of the set is arbitrary and may be appropriately selected.

Calculation of color component eigenvector (step S
) The color component eigenvector is R, G or B in the display image.
Is a vector whose elements are the values corresponding to the frequencies of each class when a histogram is created from each color component distribution as a class of each color component value. The color component eigenvectors are given by the amounts of R component, G component and B component in the display image, and three sets are obtained for one video. The geometric information of the image is gently held as the ratio of the R component, the G component and the B component.

When considering the R component distribution in the display image, if the R component distribution is IComp, R,

[Equation 7] Is. The peculiar pattern extraction unit 2a uses the R component distribution I
Comp, R are normalized to the distribution of the dynamic range D. The normalization calculation here is also the above-mentioned reflected light intensity distribution Ii.
This is the same as the transformation of nt to the normalized reflected light intensity distribution Iint ^D. That is, assuming that the R component distribution IComp, R is normalized to a distribution in which the R component value of each pixel takes a value in the range from 0 to D, IComp, ^RD
The peculiar pattern extraction unit 2a converts the R component distribution IComp, R into the normalized R component distribution IComp, R according to the following equation 8.
Transforms into ^D.

[Equation 8]

Then, the obtained normalized R component distribution ICo
In mp and ^RD , the unique pattern extraction unit 2a counts the number of pixels having respective (normalized) R component values from 0 to D. As a result, the R component distribution normalized by the dynamic range D is histogrammed with the R component value as a class, and the appearance frequency of each (normalized) R component value corresponding to the frequency of each class is obtained. The eigenpattern extraction unit 2a sets the appearance frequency of each R component value thus obtained as array data in a vector format to be an R component eigenvector EComp, R.

The G component eigenvectors EComp, G and the B component eigenvectors EComp, B can also be obtained by the same calculation process as described above (in the calculation process for obtaining the R component eigenvectors EComp, R, "R" is changed to "G", If the arithmetic processing indicated by replacing “R” with “B” is performed, the G component eigenvector E
Comp, G, B component eigenvectors EComp, B are obtained).

The eigenpattern extraction unit 2a calculates each eigenvector based on the color image data (R, G and B component values forming each pixel) from the video capturing device 1 as described above. Then, the calculated eigenvector is registered in the storage device 3 as the eigenpattern of the image represented by the color image data (step S).

When this eigenpattern is E, the eigenpattern E is the reflected light intensity eigenvector Eint, R component hue eigenvectors ETone, R, B component hue eigenvector ETone, B, R component eigenvector ECom.
p, R, G component eigenvectors EComp, G and B component eigenvectors EComp, B,

[Equation 9] Alternatively, the eigenpattern based on the number 9 of each frame is defined as E ^j , j
= 1,2, .., n, (n is the number of frames), | a | is the absolute value of "a", and Fourier (A) is the Fourier transform of "A",

[Equation 10] Alternatively, the constituent vector of the eigenpattern E according to the number 9 of each frame is defined as E _int ^j , E _{T one, R} ^j , E _{Tone, B} ^j , E.
_{Comp, R} ^j , E _{Comp, G} ^j , E _{Comp, B} ^j , j = 1,2, .., n, (n
Is the number of frames),

[Equation 11] (Where T is a symbol representing the transpose of the matrix). Note that the R component hue eigenvector ETo
Instead of ne, R and B component hue eigenvectors ETone, B, R component hue eigenvectors ETone, R and G component hue eigenvectors ETone, G, or G component hue eigenvectors ETone, G and B component hue eigenvectors E
Tone and B may be used as described above.

The eigenpattern extraction unit 2a arranges the elements of the calculated eigenvectors in the form of the right side in any of the equations 9 to 11 to generate the eigenpattern E, outputs it to the storage device 3, and records it. At this time, the color moving image data used in the above-described arithmetic processing is also output from the electronic computer 2 to the storage device 3 and recorded in association with the unique pattern E. Thereby, the unique pattern is extracted from the image of the target VO captured in step S and registered in the storage device 3, and the registration of the unique pattern of the image of the target VO is completed.

In the present moving image processing apparatus, the above-described unique pattern registration processing is performed on the object VO of a certain type (number).
Is performed in advance to construct a video database in which unique patterns of a plurality of videos are stored in the storage device 3. Then, the identification process is performed on the image of the unknown object VO (hereinafter referred to as “target image”) given as the recognition processing target as follows.

2) Identification of Target Video FIG. 3 shows the procedure of processing for identifying the target video. In this process, first, the target video is captured by the video capture device 1 (step S). As a result, the color moving image data representing the target video is supplied from the video capture device 1 to the electronic computer 2. The color moving image data here has information on the color components R (red), G (green), and B (blue) of each pixel forming the image of the unknown object VO that is the recognition processing target. , And is sent to the unique pattern extraction unit in the electronic computer 2.

The unique pattern extraction unit 2a that has received the color moving image data extracts the unique pattern of the target video by the unique pattern extraction processing similar to the above (step S).
). That is, the received color moving image data is used to perform the arithmetic processing of steps S 1 to S 5 in “(1) Registration of unique pattern of video”, and the reflected light intensity eigenvector Eint and R component hue based on the color moving image data. Eigenvector ETone, R, G component hue eigenvector ETone, G, R component eigenvector EComp,
R, G component eigenvectors EComp, G and B component eigenvectors EComp, B are calculated. Then, the elements of these eigenvectors are arranged in the shape of the right side by any of the above equations 9 to 11 to form the eigenpattern of the target video.

The peculiar pattern of the target image thus obtained is supplied from the peculiar pattern extraction section 2a to the identification calculation section 2b, and the identification calculation section 2b executes the identification calculation (step S 1). This identification calculation is a calculation for identifying the target image as one of the images stored in advance, and the matching between the unique pattern of the target image and the unique pattern registered in the image database of the storage device 3 is as follows. And evaluate.

In the matching evaluation of the unique pattern, a linear system equation representing the unique pattern of the target video is considered by linear combination of the unique patterns registered in the video database. Now, assuming that m unique patterns are registered in the video database, the system matrix (hereinafter represented by the symbol “C”) is

[Equation 12] Given in. Therefore, if the characteristic pattern of the target image is EX, the following linear system equation is obtained.

[Equation 13] X in Expression 13 is an m-th order vector whose elements are the weights of the individual patterns of the video database. here,
The vector X satisfying the equation 13 has the j-th element as 1, and
If all other elements are set to 0, the unique pattern EX is equal to the unique pattern Ej of the video database, and the target video can be identified as the video of the unique pattern Ej. That is, identifying the target video as a prestored video results in an inverse problem of finding the vector X in Expression 13 (henceforth, the vector X is referred to as “solution vector X”).

The identification calculation unit 2b includes the unique pattern extraction unit 2
While receiving the unique pattern EX of the target video from a, the registered m unique patterns are read from the storage device 3 to obtain the system matrix C. Therefore, in the identification calculation in the identification calculation unit 2b, EX and C
Since the solution vector X may be obtained with known as, the solution vector X can be calculated by using the solution method of the inverse problem used in general multivariate analysis and the like.

In this embodiment, the least squares method is used as an example of the solution. Since the eigenpattern is composed of the six eigenvectors described above, and each eigenvector is a histogram of a distribution normalized by the dynamic range D, the order of the eigenpattern is 6 times the dynamic range D. . Given that this order is n, it can be considered that the order n is usually larger than the number m of unique patterns registered in the video database, so that the system matrix C becomes a rectangular matrix with n rows and m columns. 1
3 becomes an inappropriate linear system equation), and the least squares method can be applied to derive the solution vector X. Therefore, the identification calculation unit 2b uses the eigenpattern EX from the eigenpattern extraction unit 2a and the system matrix C obtained from the storage device 3 to execute the following calculation by the least squares method to obtain the solution vector X. calculate.

[Equation 14] It is needless to say that the inappropriate solution of the linear system of the equation 13 is not limited to the equation 14 and is merely adopted as one concrete method here.

The solution vector X calculated by the identification calculation section 2b as described above is obtained by the identification determination section 2c and the result output section 2d.
The unique pattern EX of the target video is also supplied to the result output unit 2d. Then, the identification determination unit 2c makes a determination to identify a specific unique pattern based on the solution vector X (step S). For example, if one element of the solution vector X is 1 and all other elements are 0, the identification determination unit 2c sets the unique pattern of the video database weighted by the 1 element as the unique pattern of the target video. Judge as something. Further, in the determination here, when a certain element of the solution vector X is equal to or more than a predetermined constant value and another element is equal to or less than another predetermined constant value, the element having the former constant value or more is weighted. The unique pattern to be added may be determined to be the unique pattern of the target video.

The determination result of the identification determination unit 2c is supplied to the result output unit 2d, and the result output unit 2d outputs a predetermined result together with the solution vector X and the unique pattern EX from the identification calculation unit 2b ( Step S). That is, the result output unit 2d reads the unique pattern of the identified video database and the color moving image data recorded corresponding thereto from the storage device 3 based on the determination result from the identification determination unit 2c, and identifies them as those data. Computing unit 2b
And outputs the data of the solution vector X and the unique pattern EX from the output device 5. As a result, in the output device 5, the unknown object VO given as the recognition processing target is identified from the unique pattern EX of the image, the solution vector X of the identification calculation result, and the image stored in advance. The information such as the captured image and its unique pattern is displayed on the display of the display device or the printout of the printer, and the identification of the target image is completed.

The above is the configuration and operation of the moving image processing apparatus. As described above, in the present moving image processing apparatus, the distribution of reflected light intensity, hue and color components in an image in which an image is captured is normalized in a certain dynamic range and then histogrammed to obtain a unique pattern of the image. . Therefore, this unique pattern represents the relative frequency of appearance of each value of the reflected light intensity, the hue, and the color component in one image, and does not depend on the number of pixels when the video is converted into a moving image. That is, the peculiar pattern is information in which the original image information is formed into a histogram and transformed into a ratio with respect to the total number of pixels to eliminate the dependency on the number of pixels limited by hardware such as a computer screen. ing.

Further, in the above-mentioned extraction of the unique pattern,
Since the distribution of the reflected light intensity and the like is made into a histogram for each normalized class, the information of the position where the reflected light intensity and the like are distributed in the original image is lost. Therefore, the unique pattern is information that does not depend on the reference coordinates in the original image, and is extracted as information that represents the image regardless of the viewpoint when capturing the image, and the geometric shape change of the image is detected. It will not be affected by changes in the layout.

As described above, the unique pattern extracted by the moving image processing apparatus is the original feature amount of the image that does not depend on the hardware or the situation at the time of capturing the image, and is a universal pattern unique to the target image. Represents a quantity, which is an appropriate universal quantity extracted from visualization information that is not reproducible even for the same target. In this universal amount, the reflected light intensity, the hue of each color component, and each color component are independent components, and are treated as independent information. Therefore, if information is obtained even for one of these components, it is possible to extract the universal quantity and perform the identification as described above. That is, the present moving image processing apparatus can be realized for a single color component of R, G or B, a single hue of R component, G component or B component, or reflected light intensity alone, in other words, for a monochrome moving image, It can be flexibly applied to a wider range of visualization information.

Further, in the conventional universal quantity extraction method by projecting onto RGB orthogonal coordinates, the number of elements of the universal quantity is R,
The number of gradations of G and B (the number of gradations to the third power) is multiplied, whereas in the above-mentioned unique pattern, the order is 6 times the dynamic range D (the number of elements is 6 times the number of gradations, etc.). ), The number of elements of the universal quantity is smaller than before. Therefore,
According to this image processing apparatus, as described above, the universal amount that appropriately represents the original feature of the image is extracted with a smaller amount of data than before and is used for the identification process. It can be performed.

<Experimental Example> An experimental example for confirming the accuracy of moving image recognition of the present moving image processing apparatus will be shown below. In the following experimental example, the same digital video camera is used as the image capturing device 1, and the object V for constructing the image database V is used.
As O, an experiment was conducted using four types of color animation images, four types of monochrome animation images, walking motions of 11 humans, and moving images of the first half of the body. This is because the animation image assumes a moving image shot under an ideal environment. In order to examine whether it is possible to identify the differences in human clothes during walking motion in the moving images of 11 humans, we adopted two 4 moving images with different backgrounds. Video images of three humans were adopted to identify upper body movements. In the identification of the target video, the identification determination is performed by determining the element having the maximum value among the elements of the solution vector X.

Target Video Identification Experiment FIG. 4 is a diagram showing the results of an identification experiment when a color animation image in which the same target moves in different directions is used as the target video. In this figure, the upper part is an animation image in which the same target is moving in different directions.
The lower part is a graph showing the element values of the solution vector X obtained by using the Fourier frequency spectrum eigenpattern (Equation 10) with respect to the frame movement direction (the arrangements shown are the same for the identification experiment results described later). ).
The purpose of this experiment is to examine whether or not the same target image can be identified using the Fourier frequency spectrum peculiar pattern (Equation 10) with respect to the frame moving direction even if the moving direction is different.

In this identification experiment, in the solution vector X as shown in the lower graph, the No.
The element of (number) is 1, and the others are zero, and the effectiveness of the Fourier frequency spectrum peculiar pattern (Equation 10) in the frame movement direction can be confirmed.

Target Video Identification Experiment FIG. 5 is a diagram showing a result of an identification experiment when a color animation image in which different targets move in different directions is used as the target video. In this figure, the upper row is one of the database moving images in which different objects are moving in different directions.
It is a frame image. The middle row is a 1-frame image of the sample animation in which the target image moves in different directions.
The lower part is a graph showing the element values of the solution vector X obtained by using the Fourier frequency spectrum peculiar pattern (Equation 11) in the frame movement direction. Even when different targets move in different directions, it can be seen that individual target images can be identified from the element having the maximum value of the solution vector X. The purpose of this experiment is to examine whether it is possible to identify the target image using the Fourier frequency spectrum peculiar pattern (Equation 11) with respect to the frame moving direction even if the moving direction of the target image is different. In this sense, the effectiveness of the Fourier frequency spectrum peculiar pattern (Equation 11) in the frame movement direction can be confirmed.

Target Video Identification Experiment FIG. 6 is a diagram showing the results of an identification experiment when a monochrome animation image with a background in which different targets move in different directions is used as the target video. In this figure, the upper part is one frame image of a monochrome database moving image with a background in which different objects are moving in different directions. The middle row is a one-frame image of a monochrome test animation with a background in which the target image moves in different directions. The lower row is
Solution vector X obtained using the Fourier frequency spectrum eigenpattern (Equation 11) for the frame movement direction
3 is a graph showing the element values of and the element values of a solution vector X by a combining method that combines all frames into one image. It can be seen that even in a monochrome moving image with a background, individual target images can be identified from the element having the maximum value of the solution vector X even when different targets move in different directions.
The result of this experiment is a monochrome moving image with a background, and even if the moving direction of different target images is different, it is identified using the Fourier frequency spectrum peculiar pattern (Equation 11) and the synthetic image method (paragraph 0013) with respect to the frame moving direction. The point is to examine whether it is possible. In this sense, the effectiveness of the Fourier frequency spectrum eigenpattern (Equation 11) and the eigenpattern (Equation 9) based on the composite image method (paragraph 0013) in the frame movement direction can be confirmed.

Target Video Identification Experiment FIG. 7 is a diagram showing the results of an identification experiment when the motions of the first half of the body of different persons were photographed using an actual digital video camera. In this figure, the upper row shows 1 of the color database moving image with background in which different objects are performing different actions.
It is a frame image. The middle row is a one-frame image of a color sample animation with a background in which the target is performing different actions. The lower part is a graph showing the element values of the solution vector X by the composite image method (paragraph 0013) in which all frames are composited into one image. From the element that takes the maximum value of the solution vector X, it is possible to identify even if the same person performs different actions, and the validity of the unique pattern (Equation 9) according to the composite image method (paragraph 0013) can be confirmed.

<Examples of Modifications / Applications> Although one embodiment of the present invention has been described above, the embodiment of the present invention is not limited to the above. Hereinafter, other modifications or applications will be described with some examples.

(1) Applicable Visualization Information As the object of universal information extraction and identification processing, the image of the object captured by the human visual sense is used in the above-mentioned embodiment, but the present invention is not limited to this. Visualization information can be used. In particular, in the universal quantity extraction based on the present invention, since the reflected light intensity, the color component, etc. are treated as separate and independent information as described in the above embodiment,
Not only normal color images taken by a general camera or video recorder, but also images in other forms can be targeted.

For example, distributions or shapes that cannot normally be visually recognized, such as infrared rays images by a meteorological satellite, images by an electron microscope, images by MRI, fluid images by a tracer, magnetic domain images by the Kerr effect, etc. The visualized dynamic visualization information may be applied. Also,
It is also possible to apply dynamic visualization information that allows information that is not originally visually recognized to be visually recognized by a predetermined method, such as when waves such as voices and sounds of nature are displayed on a device such as an oscilloscope. It is possible.

(2) Image Generation Conditions In the above embodiment, the light source LS is constant, but if other image generation conditions such as the background are also constant, the identification accuracy is further improved. For example, matching fingerprints, iris, etc./
If the image processing device is used to determine whether the person is the same person, such as a gate of a building that is opened / closed by determining whether the person is the same person or not, the target video is necessarily the same place. It will be taken in (near the doorway). Therefore, when the video database is constructed, if the video is captured at that location in advance, it is possible to perform highly accurate identification with a constant video generation condition such as the background.

(3) Video Capture Device As the video capture device 1, a plurality of digital video cameras may be used. By doing so, it is possible to generate an information-rich unique pattern and improve the identification accuracy. In addition, using multiple video capture devices, the above-mentioned universal amount extraction, registration and identification in the video database, etc. are performed on the video obtained by each video capture device, and the processing results are compared. Then, final identification may be performed, which also improves the identification accuracy. Further, in addition to the video captured by the video capturing device, audio information or the like may be used together, which also improves the identification accuracy.

[0084]

As described above, according to the present invention, the appearance frequency of each information value from the information value distribution of pixels in the image data of the dynamic visualization information, or each frame image forming the dynamic visualization information. Since it was decided to extract the Fourier frequency spectrum obtained by Fourier transforming the appearance frequency of each information value of the data in the frame movement direction as a universal amount of the dynamic visualization information, some information indicating the pixel in the image If the image data includes a value, the universal amount of the original dynamic visualization information can be extracted. Therefore,
It can be flexibly applied to a wider range of dynamic visualization information, and the processing ability of a computer for dynamic visualization information can be made closer to that of human visual information processing.

Further, since the distribution of information values is normalized to a distribution having a value within a certain range and the normalized appearance frequency for each information value is obtained to obtain the universal amount of dynamic visualization information, this The universal amount represents the relative frequency of appearance of the original information value in one image, does not depend on the number of pixels of the original image data, and the information value of each pixel is distributed in the original image. Lost location information. Therefore, according to the present invention, it is possible to extract a universal quantity that appropriately represents the original characteristic of the dynamic visualization information, without depending on the hardware or the situation when capturing the dynamic visualization information.

Further, when the image data containing the component values of the three primary colors of each pixel is used, a plurality of information value distributions obtained based on these component values are normalized,
Since the frequency of occurrence of each normalized information value is the universal quantity of the dynamic visualization information, the number of elements of the obtained universal quantity is the number of values that can be taken by the normalized information value. It is only the amount multiplied by (the number corresponding to the above-mentioned plurality), and as described above, it is extracted with a small amount of data that is a universal amount that appropriately represents the original feature of the dynamic visualization information. As a result, the amount of data handled by the computer can be reduced, and quick and accurate processing can be performed.

Since the image processing according to the present invention uses the above-mentioned universal amount of extraction, it can be applied to images of dynamic visualization information of various forms, and it can be applied accurately and quickly. Image identification can be performed on
It is possible to realize moving image recognition that is closer to the human visual information processing ability.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a moving image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a processing procedure for registering a unique pattern of a video by the moving image processing apparatus.

FIG. 3 is a flowchart showing a processing procedure for identifying a target video by the moving image processing apparatus.

FIG. 4 is a diagram showing an identification experiment result when the same color target image without a background moves differently in the same moving image processing apparatus.

FIG. 5 is a diagram showing an identification experiment result when different color target images having no background perform different movements in the same moving image processing apparatus.

FIG. 6 is a diagram showing an identification experiment result when a monochrome target image with a background moves differently in the moving image processing apparatus.

FIG. 7 is a diagram showing an identification experiment result when different persons actually perform different actions using the digital video camera in the same moving image processing apparatus.

FIG. 8 is a diagram conceptually showing how a pixel on a computer screen is projected onto RGB orthogonal coordinates in a conventional universal amount extraction method.

[Explanation of symbols]

1 Video capture device 2 electronic calculator 2a Unique pattern extraction unit 2b Identification calculation unit 2c Identification determination unit 2d result output section 3 storage devices 5 Output device LS light source VO object

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] December 12, 2001 (2001.12.
12)

[Procedure Amendment 1]

[Document name to be amended] Statement

[Name of item to be amended] Detailed explanation of the invention

[Correction method] Change

[Correction content]

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for extracting a universal quantity that can be processed by a computer from dynamic visualization information recognized by human vision, and further, utilizing the universal quantity, the dynamic visualization information. The present invention also relates to a moving image processing technique for identifying images.

[0002]

2. Description of the Related Art In this specification, visualization information refers to information represented in a form recognizable by human vision.
An image of an object captured by the human eye is a typical example of visualization information, and an image taken by a general camera or a digital camera corresponds to this. In addition, visualization of distributions or shapes that cannot normally be visually perceived, such as infrared images by a meteorological satellite, images by an electron microscope, magnetic domain images by the Kerr effect, etc., is one type of visualization information. In addition, even if information such as a sound wave of nature or a sound of nature is displayed on a device such as an oscilloscope, information that is originally recognized as a sense other than visual can be visually recognized as an image by a predetermined method. Is visualization information.

Further, in the present invention, the dynamic visualization information is movement of a person photographed by a photographing camera or a digital video, infrared image at each time by a meteorological satellite, migration image of virus by an electron microscope, and visualization information. I will say something that changes over time. That is,
An image taken by a general camera or a digital camera shows a section of dynamic visualization information that is obtained by cutting dynamic visualization information that changes with time at a specific time. In that sense, the dynamic visualization information is shown. Can be said to be a more generalized version of static visualization information.

More generally, the dynamic visualization information in this specification does not necessarily have to change along the time axis, and may be, for example, another axis that is not limited to the time axis. Visualization information that changes along with it is called dynamic visualization information. These pieces of dynamic visualization information differ depending on the conditions of the natural world when they are visualized, and even if the objects are the same, the dynamic visualization information that completely matches is not reproduced.

On the other hand, the human brain recognizes the dynamic information of the outside world by processing the audiovisual information, and the audiovisual information as the target includes a language or a score coded according to a certain configuration rule. There are coded audiovisual information and non-coded dynamic visualization information, that is, non-coded audiovisual information such as movies. The dynamic visualization information of such movies and animations is data based on the human visual information processing ability of processing the latter non-coded audiovisual dynamic information. It is considered that the universal amount of is sensed and recognized. Therefore, in order to realize human visual information processing ability in artificial intelligence by a computer, it is necessary to extract some universal amount of computer-processible dynamic visualization information from the dynamic visualization information.

Further, for example, in the case of performing person identification by image identification, it is easily expected that dynamic visualization information having a large amount of information can perform identification with higher accuracy than static visualization information. But in this regard, rather than a static photo, a so-called still photo,
It is empirically known that a movie can recognize a person much more accurately.

By the way, as a method for extracting a universal amount from a static image, pixel color components R (red), G (green) and B are used.
There is a method in which an image on a computer screen is projected onto a three-dimensional RGB orthogonal coordinate system having (blue) as a coordinate axis and the projected pattern is used as a universal amount of the image. FIG. 8 conceptually shows a projection operation performed by the universal quantity extraction method. The image displayed on the computer screen is a set of pixels arranged in a two-dimensional plane. Therefore, as shown on the left side of the figure, the position on the screen is represented by xy orthogonal coordinates, and the i-th pixel in the x direction. The position is xi, and the j-th pixel position in the y direction is yj. Further, the functions that give R, G, and B color components to the position are fr, fg, and fb, respectively,
The color component values of R, G, and B that form the pixel at the position (xi, yj) are fr (xi, yj) and fg (xi, y, respectively).
j) and fb (xi, yj).

Next, the pixel at the position (xi, yj) is projected onto the point on the RGB orthogonal coordinates to which the color component value corresponds. That is, as shown on the right side of the figure, in RGB rectangular coordinates, R = fr (xi, yj) and G = fg (xi, y
j) and the point (ro, gp, B = fb (xi, yj))
bq) corresponds to the pixel at the position (xi, yj), and the function fRGB (ro, gp, bq) at that point
The value of is incremented by one. Where o, p, q
Is an integer corresponding to each gradation of R, G, and B on the screen, ro is a color component value corresponding to the o-th gradation of R, and gp is a p-th gradation of G. The corresponding color component value, bq, represents the color component value corresponding to the q-th gradation of B. The function fRGB (ro, gp, b
The initial value of q) is 0 at all points.

Such a projection operation is performed for each pixel, and all the pixels at the position (xi, yj) corresponding to the pixel number I from 1 to x and the pixel number J from 1 to y are projected on the RGB orthogonal coordinates. . This gives the function fRGB (r
The value of each point of (o, gp, bq) represents the number of pixels having the corresponding color component value in the image on the screen, and forms a pattern unique to the image. In the above operation, the information about the geometrical coordinates of the static image is removed, and only the intensity distribution information of each RGB component is extracted. Normalize this pattern as needed,
By expressing it in a vector format, it is used as a universal quantity of the image and is used for image processing by a computer.

[0010]

However, according to human visual information processing ability, not only a color image having R, G and B color components but also one or two of these color components is used. It is also possible to recognize dynamic visualization information such as a displayed image or a gray image without color. On the other hand, in the universal amount extraction method for the image described above, each pixel is set to R, G
And B are projected onto one point on the RGB orthogonal coordinates according to the color component values. That is, the projection point is determined according to how the three primary colors are combined in each pixel, and the set of three color component values of R, G, and B that configure each pixel is used as one set of information. is doing. Therefore, the above-mentioned universal quantity extraction method basically targets universal quantity extraction from color images, and is suitable for handling visualization information of achromatic colors (grayscale) and visualization information represented as binary data. Absent.

In other words, the universal quantity obtained by the above universal quantity extraction method is the function fR projected on the RGB orthogonal coordinates.
Since it is composed of the values of each point of GB (ro, gp, bq), the number of elements of the universal quantity is the number obtained by multiplying the number of gradations of R, G and B, for example, R, G and B. If the number of gradations is 256, then 2563 is obtained. For this reason,
Since there is a large amount of universal data to be handled, it may be difficult to process it quickly and accurately by a computer. On the other hand, if the universal amount of the visualization information can be made into the light intensity information consisting of a single component, the number of elements of the universal amount is overwhelmingly small, and in the case of the above example, only 256 is used. You can

Furthermore, since the above method is intended for static visualization information, it cannot be used to extract a universal quantity from dynamic visualization information which is a set of a plurality of static visualization information. Furthermore, considering that the amount of information that dynamic visualization information has is equivalent to an extremely large amount of static visualization information, a huge amount of information processing is required unless it is a method unique to universal quantity extraction from dynamic visualization information. Therefore, in reality, it may be impossible for the computer to perform the calculation.

The present invention has been made in view of such circumstances, and can be flexibly applied to a wider range of dynamic visualization information, and the amount of data handled by a computer can be reduced to speed up the operation. To provide a new method for extracting a universal amount of dynamic visualization information that enables accurate and accurate processing, and makes it possible to bring the processing ability of dynamic visualization information by a computer closer to that of human visual information processing. It is an object.

Further, the present invention can be applied to moving images of various forms of dynamic visualization information by utilizing the universal quantity obtained by such a universal quantity extraction method, and
It is an object of the present invention to provide a moving image processing technique capable of realizing appropriate dynamic image recognition.

[0015]

In order to achieve the above object, a method for extracting a universal quantity of dynamic visualization information according to the present invention is
The appearance frequency for each information value from the information value distribution of pixels throughout the frame image data forming the dynamic visualization information, or the appearance frequency for each information value of each frame image data forming the dynamic visualization information A Fourier frequency spectrum obtained by performing Fourier transform in the frame moving direction is extracted as a universal amount of the dynamic visualization information.

More specifically, according to the universal quantity extraction method for dynamic visualization information based on the present invention, image data for each frame of a plurality of frames forming the dynamic visualization information is acquired, and for each frame. An appearance frequency distribution is obtained for information values defined for a plurality of pixels forming image data, and the appearance frequency distribution is orthogonally transformed in the frame direction to extract a universal amount of dynamic visualization information.

According to one embodiment of the present invention, in the process of acquiring image data in the above method, moving image data including component values of three primary colors forming each pixel of the frame is acquired.

According to another embodiment of the present invention, the information value is at least one of a light intensity value, a hue and a color component value.

According to still another embodiment of the present invention, the appearance frequency distribution is normalized before orthogonally transforming the appearance frequency distribution in the frame direction. The appearance frequency distribution is normalized so that the appearance frequency distribution for each frame has a value in a certain range. Alternatively, the normalization of the appearance frequency distribution is
It is performed so that the maximum value of the appearance frequency in all frames becomes a predetermined value.

According to one embodiment of the present invention, a discrete Fourier transform is used as the orthogonal transform in the frame direction. Further, the contents shown in the above-mentioned embodiments are not alternatives, and it is possible to combine them and carry out at the same time.

Further, according to another aspect of the present invention, a method for evaluating the similarity of objects based on the universal quantity extraction of dynamic visualization information is proposed. That is, the first universal quantity related to the dynamic visualization information of the reference object is extracted by the method described in any one of the above, and the second universal quantity related to the dynamic visualization information of the unknown object is extracted by the method described in any one of the above.
Is extracted, and the similarity between the reference object and the unknown object is evaluated based on the first universal quantity and the second universal quantity. Here, the method for extracting the first universal quantity and the method for extracting the second universal quantity may be the same or different.

According to still another aspect of the present invention,
A method to identify an object based on universal quantity extraction of dynamic visualization information is proposed. That is, the first universal quantity relating to the dynamic visualization information of a plurality of reference objects is extracted by the method described in any one of the above, and the first universal quantity regarding the dynamic visualization information of the unknown object is extracted by the method described in any one of the above. This is a method of identifying a target object and an unknown object by extracting 2 universal quantities and comparing the 1st universal quantity and the 2nd universal quantity. Here, as in the above, the method for extracting the first universal quantity and the method for extracting the second universal quantity do not necessarily have to be the same.

Furthermore, according to yet another aspect of the present invention, an apparatus for implementing the above method is proposed. That is, for a plurality of frames forming the dynamic visualization information, a means for acquiring image data for each frame, and an appearance frequency distribution for information values defined for a plurality of pixels forming the image data for each frame are displayed. A universal quantity extraction device for dynamic visualization information, comprising: a means for acquiring and a means for orthogonally transforming the appearance frequency distribution in a frame direction to extract a universal quantity of dynamic visualization information. The apparatus includes means for extracting a first universal quantity relating to dynamic visualization information of a reference object by any one of the methods described above, and dynamic visualization of an unknown object by any one of the methods described above. Similarity of objects including means for extracting a second universal quantity related to information, and means for evaluating the similarity between the reference object and the unknown object based on the first universal quantity and the second universal quantity It can be developed into an evaluation device. Further, the apparatus includes means for extracting a first universal quantity relating to the dynamic visualization information of a plurality of reference objects by the method according to any one of the above, and similarly, a means for extracting an unknown object by the method according to any one of the above. An object including means for extracting a second universal quantity related to dynamic visualization information, and means for identifying a target object and an unknown object by comparing the first universal quantity and the second universal quantity The identification device can be developed.

According to one specific embodiment of the present invention, first, the dynamic visualization information is taken in to obtain the moving image data of the dynamic visualization information, and the pixels are displayed in the display image by the moving image data. The distribution of information values included in the information that defines visible light is normalized to a distribution that takes a value within a certain range. Then, the appearance frequency of each normalized information value is obtained from the normalized distribution, and the obtained appearance frequency distribution or the appearance frequency of each information value of the frame image data that constitutes the dynamic visualization information is obtained. A Fourier frequency spectrum obtained by performing Fourier transform in the frame moving direction is extracted as a universal amount of the dynamic visualization information.

Here, as the moving image data, the component values of the three primary colors forming each pixel in the combined image obtained by combining all the frames forming the dynamic visualization information of an arbitrary number of frames so as not to overlap each other. You may use what contains. In this case, a plurality of information value distributions that determine the visible light to be displayed are obtained based on those component values, each of the plurality of distributions is normalized, and there is a normalized appearance frequency distribution of each information value. Hereinafter, this method is referred to as a synthetic image method.

Further, as the moving image data, data including component values of the three primary colors forming each pixel in each frame image forming the dynamic visualization information is used, and the display visible light is displayed based on these component values. Obtain multiple distributions of information values to be defined, normalize each of these multiple distributions, obtain the normalized appearance frequency distribution of each information value, and obtain the universal amount of the dynamic visualization information, that is, the information of each frame. As the value, the light intensity value, the hue, and the color component value of the pixel in each frame are combined and obtained. Next, a Fourier frequency spectrum in which the light intensity value, the hue, and the color component value of the pixels of all the frames are Fourier-transformed in the frame movement direction and the frame movement information phase is deleted may be used. In this case, the information value for which the distribution can be obtained is the Fourier frequency spectrum in which the light intensity value, the hue, and the color component value of the pixels through all the frames are mixed.

As the moving image data, data containing component values of the three primary colors forming each pixel in each frame image forming the dynamic visualization information is used, and the visible light is displayed based on those component values. Obtain multiple distributions of information values to be defined, normalize each of these multiple distributions, obtain the normalized appearance frequency distribution of each information value, and obtain the universal amount of the dynamic visualization information, that is, the information of each frame. As values, the light intensity value, the hue, and the color component value of the pixel in each frame are obtained. Next, the light intensity value, the hue, and the color component value of the pixel for all the frames, and the light intensity value, the hue, and the hue of the pixel through all the frames obtained by combining the light intensity value, the hue, and the color component value of the pixel respectively. It is also possible to use a Fourier frequency spectrum in which the frame shift information phase is deleted by Fourier transforming the color component values in the frame shift direction. The information values that can be obtained in this case include the light intensity value of the pixel through all frames, the hue and the color component value, and the respective Fourier frequency spectra.

On the other hand, in the image processing method according to the present invention, the universal quantity of each of the plurality of dynamic visualization information is extracted in advance by utilizing the universal quantity extraction method. Then, the universal quantity of the dynamic visualization information of the unknown target is extracted by a similar universal quantity extraction method, and the matching with each universal quantity extracted in advance is evaluated to obtain an image of the dynamic visualization information of the unknown object. Is identified as an image of any of the plurality of dynamic visualization information.

The image processing apparatus according to the present invention further comprises first extracting means for extracting in advance the universal amount of each of a plurality of pieces of dynamic visualization information using the universal amount extracting method, and the first extracting means.
Of the dynamic visualization information of the unknown object by the second extracting means for executing the same universal quantity extracting method as the first extracting means. Extract the amount. Then, as the identification means for identifying the image, the dynamic visualization information of the unknown target is evaluated by evaluating the matching between the universal quantity extracted by the second extraction means and the respective universal quantities stored in the storage means. Of the dynamic visualization information is identified as an image of any of the plurality of dynamic visualization information.

[0030]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. The present embodiment is an example in which part of human visual information processing capability is reproduced by a computer. According to human visual information processing ability, when an image captured by the naked eye is given as dynamic visualization information, the storage of the image information, the perception of the significance of the image information, and the recognition of the image information (other image information And the like) are performed. In order to realize these operations by a computer, in this embodiment described below, the video information is stored by extracting and recording a universal amount from a given video, and a linguistic description showing the significance of the video information is provided. The meaning of the video information is perceived by giving it, and the video information is recognized by evaluating the matching with the universal amount of the video information stored in advance.

<Structure> FIG. 1 is a diagram showing the structure of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus is composed of a video capture device 1, a computer 2, a storage device 3, an input device 4 and an output device 5 as shown in the figure.
An image representing a video of the object VO given as a processing target is handled.

The image capturing device 1 is a means for acquiring image information for capturing the image of the object VO, and is constituted by a digital video camera or the like, and color moving image data representing the captured image is used as the image information of the object VO. Electronic calculator 2
Supply to. The object VO may be any object that gives dynamic visualization information recognized by human vision, such as a human being, a car, etc., and the image capturing device 1 uses reflected light obtained by illuminating the object VO with the light source LS. Capture video by.
When capturing the respective images of the plurality of target objects VO, the same light source is always used as the light source LS so that the image generation conditions of the plurality of target objects VO are always constant (reflection depending on the light source). Make sure there is no change in information in the optical image).

Here, the color moving image data from the image capturing device 1 is data that forms an image (visualization information) by the image displayed on the computer screen, and is regularly displayed on the computer screen. It has information of visible light to be displayed in each pixel arranged in. That is, the video capturing device 1 displays the object V on the computer screen.
It is composed of a plurality of frames that form a moving image of O.
Color components R (red) and G of each pixel forming each frame
Information of (green) and B (blue) is supplied to the electronic computer 2 as the color moving image data. In such color moving image data, the video information in the display image depends on the resolution (total number of pixels) of the computer screen and the number of frames, and the information amount of each pixel of the image forming each frame is displayed on the computer screen. Although it depends on the dynamic range of displayable red component, green component, and blue component (range of gradation values that each color component can take), in the present embodiment, these are determined in accordance with the specifications of the electronic computer 2, Video capture device 1
Always supplies color moving image data having a resolution and a dynamic range according to a certain standard conforming to the specifications to the electronic computer 2.

The electronic computer 2 is a computer having predetermined arithmetic means, storage means, control means, etc. necessary for moving image information processing, and each processing operation by the moving image processing apparatus (described later).
It is configured by a computer or the like that has loaded a program for executing. The electronic computer 2 includes a unique pattern extraction unit 2 which is realized by an arithmetic means etc. by the program.
a, identification calculation unit 2b, identification determination unit 2c, and result output unit 2
d is provided as a configuration peculiar to the moving image processing apparatus. A storage device 3, an input device 4, and an output device 5 are connected to the electronic computer 2 as peripheral devices.

The unique pattern extraction unit 2a receives the color moving image data supplied from the image capturing device 1 and extracts the unique pattern of the image represented by the color moving image data.
The unique pattern extracted here corresponds to a universal amount of video (dynamic visualization information), and the unique pattern extraction unit 2a outputs the extracted unique pattern to the storage device 3,
Alternatively, it is supplied to the identification calculation unit 2b. The specific content of the unique pattern will be clarified in the operation description to be described later together with the extraction processing.

The identification calculation unit 2b includes a unique pattern extraction unit 2
An operation for identifying the unique pattern supplied from a as one of the previously stored unique patterns is performed, and the identification determination unit 2c determines the specific unique pattern identified based on the operation result. . Result output unit 2d
Receives the calculation result of the identification calculation unit 2b and the determination result of the identification determination unit 2c, and outputs a predetermined result to the output device 5. Note that the specific contents of the processing by the identification calculation unit 2b, the identification determination unit 2c, and the result output unit 2d will be described in detail later in the operation description.

The storage device 3 records and registers (holds) the unique pattern output from the unique pattern extraction unit 2a. By performing this registration for images of a plurality of objects VO, an image database in which the unique patterns of those images are accumulated is built in the storage device 3. When the identification calculation unit 2b refers to the unique pattern stored in advance, the image database in the storage device 3 is used.

The input device 4 is an input device of the electronic computer 2 which is composed of a keyboard, a pointing device and the like. The input device 4 is used to give a linguistic description representing the meaning of the original video information to the unique pattern registered in the storage device 3 (for example, the object VO is a human being, an animal, or the like). Enter a predetermined linguistic description such as). As a result, the unique pattern extraction unit 2a
To output the meaning of the original video information together with the unique pattern to the storage device 3 so that the meaning of the video information can be perceived. However, the input of the linguistic description is an arbitrary operation, and the operator may appropriately input it.

The output device 5 is an external output device of the electronic computer 2 constituted by a display device, a printer, etc., and the resolution of the display device depends on the specifications of the electronic computer 2 on which the video information in the above-mentioned display image depends. It corresponds to.
The dynamic range on which the information amount of each pixel depends depends on the specification (display setting) of the electronic computer 2 according to the standard such as 24-bit full color so that it can be displayed on the display device of the output device 5. The amount is predetermined.

<Operation> (1) Registration of Unique Pattern of Video Next, the operation of the moving image processing apparatus having the above configuration will be described. FIG. 2 is a flowchart showing a processing procedure for registering a unique pattern of an image. This process is performed on the object V
This is a process of extracting a unique pattern from the O image and registering it in the storage device 3 described above.

When registering the unique pattern of the image, first, the image of the object VO to be made into a database image to be stored in advance is captured by the image capturing device 1 (step S). As a result, the color image data representing the image of the object VO is supplied from the image capturing device 1 to the electronic computer 2.

Here, the color moving image data from the image capturing device 1 is composed of a plurality of frame images forming a moving image of the object VO, each frame image is composed of pixels, and the color component R of each pixel is It has information of (red), G (green), and B (blue). In a display image based on such color moving image data, each pixel displays visible light determined by the combination of reflected light intensity, hue, and color component, and video information is expressed by their geometrical arrangement. Therefore,
In the present moving image processing apparatus, three elements of reflected light intensity, hue, and color component for one frame image in such video information are used as constituent elements of the basic unique pattern, and the three elements are histogram-formed. The light intensity eigenvector, the hue eigenvector, and the color component eigenvector are calculated by arithmetic processing in the eigenpattern extraction unit 2a based on the color image data from the video capturing device 1 (steps S1 to S4).

The reflected light intensity eigenvectors, hue eigenvectors, and color component eigenvectors of one frame image of a moving image or a composite image (composite image method described in paragraph 0013) in which all frames are composited so as not to overlap are described below. The calculation processing in the unique pattern extraction unit 2a will be sequentially described. Although the order of description is in accordance with the step progress of FIG. 2, the actual arithmetic processing in the computer does not necessarily have to be performed in this order, and can be changed as appropriate.

Calculation of reflected light intensity eigenvector (step S2) The reflected light intensity eigenvector is a value corresponding to the frequency of each class when a histogram is created from the distribution of the reflected light intensity in the display image with each value of the reflected light intensity as a class. It is a vector that is an element. The reflected light intensity value of the i-th pixel in one frame of color image data is Iint, i, and the color component values of R, G, and B of the same pixel are Ri and G, respectively.
Given i and Bi, the reflected light intensity value Iint, i is given by the scalar sum or vector sum of the color component values Ri, Gi and Bi. That is,

[Equation 1] Or

[Equation 2] (Whether the reflected light intensity value is obtained by any of the equations 1 and 2 is arbitrary and may be appropriately selected).

Therefore, if the reflected light intensity distribution in the display image based on the color image data of one frame is Iint, the reflected light intensity distribution Iint is

[Equation 3] Given in. T is the total number of pixels in one display image, and the unique pattern extraction unit 2a performs the calculation of Formula 1 or Formula 2 for each pixel from the first to the Tth, and the reflected light intensity of Formula 3 is obtained. Obtain the distribution Iint.

Next, the reflected light intensity distribution Iint is a distribution of the dynamic range D (a distribution in which the reflected light intensity value takes a value in the range from 0 to a predetermined integer D. Hereinafter, this distribution is represented by the symbol "IintD", "Normalized reflected light intensity distribution Iin
tD ”. ). Here, "Round [a]
"Indicates the operation of converting the value" a "in parentheses into an integer, and" Ma
Suppose that x [A] ”represents an operation for finding the maximum value element in the set“ A ”in parentheses, the unique pattern extraction unit 2
a transforms the reflected light intensity distribution Iint into the normalized reflected light intensity distribution IintD by the following equation 4.

[Equation 4] The dynamic range D is appropriately selected in consideration of the hue and the resolution of color components described later. For example, when the gradation of the color component has the lowest resolution and the number of gradations is 256, the value of D is set to 256, for example.

Thereafter, the obtained normalized reflected light intensity distribution I
At intD, the unique pattern extraction unit 2a counts the number of pixels that take each (normalized) reflected light intensity value from 0 to D. Thereby, the reflected light intensity distribution normalized by the dynamic range D is histogrammed with the reflected light intensity value as a class, and the appearance frequency of each (normalized) reflected light intensity value corresponding to the frequency of each class is obtained. The peculiar pattern extraction unit 2a sets the appearance frequency of each reflected light intensity value thus obtained as array data in a vector format to obtain a reflected light intensity eigenvector Eint. The reflected light intensity eigenvector Eint is an eigenvector obtained for one image.

Calculation of Hue Eigenvector (Step S3) The hue eigenvector is a vector whose element is a value corresponding to the frequency of each class when a histogram is created with the value of each hue as the class from the distribution of the hue in each pixel in the display image. Is. Hue here means R, G, and B in each pixel.
Is the composition ratio of the color component values of R, G, or B and is represented by the ratio of the R, G, or B color component values to the reflected light intensity value. Since the sum of these three color component values is given by the reflected light intensity value, it is the hue for any two of the color components R, G and B that has independent information for the reflected light intensity value. is there.

The hue distribution in the display image for each color component represents the color component composition ratio in each pixel in the display image. For example, when considering the hue distribution of the R component, the hue distribution of the R component in the display image is ITo
Let ne and R be the reflected light intensity values Iint,
By the color component values Ri of i and R,

[Equation 5] Given in. In this case, the unique pattern extraction unit 2a
The right side of the equation (5) is divided for each of the first to Tth pixels to obtain the R component hue distribution Itone, R. Note that the R component hue in each pixel corresponds to the division formula for each pixel on the right side of Equation 5, and "Ri / Iint, i" described is the R component hue of the i-th pixel (hereinafter, this The R component hue of the i-th pixel is represented by the symbol "ITone, R, i").

Next, the R component hue distribution Itone, R is normalized to the distribution of the dynamic range D. This normalization calculation is similar to the above-described transformation of the reflected light intensity distribution Iint into the normalized reflected light intensity distribution IintD. That is, if the normalized R component hue distribution IDTONE, R is normalized to a distribution in which the R component hue of each pixel takes a value in the range from 0 to D, then IDTONE, R
The unique pattern extraction unit 2a converts the R component hue distribution IDTONE, R into a normalized R component hue distribution ID according to the following equation 6.
Transforms into TONE and R.

[Equation 6]

Then, the obtained normalized R component hue distribution I
In DTONE, R, the unique pattern extraction unit 2a
Count the number of pixels taking each (normalized) R component hue from 0 to D. Thereby, the hue distribution of the R component normalized by the dynamic range D is histogrammed with the R component hue as a class, and the appearance frequency of each R component hue corresponding to the frequency of each class (normalized) is obtained. The peculiar pattern extraction unit 2a sets the appearance frequency of each R component hue thus obtained as array data in a vector format to be the R component hue eigenvector ETone, R.

The G component hue eigenvectors ETone, G and the B component hue eigenvectors ETone, B can also be obtained by the same calculation processing as described above (in the calculation processing for obtaining the R component hue eigenvectors ETone, R described above,
If the arithmetic processing indicated by replacing "R" with "G" and "R" with "B" is performed, G component hue eigenvectors ETone, G, B component hue eigenvectors E respectively.
Tone, B is obtained). Unique pattern extraction unit 2a
Are R component hue eigenvectors ETone, R, G component hue eigenvectors ETone,
Two sets of eigenvectors of the G and B component hue eigenvectors ETone, B are calculated. This is because the reflected light intensity eigenvector Eint includes information about the sum of color component values, and the degree of freedom is reduced by one. 2
Which combination is used as the eigenvector of the set is arbitrary and may be appropriately selected.

Calculation of color component eigenvector (step S
4) The color component eigenvector is R, G or B in the display image.
Is a vector whose elements are the values corresponding to the frequencies of each class when a histogram is created from each color component distribution as a class of each color component value. The color component eigenvectors are given by the amounts of R component, G component and B component in the display image, and three sets are obtained for one video. The geometric information of the image is gently held as the ratio of the R component, the G component and the B component.

When considering the R component distribution in the display image, if the R component distribution is IComp, R,

[Equation 7] Is. The peculiar pattern extraction unit 2a uses the R component distribution I
Comp, R are normalized to the distribution of the dynamic range D. The normalization calculation here is also the above-mentioned reflected light intensity distribution Ii.
This is the same as the transformation of nt to the normalized reflected light intensity distribution IintD. That is, when the normalized R component distribution IComp, R is normalized to a distribution in which the R component value of each pixel takes a value in the range of 0 to D, IComp, RD
The peculiar pattern extraction unit 2a converts the R component distribution IComp, R into the normalized R component distribution IComp, R according to the following equation 8.
Transforms into D.

[Equation 8]

Then, the obtained normalized R component distribution ICo
In mp and RD, the unique pattern extraction unit 2a counts the number of pixels that take each (normalized) R component value from 0 to D. As a result, the R component distribution normalized by the dynamic range D is histogrammed with the R component value as a class, and the appearance frequency of each (normalized) R component value corresponding to the frequency of each class is obtained. The eigenpattern extraction unit 2a sets the appearance frequency of each R component value thus obtained as array data in a vector format to be an R component eigenvector EComp, R.

The G component eigenvectors EComp, G and the B component eigenvectors EComp, B can also be obtained by the same calculation process as described above (in the calculation process for obtaining the R component eigenvectors EComp, R, "R" is changed to "G", If the arithmetic processing indicated by replacing “R” with “B” is performed, the G component eigenvector E
Comp, G, B component eigenvectors EComp, B are obtained).

The eigenpattern extraction unit 2a calculates each eigenvector based on the color image data (R, G and B component values forming each pixel) from the video capturing device 1 as described above. Then, the calculated eigenvector is registered in the storage device 3 as the eigenpattern of the image represented by the color image data (step S5).

When this eigenpattern is E, the eigenpattern E is the reflected light intensity eigenvector Eint, R component hue eigenvectors ETone, R, B component hue eigenvector ETone, B, R component eigenvector ECom.
p, R, G component eigenvectors EComp, G and B component eigenvectors EComp, B,

[Equation 9] Alternatively, the unique pattern according to the number 9 of each frame is set to Ej, j
= 1,2, .., n, (n is the number of frames), | a | is the absolute value of "a", and Fourier (A) is the Fourier transform of "A",

[Equation 10] Alternatively, the constituent vector of the eigenpattern E based on the number 9 of each frame is defined as Eintj, ETone, Rj, ETone, Bj, ECom.
If p, Rj, EComp, Gj, EComp, Bj, j = 1,2, .., n, (n is the number of frames),

[Equation 11] (Where T is a symbol representing the transpose of the matrix). Note that the R component hue eigenvector ETon
Instead of e, R and B component hue eigenvectors ETone, B, R component hue eigenvectors ETone, R and G component hue eigenvectors ETone, G or G component hue eigenvectors ETone, G and B component hue eigenvectors ETone, G
One and B may be used as described above.

The eigenpattern extraction unit 2a arranges the elements of the calculated eigenvectors in the form of the right side in any of the equations 9 to 11 to generate the eigenpattern E, outputs it to the storage device 3, and records it. At this time, the color moving image data used in the above-described arithmetic processing is also output from the electronic computer 2 to the storage device 3 and recorded in association with the unique pattern E. Thereby, the unique pattern is extracted from the image of the target VO captured in step S and registered in the storage device 3, and the registration of the unique pattern of the image of the target VO is completed.

In the present moving image processing apparatus, the above-described unique pattern registration processing is performed on the object VO of a certain type (number).
Is performed in advance to construct a video database in which unique patterns of a plurality of videos are stored in the storage device 3. Then, the identification process is performed on the image of the unknown object VO (hereinafter referred to as “target image”) given as the recognition processing target as follows.

Identification of Target Video FIG. 3 shows a procedure of processing for identifying a target video. In this process, first, the target image is captured by the image capturing device 1 (step S10). As a result, the color moving image data representing the target video is supplied from the video capture device 1 to the electronic computer 2. The color moving image data here has information on the color components R (red), G (green), and B (blue) of each pixel forming the image of the unknown object VO that is the recognition processing target. , And is sent to the unique pattern extraction unit in the electronic computer 2.

The unique pattern extraction unit 2a that has received the color moving image data extracts the unique pattern of the target video by the unique pattern extraction processing similar to the above (step S).
11). That is, the received color moving image data is used to perform the arithmetic processing of steps S 1 to S 5 in “(1) Registration of unique pattern of video”, and the reflected light intensity eigenvector Eint and R component hue based on the color moving image data. Eigenvector ETone, R, G component hue eigenvector ETone, G, R component eigenvector ECom
The p, R, G component eigenvectors EComp, G and the B component eigenvectors EComp, B are calculated. Then, the elements of these eigenvectors are arranged in the shape of the right side by any of the above equations 9 to 11 to form the eigenpattern of the target video.

The unique pattern of the target image obtained in this way is supplied from the unique pattern extraction section 2a to the identification calculation section 2b, and the identification calculation section 2b executes the identification calculation (step S12). This identification calculation is a calculation for identifying the target image as one of the images stored in advance,
The matching between the unique pattern of the target video and the unique pattern registered in the video database of the storage device 3 is evaluated as follows.

In the matching evaluation of the unique pattern, a linear system equation representing the unique pattern of the target video is considered by linear combination of the unique patterns registered in the video database. Now, assuming that m unique patterns are registered in the video database, the system matrix (hereinafter represented by the symbol “C”) is

[Equation 12] Given in. Therefore, if the characteristic pattern of the target image is EX, the following linear system equation is obtained.

[Equation 13] X in Expression 13 is an m-th order vector whose elements are the weights of the individual patterns of the video database. here,
The vector X satisfying the equation 13 has the j-th element as 1, and
If all other elements are set to 0, the unique pattern EX is equal to the unique pattern Ej of the video database, and the target video can be identified as the video of the unique pattern Ej. That is, identifying the target video as a prestored video results in an inverse problem of finding the vector X in Equation 13 (for this reason, the vector X is hereinafter referred to as a “solution vector X”).

The identification calculation unit 2b includes the unique pattern extraction unit 2
While receiving the unique pattern EX of the target video from a, the registered m unique patterns are read from the storage device 3 to obtain the system matrix C. Therefore, in the identification calculation in the identification calculation unit 2b, EX and C
Since the solution vector X may be obtained with known as, the solution vector X can be calculated by using the solution method of the inverse problem used in general multivariate analysis and the like.

In this embodiment, the least squares method is used as an example of the solution. Since the eigenpattern is composed of the six eigenvectors described above, and each eigenvector is a histogram of a distribution normalized by the dynamic range D, the order of the eigenpattern is 6 times the dynamic range D. . Given that this order is n, it can be considered that the order n is usually larger than the number m of unique patterns registered in the video database, so that the system matrix C becomes a rectangular matrix with n rows and m columns. 1
3 becomes an inappropriate linear system equation), and the least squares method can be applied to derive the solution vector X. Therefore, the identification calculation unit 2b uses the eigenpattern EX from the eigenpattern extraction unit 2a and the system matrix C obtained from the storage device 3 to execute the following calculation by the least squares method to obtain the solution vector X. calculate.

[Equation 14] It is needless to say that the inappropriate solution of the linear system of the equation 13 is not limited to the equation 14 and is merely adopted as one concrete method here.

The solution vector X calculated by the identification calculation section 2b as described above is obtained by the identification determination section 2c and the result output section 2d.
The unique pattern EX of the target video is also supplied to the result output unit 2d. Then, the identification determination unit 2c determines to identify a specific unique pattern based on the solution vector X (step S13). For example, if one element of the solution vector X is 1 and all other elements are 0, the identification determination unit 2c sets the unique pattern of the video database weighted by the 1 element as the unique pattern of the target video. Judge as something. Further, in the determination here, when a certain element of the solution vector X is equal to or more than a predetermined constant value and another element is equal to or less than another predetermined constant value, the element having the former constant value or more is weighted. The unique pattern to be added may be determined to be the unique pattern of the target video.

The determination result of the identification determination unit 2c is supplied to the result output unit 2d, and the result output unit 2d outputs a predetermined result together with the solution vector X and the unique pattern EX from the identification calculation unit 2b ( Step S14). That is, the result output unit 2d reads the unique pattern of the identified video database and the color moving image data recorded corresponding thereto from the storage device 3 based on the determination result from the identification determination unit 2c, and identifies them as those data. Computing unit 2b
And outputs the data of the solution vector X and the unique pattern EX from the output device 5. As a result, in the output device 5, the unknown object VO given as the recognition processing target is identified from the unique pattern EX of the image, the solution vector X of the identification calculation result, and the image stored in advance. The information such as the captured image and its unique pattern is displayed on the display of the display device or the printout of the printer, and the identification of the target image is completed.

The above is the configuration and operation of the moving image processing apparatus. As described above, in the present moving image processing apparatus, the distribution of reflected light intensity, hue and color components in an image in which an image is captured is normalized in a certain dynamic range and then histogrammed to obtain a unique pattern of the image. . Therefore, this unique pattern represents the relative frequency of appearance of each value of the reflected light intensity, the hue, and the color component in one image, and does not depend on the number of pixels when the video is converted into a moving image. That is, the peculiar pattern is information in which the original image information is formed into a histogram and transformed into a ratio with respect to the total number of pixels to eliminate the dependency on the number of pixels limited by hardware such as a computer screen. ing.

Further, in the above-mentioned extraction of the unique pattern,
Since the distribution of the reflected light intensity and the like is made into a histogram for each normalized class, the information of the position where the reflected light intensity and the like are distributed in the original image is lost. Therefore, the unique pattern is information that does not depend on the reference coordinates in the original image, and is extracted as information that represents the image regardless of the viewpoint when capturing the image, and the geometric shape change of the image is detected. It will not be affected by changes in the layout.

As described above, the unique pattern extracted by the moving image processing apparatus is the original feature amount of the image that does not depend on the hardware or the situation at the time of capturing the image, and is a universal pattern unique to the target image. Represents a quantity, which is an appropriate universal quantity extracted from visualization information that is not reproducible even for the same target. In this universal amount, the reflected light intensity, the hue of each color component, and each color component are independent components, and are treated as independent information. Therefore, if information is obtained even for one of these components, it is possible to extract the universal quantity and perform the identification as described above. That is, the present moving image processing apparatus can be realized for a single color component of R, G or B, a single hue of R component, G component or B component, or reflected light intensity alone, in other words, for a monochrome moving image, It can be flexibly applied to a wider range of visualization information.

Further, in the conventional universal quantity extraction method by projecting onto RGB orthogonal coordinates, the number of elements of the universal quantity is R,
The number of gradations of G and B (the number of gradations to the third power) is multiplied, whereas in the above-mentioned unique pattern, the order is 6 times the dynamic range D (the number of elements is 6 times the number of gradations, etc.). ), The number of elements of the universal quantity is smaller than before. Therefore,
According to this image processing apparatus, as described above, the universal amount that appropriately represents the original feature of the image is extracted with a smaller amount of data than before and is used for the identification process. It can be performed.

<Experimental Example> An experimental example for confirming the accuracy of moving image recognition of the present moving image processing apparatus will be shown below. In the following experimental example, the same digital video camera is used as the image capturing device 1, and the object V for constructing the image database V is used.
As O, an experiment was conducted using four types of color animation images, four types of monochrome animation images, walking motions of 11 humans, and moving images of the first half of the body. This is because the animation image assumes a moving image shot under an ideal environment. In order to examine whether it is possible to identify differences in human clothes during walking motion in the moving images of 11 humans, two 4 moving images with different backgrounds were adopted. Video images of three humans were adopted to identify the upper body movements. In the identification of the target video, the identification determination is performed by determining the element having the maximum value among the elements of the solution vector X.

Target Video Identification Experiment FIG. 4 is a diagram showing the results of an identification experiment when a color animation image in which the same target moves in different directions is used as the target video. In this figure, the upper part is an animation image in which the same target is moving in different directions.
The lower part is a graph showing the element values of the solution vector X obtained by using the Fourier frequency spectrum peculiar pattern (Equation 10) with respect to the frame moving direction (these arrangements are the same for the identification experiment results described later). . The purpose of this experiment is to examine whether or not the same target image can be identified using the Fourier frequency spectrum peculiar pattern (Equation 10) with respect to the frame moving direction even if the moving direction is different.

In this identification experiment, in the solution vector X as shown in the lower graph, the No.
The element of (number) is 1, and the others are zero, and the effectiveness of the Fourier frequency spectrum peculiar pattern (Equation 10) in the frame movement direction can be confirmed.

Target Video Identification Experiment FIG. 5 is a diagram showing a result of an identification experiment when a color animation image in which different targets move in different directions is used as the target video. In this figure, the upper row is one of the database moving images in which different objects are moving in different directions.
It is a frame image. The middle row is a 1-frame image of the sample animation in which the target image moves in different directions.
The lower part is a graph showing the element values of the solution vector X obtained by using the Fourier frequency spectrum peculiar pattern (Equation 11) in the frame movement direction. Even when different targets move in different directions, it can be seen that individual target images can be identified from the element having the maximum value of the solution vector X. The purpose of this experiment is to examine whether it is possible to identify the target image using the Fourier frequency spectrum peculiar pattern (Equation 11) with respect to the frame moving direction even if the moving direction of the target image is different. In this sense, the effectiveness of the Fourier frequency spectrum peculiar pattern (Equation 11) in the frame movement direction can be confirmed.

Target Video Identification Experiment FIG. 6 is a diagram showing the results of an identification experiment when a monochrome animation image with a background in which different targets move in different directions is used as the target video. In this figure, the upper part is one frame image of a monochrome database moving image with a background in which different objects are moving in different directions. The middle row is a one-frame image of a monochrome test animation with a background in which the target image moves in different directions. The lower row is
Solution vector X obtained by using the Fourier frequency spectrum proper pattern (Equation 11) for the frame movement direction
3 is a graph showing the element values of and the element values of a solution vector X by a combining method that combines all frames into one image. It can be seen that even in a monochrome moving image with a background, individual target images can be identified from the element having the maximum value of the solution vector X even when different targets move in different directions.
The result of this experiment is a monochrome moving image with a background. Even if the moving directions of different target images are different, the Fourier frequency spectrum peculiar pattern (Equation 11) and the synthetic image method (paragraph 0013) are used for the frame moving direction. The point is to examine whether they can be identified. In this sense, the effectiveness of the Fourier frequency spectrum eigenpattern (Equation 11) and the eigenpattern (Equation 9) based on the composite image method (paragraph 0013) in the frame movement direction can be confirmed.

Target Video Identification Experiment FIG. 7 is a diagram showing the results of an identification experiment when the motions of the first half of the body of different persons were photographed using an actual digital video camera. In this figure, the upper row shows 1 of the color database moving image with background in which different objects are performing different actions.
It is a frame image. The middle row is a one-frame image of a color sample animation with a background in which the target is performing different actions. The lower part is a graph showing the element values of the solution vector X by the composite image method (paragraph 0013) in which all frames are composited into one image. From the element that takes the maximum value of the solution vector X, it is possible to identify even if the same person performs different actions, and the validity of the unique pattern (Equation 9) according to the composite image method (paragraph 0013) can be confirmed.

<Examples of Modifications / Applications> Although one embodiment of the present invention has been described above, the embodiment of the present invention is not limited to the above. Hereinafter, other modifications or applications will be described with some examples.

(1) Applicable Visualization Information As the object of universal information extraction and identification processing, the image of the object captured by the human visual sense is used in the above-mentioned embodiment, but the present invention is not limited to this. Visualization information can be used. In particular, in the universal quantity extraction based on the present invention, since the reflected light intensity, the color component, etc. are treated as separate and independent information as described in the above embodiment,
Not only normal color images taken by a general camera or video recorder, but also images in other forms can be targeted.

For example, distributions or shapes that cannot normally be visually recognized, such as infrared rays images by a meteorological satellite, images by an electron microscope, images by MRI, fluid images by a tracer, magnetic domain images by the Kerr effect, etc. The visualized dynamic visualization information may be applied. Also,
It is also possible to apply dynamic visualization information that allows information that is not originally visually recognized to be visually recognized by a predetermined method, such as when waves such as voices and sounds of nature are displayed on a device such as an oscilloscope. It is possible.

(2) Image Generation Conditions In the above embodiment, the light source LS is constant, but if other image generation conditions such as the background are also constant, the identification accuracy is further improved. For example, matching fingerprints, iris, etc./
If the image processing device is used to determine whether the person is the same person, such as a gate of a building that is opened / closed by determining whether the person is the same person or not, the target video is necessarily the same place. It will be taken in (near the doorway). Therefore, when the video database is constructed, if the video is captured at that location in advance, it is possible to perform highly accurate identification with a constant video generation condition such as the background.

(3) Video Capture Device As the video capture device 1, a plurality of digital video cameras may be used. By doing so, it is possible to generate an information-rich unique pattern and improve the identification accuracy. In addition, using multiple video capture devices, the above-mentioned universal amount extraction, registration and identification in the video database, etc. are performed on the video obtained by each video capture device, and the processing results are compared. Then, final identification may be performed, which also improves the identification accuracy. Furthermore, in addition to the video captured by the video capturing device 1, audio information or the like may be used together, which also improves the identification accuracy.

[0084]

As described above, according to the present invention, the appearance frequency of each information value from the information value distribution of pixels in the image data of the dynamic visualization information, or each frame image forming the dynamic visualization information. Since it was decided to extract the Fourier frequency spectrum obtained by Fourier transforming the appearance frequency of each information value of the data in the frame movement direction as a universal amount of the dynamic visualization information, some information indicating the pixel in the image If the image data includes a value, the universal amount of the original dynamic visualization information can be extracted. Therefore,
It can be flexibly applied to a wider range of dynamic visualization information, and the processing ability of a computer for dynamic visualization information can be made closer to that of human visual information processing.

Further, since the distribution of information values is normalized to a distribution having a value within a certain range and the normalized appearance frequency for each information value is obtained to obtain the universal amount of dynamic visualization information, this The universal amount represents the relative frequency of appearance of the original information value in one image, does not depend on the number of pixels of the original image data, and the information value of each pixel is distributed in the original image. Lost location information. Therefore, according to the present invention, it is possible to extract a universal quantity that appropriately represents the original characteristic of the dynamic visualization information, without depending on the hardware or the situation when capturing the dynamic visualization information.

Further, when the image data containing the component values of the three primary colors of each pixel is used, a plurality of information value distributions obtained based on these component values are normalized,
Since the frequency of occurrence of each normalized information value is the universal quantity of the dynamic visualization information, the number of elements of the obtained universal quantity is the number of values that can be taken by the normalized information value. It is only the amount multiplied by (the number corresponding to the above-mentioned plurality), and as described above, it is extracted with a small amount of data that is a universal amount that appropriately represents the original feature of the dynamic visualization information. As a result, the amount of data handled by the computer can be reduced, and quick and accurate processing can be performed.

Since the image processing according to the present invention uses the above-mentioned universal amount of extraction, it can be applied to images of dynamic visualization information of various forms, and it can be applied accurately and quickly. Image identification can be performed on
It is possible to realize moving image recognition that is closer to the human visual information processing ability.

[Procedure Amendment 2]

[Document name to be amended] Statement

[Name of item to be corrected] Brief description of the drawing

[Correction method] Change

[Correction content]

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a moving image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a processing procedure for registering a unique pattern of a video by the moving image processing apparatus.

FIG. 3 is a flowchart showing a processing procedure for identifying a target video by the moving image processing apparatus.

FIG. 4 is a diagram showing an identification experiment result when the same color target image without a background moves differently in the same moving image processing apparatus.

FIG. 5 is a diagram showing an identification experiment result when different color target images having no background perform different movements in the same moving image processing apparatus.

FIG. 6 is a diagram showing an identification experiment result when a monochrome target image with a background moves differently in the moving image processing apparatus.

FIG. 7 is a diagram showing an identification experiment result when different persons actually perform different actions using the digital video camera in the same moving image processing apparatus.

FIG. 8 is a diagram conceptually showing how a pixel on a computer screen is projected onto RGB orthogonal coordinates in a conventional universal amount extraction method.

[Explanation of symbols] 1 Video capture device 2 electronic calculator 2a Unique pattern extraction unit 2b Identification calculation unit 2c Identification determination unit 2d result output section 3 storage devices 5 Output device LS light source VO object

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C066 AA13 CA00 EF00 GA01 KD01                       KD06 KE01 KE03 KE09                 5L096 AA02 AA06 EA12 FA23 FA35                       HA02

Claims (11)

[Claims] 1. A process of acquiring image data for each frame of a plurality of frames forming dynamic visualization information, and an appearance of information values defined for a plurality of pixels forming the image data of each frame. A method for extracting a universal quantity of dynamic visualization information, comprising a step of obtaining a frequency distribution and a step of orthogonally transforming the appearance frequency distribution in a frame direction to extract a universal quantity of dynamic visualization information. 2. The universal method according to claim 1, wherein the process of acquiring the image data is a process of acquiring moving image data including component values of three primary colors forming each pixel of the frame. Quantity extraction method. 3. The method according to claim 1, wherein the information value is at least one of a light intensity value, a hue, and a color component value. 4. The method according to claim 1, further comprising normalizing the appearance frequency distribution before orthogonally transforming the appearance frequency distribution in a frame direction. 5. The process of normalizing the appearance frequency distribution comprises:
5. The method according to claim 1, wherein the appearance frequency distribution for each frame is normalized so as to take a value in a certain range. 6. The process of normalizing the appearance frequency distribution comprises:
5. The method according to claim 1, wherein normalization is performed so that the maximum value of the appearance frequency in all frames becomes a predetermined value. 7. The method according to claim 1, wherein the orthogonal transform in the frame direction is a discrete Fourier transform. 8. A process of extracting a first universal quantity relating to dynamic visualization information of a reference object by the method according to claim 1, and the method according to claim 1. A process of extracting a second universal quantity related to the dynamic visualization information of the unknown object by the method, and a method of evaluating the similarity between the reference object and the unknown object based on the first universal quantity and the second universal quantity. 9. A process of extracting a first universal quantity relating to dynamic visualization information of a plurality of reference objects by the method according to claim 1, and the method according to claim 1. And a method of identifying a target object and an unknown object on the basis of the first universal quantity and the second universal quantity. 10. A process for extracting a first universal quantity related to dynamic visualization information of a plurality of reference objects by the method according to claim 1, and a method according to claim 1. The method of extracting the second universal quantity relating to the dynamic visualization information of the unknown object by the method of 1. and the method of identifying the common unknown object image with high accuracy by convolution of the identification results by using them independently. 11. A unit for acquiring image data for each frame for a plurality of frames forming dynamic visualization information, and an information value defined for a plurality of pixels forming the image data for each frame. A universal quantity extraction device for dynamic visualization information, comprising: means for acquiring a frequency distribution; and means for orthogonally transforming the appearance frequency distribution in a frame direction to extract a universal quantity of dynamic visualization information.