WO2013128839A1 - Image recognition system, image recognition method and computer program - Google Patents

Description

Image recognition system, image recognition method, and computer program

The present invention relates to an information processing system technology that can be used for image recognition.

The detection in the image recognition system is to identify and specify an image area belonging to a specific category such as a pedestrian or a vehicle from the other image areas on the input image. The tracking in the image recognition system is to continuously specify the tracking target acquired by the above detection or the tracking target reflected in the artificially specified image area using time-series images. .

An image recognition system using an information processing apparatus is used for recognizing an image shot using an in-vehicle camera or the like. Patent Documents 1 to 3 disclose examples of related image recognition systems.

The image recognition system disclosed in Patent Document 1 is an image recognition system that uses both detection and tracking. The image recognition system includes a person detector and a person tracker that operate independently of each other. The person detector registers the person image area detected at each time as a tracking target. The person tracking device tracks the registered tracking target according to time series. As a result, this image recognition system detects a person from the background image, and specifies the locus of movement of the detected person.

The object tracking method disclosed in Patent Document 2 can accurately track an object even when an object similar to the object exists around the object. For this purpose, an object tracking device that employs the above method specifies a positional relationship determination unit that determines a positional relationship in which similar objects exist in the preceding and following images in the captured video space, and specifies a position in which the similar objects exist. A similar object position specifying unit. If the tracking device can identify the position of the object from the movement of the object, the tracking device identifies the position of the object from the movement. On the other hand, if the tracking device cannot be identified from the movement of the object, the tracking device searches the image for the position of the object.

The object tracking device disclosed in Patent Document 3 is a device that reduces the possibility of erroneous tracking. The object tracking device first calculates a feature quantity representing an image and a feature quantity representing a particle. Next, the tracking device calculates the likelihood that the particle is present at the position of the target object, and calculates the position of the target object according to the calculated likelihood. If a plurality of objects exist and they overlap, the tracking device tracks the object by correcting the likelihood.

JP 2010-0727223 A JP 2011-192092 A JP 2011-170684 A

The problem with the devices disclosed in the above-mentioned related literatures is that there is a high possibility of losing the target to be tracked by changing the background image or apparently changing the tracked target. As a result, these devices are likely to be unable to track the object to be tracked.

The reason is that the image recognition system disclosed in Patent Document 1 calculates the detection result and the tracking result independently, and adopts the result only when the calculation results match. As a result, the image recognition system cannot output a tracking result when either one of the detection result and the tracking result as described above fails in the calculation.

Each of the object tracking devices disclosed in Patent Documents 2 to 3 has only one type of object tracking device. Therefore, there is a high possibility that these object tracking devices cannot track the tracking target.

The main object of the present invention is to provide an image recognition system or the like that reduces the likelihood of detection and / or tracking failure.

In order to achieve the above object, an image recognition system according to the present invention has the following configuration.

An image recognition system according to the present invention, in an input image, identifies an image region where a tracking target category exists, and an identification calculation unit that calculates the identification frequency of a tracking target belonging to the tracking target category;
A tracking calculation unit for calculating a tracking frequency related to the position of the tracking target;
A weighting is determined according to a predetermined method for the results output by the identification calculation unit and the tracking calculation unit, and an integrated calculation unit that integrates the results according to the weighting,
It is characterized by providing.

As another aspect of the present invention, an image recognition method according to the present invention includes:
In the input image, an image region where the tracking target category exists is identified, and the identification frequency of the tracking target belonging to the tracking target category and the tracking frequency related to the position of the tracking target are calculated, and the identification frequency and the tracking frequency On the other hand, weighting is determined according to a predetermined method, and the identification frequency and the tracking frequency are integrated according to the weighting.

The object is also achieved by a computer program for realizing the image recognition system having the above-described configuration and the corresponding method by a computer, and a computer-readable storage medium storing the computer program. .

According to the present invention, it is possible to provide an image recognition system or the like that reduces the possibility of failure in detection and / or tracking.

1 is a block diagram illustrating a configuration of an image recognition system according to a first embodiment of the present invention. It is a flowchart which shows the procedure of the process which the image recognition system which concerns on the 1st Embodiment of this invention performs. It is a flowchart of another form which shows the procedure of the process which the image recognition system which concerns on the 1st Embodiment of this invention performs. It is a block diagram which shows roughly the hardware constitutions of the calculation processing apparatus which can implement | achieve the image recognition system which concerns on embodiment. It is a figure which shows the structure of the image recognition system which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the process which the image recognition system which concerns on the 3rd Embodiment of this invention performs. It is a flowchart which shows the procedure of the process which the image recognition system which concerns on the 4th Embodiment of this invention performs. It is a flowchart which shows the procedure of the process which the image recognition system which concerns on the 5th Embodiment of this invention performs. It is a flowchart which shows the procedure of the process which the image recognition system which concerns on the 6th Embodiment of this invention performs. It is a figure which shows the relationship of the time in a time series, and frequency in the 1st frequency calculation process which the identification calculating part which concerns on the 1st Embodiment of this invention performs. It is a figure which shows the relationship between the time in a time series, and a frequency in the 2nd frequency calculation process which the tracking calculating part which concerns on the 1st Embodiment of this invention performs. It is a figure which shows the relationship between the time in a time series, and a frequency in the 3rd frequency calculation process which the tracking calculating part which concerns on the 1st Embodiment of this invention performs. The figure which shows the relationship between the time in a time series, each frequency in 1st, 2nd, 3rd type frequency calculation processing, and the integrated result value of the image recognition system which concerns on the 1st Embodiment of this invention. It is. It is a figure showing the relationship between a position and a scale in identification frequency. It is a figure showing transition of the value which a discriminating degree takes, when a position and a scale correspond one to one. It is a figure showing an example of the method of expressing the transition of the value which a discrimination frequency takes when a position and a scale respond | correspond one to one. It is a figure showing transition of the value which a discrimination frequency takes when there are a plurality of scales. It is a figure showing an example of the method of expressing the transition of the value which a discrimination frequency takes when there are two or more scales. It is a figure which shows an example of distribution of tracking frequency.

Next, embodiments of the present invention will be described in detail with reference to the drawings. The detection in the embodiment of the present invention described below refers to identifying an object belonging to a specific category such as a pedestrian or a vehicle by distinguishing it from other image areas in the input image. Similarly, tracking in the embodiment of the present invention described below is performed by detecting or artificially specifying, on a time-series image, an image region of the same individual object in an image region designated as a tracking target. It means to keep doing.

<First Embodiment>
FIG. 1 is a block diagram showing a configuration of an image recognition system according to the first embodiment of the present invention. Referring to FIG. 1, the image recognition system 104 according to the present embodiment includes an integrated calculation unit 101, an identification calculation unit 102, and a tracking calculation unit 103.

The identification calculation unit 102 analyzes the input time-series image. The identification calculation unit 102 identifies “an image region in which a tracking target category (a tracking target category such as a pedestrian or a vehicle according to a purpose) exists” as a background image. In each region on the input image, the identification calculation unit 102 is a likelihood that is “an image region in which a tracking target category exists” or a value equivalent to the likelihood (hereinafter, these are collectively referred to as “discrimination frequency”. Is calculated for each tracking category.

The identification calculation unit 102 can be realized by a machine learning method or the like as described below. First, the identification calculation unit 102 represents an image with a feature amount. Next, the identification calculation unit 102 identifies the image data expressed using the feature amount using a classifier configured by a machine learning method or the like. The identification calculation unit 102 can employ Haar-like features, HoG features, and the like as feature amounts. Further, the identification calculation unit 102 can employ a Support Vector Machine (SVM), an Adaptive Boosting (AdaBoost) method, or the like as a machine learning method. The identification calculation unit 102 is not limited to the above-described example, and the identification calculation unit 102 may be realized using a feature quantity other than the above-described feature quantity or a machine learning method other than the above-described machine learning method. Good.

The identification calculation unit 102 calculates the identification frequency PD (x, S) according to the coordinates x = (u, v) on the image (hereinafter simply referred to as x) and the image area scale S. The identification frequency PD (x, S) is normalized to a value from 0.0 to 1.0 according to the coordinate x on the image and the scale S (FIG. 10). In other words, the discriminating frequency is not a discrete value of 0 or 1, but a value of 0.0 to 1.0, which is not a binary value. The detection in the embodiment of the present invention refers to calculating a non-binary value of 0.0 to 1.0.

Scale S is defined from values such as parameters including image width and height. For example, the scale S is a fixed value corresponding to the physical arrangement between the real world and the camera image and the appropriate size in the image of the tracking target object at each coordinate x determined from the optical system on a one-to-one basis. Can be defined as Alternatively, the scale S may be defined as a result of calculating the discrimination frequency for each of a plurality of scales with respect to the same coordinate x. The definition of the scale S is not limited to the examples shown above.

When the scale S is defined in a one-to-one correspondence with the coordinate x, the identification frequency PD (x, S) depends only on x. In this case, the distribution of PD (x, S) = PD (x) (FIG. 11A) can be expressed as a table that associates the value of x, the value of S, and PD (x, S) (FIG. 11B).

When the scale S is independent of the coordinate x, for example, if the scale S is expressed as at least one scale (S1, S2,..., Sm), the identification frequency PD (x, S) is respectively Discriminating frequencies PD (x, S1), PD (x, S2),. . . , PD (x, Sn) (FIG. 12A). In that case, the distribution of PD (x, S) is x and PD (x, S1), PD (x, S2),. . . , PD (x, Sn) can be expressed as a table (FIG. 12B).

The tracking calculation unit 103 analyzes at least one input time-series image. In the analysis, for at least one tracking target, for each tracking target, the likelihood related to the tracking position, or a value equivalent to the likelihood (hereinafter, the two are collectively referred to as “tracking frequency”). calculate.

The tracking calculation unit 103 can employ template matching using image features, a derivation method thereof, and the like as a method of calculating the tracking frequency. Furthermore, the tracking calculation unit 103 can also use estimation methods such as linear interpolation, curve fitting, Kalman filter, and Particle filter from the track of the tracking target as a method of calculating the tracking frequency. Furthermore, the tracking calculation unit 103 can also employ the similarity between the registered image feature and the image feature at the coordinate x existing within the search range as the tracking frequency. The method of calculating the tracking frequency in the tracking calculation unit 103 is not limited to the method exemplified above. Further, the tracking frequency in the tracking calculation unit 103 is not limited to the similarity exemplified above.

The tracking calculation unit 103 can employ, as the image feature, an HS color histogram inside the image region registered as a tracking target, an edge base feature such as HoG, or the like. Further, the tracking calculation unit 103 can employ a Bhattacharya distance or the like as a similarity calculation method. The image features in the tracking calculation unit 103 are not limited to the image features exemplified above. Similarly, the similarity calculation method in the tracking calculation unit 103 is not limited to the similarity calculation method exemplified above.

Further, the tracking calculation unit 103 can also retain the image feature in the image region at the time of registering the tracking target as the image feature without depending on the time. Alternatively, the tracking calculation unit 103 may update the image feature in time series according to the image feature of the region defined from the tracking position from the time when the tracking target is registered to the previous time and the scale. . For example, image features f ′ (0),..., F ′ (t−1) acquired from the tracking target region at the tracking position from the registration time (0) to the previous time (t−1). Is defined as the weighted average fm (t−1), the tracking calculation unit 103 can update the image according to the image feature at each coordinate x and the comparison result. The image feature processing method in the tracking calculation unit 103 is not limited to the above method.

The tracking calculation unit 103 calculates the tracking frequency PT (x, Oi) of the i-th tracking target Oi at the coordinate x on the image. The tracking frequency PT (x, Oi) is normalized to a real value from 0.0 to 1.0 (FIG. 13).

The integrated calculation unit 101 integrates the calculation values of the identification calculation unit 102 and the tracking calculation unit 103, registers a new tracking target, updates and deletes the tracking information of the existing tracking target, and records past tracking information. Save.

The result integration in the integrated calculation unit 101 corresponds to the weighting after determining the weighting of each frequency using the identification frequency calculated by the identification calculation unit 102 and the tracking frequency calculated by the tracking calculation unit 103 as inputs. To calculate the integration result value. The weight is determined according to a predetermined method. The integrated calculation unit 101 determines the position of the tracking target according to the integration result value.

As a weighting method, the integration calculation unit 101 generates a weight at least once in a predetermined range and determines the maximum integration result value as described above, or sets a specific frequency in advance. For example, a method of gradually increasing the corresponding weight can be employed. Further, the integrated calculation unit 101 changes the weight according to a method of fixing the weight in advance so as to average at least two or more frequencies as a weighting method, or according to a deviation from the average of each frequency. Methods can also be employed. The weighting method in the integrated arithmetic unit 101 is not limited to the above, and methods other than these methods may be adopted.

The integrated calculation unit 101 extracts, as a tracking candidate, an image region with a high likelihood that is an image region in which an object category to be tracked exists, according to the identification frequency PD (x, S) output from the identification calculation unit 102. To do. If the integrated calculation unit 101 determines that the region extracted as the tracking candidate is not the same as the registered tracking target, it registers as the reference tracking target.

For example, the integrated calculation unit 101 extracts, as a tracking candidate region, a region related to the coordinate x and the scale S when the maximum value is obtained in a region where the identification frequency PD (x, S) is equal to or greater than a predetermined threshold. Next, the integrated calculation unit 101 excludes the already registered tracking candidate area from the new tracking target in the search range of the tracking target information in the tracking candidate area extracted as described above, and the others Register as a new tracking target. The integrated calculation unit 101 can be realized by such a processing method. However, it is not limited to the processing method described above.

The integrated calculation unit 101 has the previous time tracking information Oi (t−1) = {position coordinates: yi (t−1), search range: Ai (t−1), scale: Si (t ―1)} is held. Based on the tracking information and the information acquired at the current time, the integrated arithmetic unit 101 stores the tracking information Oi (t) at the current time. The tracking information Oi (t) is tracking information used in the next processing at time t + 1.

The scale Si (t) of the i-th tracking target at the current time t may be the same as the value of Si (t−1) of the i-th tracking target at the previous time (t−1), or time series You may update according to. For example, when the scale parameter S in the identification calculation unit 102 is uniquely defined for the image coordinate x, the scale Si (t) is a scale corresponding to the tracking position yi (t) determined at the current time. S. Further, for example, when the scale parameter S in the identification calculation unit 102 is independent of the image coordinate x, the scale Si (t) is set to S that maximizes the integration frequency. The definition of scale Si (t) is not limited to the method shown above, and other methods may be used.

The search range A (t) corresponding to the position coordinates at the current time (t) is an area range that is assumed to include the tracking target at the next time moved from the position coordinates y (t) at the current time. . Therefore, for example, the search range A (t) is defined as a region that is a constant multiple of the scale S (t) with y (t) as the center.

For example, when the scale is defined as S (t) = (w, h) (w: width of the tracking target, h: height), the search range A (t) is
A (t) = {image region in the range of ± (w, h) centered on y (t)},
Can be defined as The search range A (t) can also be defined as being independent of the scale.

For example, the search range A (t) may be defined as an area that is a constant multiple of | y (t) −y (t−1) | based on the history of changes in which the tracking target has moved up to the previous time. it can.

The definition of the scale Si (t) and the search range A (t) is not limited to the above-described definition method, and may be defined by other definition methods.

FIG. 2A is a flowchart showing a procedure of processing executed by the image recognition system according to the first embodiment of the present invention. FIG. 2B is a flowchart of another form showing a procedure of processing executed by the image recognition system according to the first embodiment of the present invention. Next, the operation of the image recognition system according to the first embodiment of the present invention will be described with reference to FIGS. 1 and 2A.

The identification calculation unit 102 identifies “image area in which the tracking target category exists” in the input time-series image as a background image. In this case, the identification calculation unit 102 calculates an identification frequency that is “an image area in which a tracking target category exists” in each area on the input image (step S201).

Next, the tracking calculation unit 103 analyzes the input at least one time series image. In the analysis, the tracking frequency related to the tracking position is calculated for at least one tracking object (step S202). Referring to FIG. 2B, the order of the process of step S201 and the process of step S202 is reversed. This is because the process in step S201 and the process in step S202 can operate independently. That is, the identification calculation unit 102 may execute the identification power calculation and the tracking power calculation in any order, and may execute them simultaneously.

The integrated calculation unit 101 determines the weight according to a predetermined method from the two values of the identification frequency calculated by the identification calculation unit 102 and the tracking frequency calculated by the tracking calculation unit 103, and refers to the weight, and the integrated result value Is calculated (step S203).

As a result, the image recognition system 104 increases the weighting of the other frequency even when the frequency of either the identification calculation unit 102 or the tracking calculation unit 103 falls below a predetermined range due to the cause of image noise or the like. By doing so, the influence on the integrated result value is suppressed, and it is possible to continuously perform tracking on the tracking target. Thereby, the image recognition system 104 can reduce the possibility of detection or tracking failure in the image recognition system or the like.

FIG. 9A is a diagram illustrating a relationship between time and frequency in time series in the first frequency calculation process performed by the identification calculation unit. FIG. 9B is a diagram illustrating a relationship between time and frequency in time series in the second frequency calculation process performed by the tracking calculation unit. FIG. 9C is a diagram illustrating a relationship between time and frequency in time series in the third frequency calculation process performed by the tracking calculation unit. FIG. 9D shows the time in time series, the respective frequencies in the first, second, and third types of frequency calculation processing, and the integration result value of the image recognition system according to the first embodiment of the present invention. It is a figure which shows a relationship.

These three types of frequencies and the integration result value calculated by the integration calculation unit 101 have a smaller reliability as the value is smaller, and the higher the value, the higher the reliability. Represents that That is, the integrated calculation unit 101 indicates that the tracking target can be recognized as the reliability value is high, and conversely, the tracking target is not recognized as the reliability value is low.

Referring to FIG. 9A, the transition of the identification frequency calculated by the identification calculation unit 102 will be described. The frequency keeps a high value while the time is early, but the reliability temporarily decreases due to the influence of noise mixed in the time-series image. In this case, the identification calculation unit 102 cannot identify the “image area in which the tracking target category exists” from the background image during the period in which the reliability is low.

If the time further advances, occlusion may occur where the target to be tracked is hidden behind the background of other targets. When the occlusion occurs, the identification calculation unit 102 cannot distinguish the “image area where the tracking target category exists” from the background image for a long period of time. As a result, the frequency calculated by the identification calculation unit 102 decreases over a long period of time.

The transition of the tracking frequency calculated by the tracking calculation unit 103 will be described with reference to FIGS. 9A and 9B. The processing method in the tracking calculation unit 103 is different from the processing method in the identification calculation unit 102. Therefore, the transition of the frequency output from the tracking calculation unit 103 and the transition of the frequency output from the identification calculation unit 102 are different from each other.

The identification frequency calculated by the identification calculation unit 102 temporarily decreases due to the appearance of an image having similar characteristics. Similarly, the identification calculation unit 102 cannot identify the “image area in which the tracking target category exists” from the background image for a long time due to the occlusion. Meanwhile, the identification frequency calculated by the identification calculation unit 102 decreases over a long period of time.

However, since the occurrence time of noise that causes a decrease in the identification frequency is different from the occurrence time of the cause that causes a decrease in the tracking frequency, the image recognition system according to the first embodiment of the present invention integrates these frequencies. By doing, it can reduce that a frequency falls.

9C has a different configuration from the tracking calculation unit 103 in FIG. 9B. The transition of the tracking frequency calculated by the tracking calculation unit 103 will be described with reference to FIG. 9C. The tracking frequency is lowered by the non-linear motion of the tracked object. However, the tracking frequency does not decrease even if occlusion occurs. This behavior is different from the two frequency behaviors described above.

The integration result value according to the present embodiment will be described with reference to FIGS. 9A, 9B, 9C, and 9D. Referring to FIG. 9A, FIG. 9B, and FIG. 9C, it is common that the curves representing the respective frequencies have a time region in which the frequencies are low. However, the times when the frequency decreases are different from each other. On the other hand, referring to the solid line in FIG. 9D, the curve representing the integration result value described above has high reliability in the entire time domain. That is, it means that the image recognition system 104 continues to recognize the tracking target.

That is, according to the first embodiment of the present invention, it is possible to provide an image recognition system or the like that reduces the possibility of failure in detection and / or tracking.

In the above description, an example in which the frequency is three types has been described, but even if there are two types, according to the first embodiment of the present invention, detection and / or tracking may fail. It is possible to provide an image recognition system or the like with reduced performance. Similarly, even when the frequency is four or more, according to the first embodiment of the present invention, it is possible to provide an image recognition system or the like that can reduce the possibility of failure in detection and / or tracking.

In the above description, both the identification calculation unit 102 and the tracking calculation unit 103 have been normalized to output real values from 0.0 to 1.0, but are not necessarily from 0.0 to 1.0. It need not be a real number. That is, for the outputs of the identification calculation unit 102 and the tracking calculation unit 103, in addition to the above, it is possible to use numerical values whose value ranges match and the order relationship can be defined.

<Second Embodiment>
Next, a second embodiment based on the above-described first embodiment will be described.

In the following description, the characteristic parts according to the present embodiment will be mainly described, and the same reference numerals will be given to the same configurations as those in the first embodiment described above, and the redundant description will be omitted. To do.

FIG. 4 is a diagram showing a configuration of an image recognition system according to the second embodiment of the present invention. Next, the configuration of the image recognition system according to the second embodiment will be described with reference to FIG.

The tracking calculation unit 407 includes a first tracking calculation unit 405 that calculates a tracking frequency according to a calculation method using an image feature, a second tracking calculation unit 406 that calculates a tracking frequency according to a calculation method using information other than the image feature, Is provided. The image recognition system 408 includes an identification calculation unit 102, a tracking calculation unit 407, and an integrated calculation unit 101.

The first tracking calculation unit 405 compares the feature extracted from the image in the past with the input image, and calculates the tracking frequency based on the comparison result. The first tracking calculation unit 405 calculates the tracking frequency using, for example, template matching using an image feature and a derivation method thereof. The calculation method using the image feature used by the first tracking calculation unit 405 is not limited to the method exemplified here.

The second tracking calculation unit 406, for example, analyzes one or more input time-series images, and then employs each calculation method such as linear interpolation, curve fitting, Kalman filter, and Particle filter from the track to be tracked. Calculate the tracking frequency. The calculation method employed by the second tracking calculation unit 406 is not limited to the method illustrated here.

The identification calculation unit 102, the first tracking calculation unit 405, and the second tracking calculation unit 406 respectively calculate the frequency according to mutually different processing methods. As a result, the calculation results obtained from these three calculation units are likely to be different from each other.

As a result, the image recognition system 408 determines that any two of the identification calculation unit 102, the first tracking calculation unit 405, and the second tracking calculation unit 406 have a predetermined range depending on the cause of image noise or the like. Even when the value is less than, the weighting of the frequency of the other calculation unit is increased. By increasing the weight, the influence on the integrated result value is suppressed, and the tracking can be continued. As a result, the image recognition system 408 can further reduce the possibility of detection or tracking failure in the image recognition system or the like.

That is, according to the second embodiment of the present invention, it is possible to provide an image recognition system or the like that reduces the possibility of failure in detection and / or tracking. Furthermore, since the tracking calculation unit 407 calculates two or more different tracking frequencies, according to the second embodiment of the present invention, an image with reduced possibility of detection and / or tracking failure. A recognition system can be provided.

Here, although the flow of the present embodiment has been described with reference to FIG. 4, the execution order of the process of step S201 and the process of step S202 may be interchanged or may be executed simultaneously.

<Third Embodiment>
Next, a third embodiment based on the first and second embodiments described above will be described.

In the following description, the characteristic part according to the present embodiment will be mainly described, and the same configurations as those in the first and second embodiments described above are denoted by the same reference numerals and overlapped. Description is omitted. Since the second embodiment is based on the first embodiment, differences from the first embodiment will be described. The same description applies to the third embodiment based on the second embodiment, which is not described here.

FIG. 5 is a flowchart showing a procedure of processing executed by the image recognition system according to the third embodiment of the present invention. Here, the operation of the image recognition system 104 according to the third embodiment will be described in detail with reference to FIGS. 1 and 5.

Equation 1 is an equation for calculating the result integrated value. The function f is a monotonically increasing function whose domain is the identification frequency calculated by the identification calculation unit 102 (step S201) or the tracking frequency calculated by the tracking calculation unit 103 (step S202). Represents an association. The calculation of the result integration value is to take the sum of the product of the real number and the weight W for all frequencies.

As shown in Equation 1, the integrated calculation unit 101 includes a function value monotonically increasing to the identification frequency calculated by the identification calculation unit 102 and a function value monotonically increasing to the tracking frequency calculated by the tracking calculation unit 103. A weighted arithmetic mean is calculated (step S503), the weighting of each frequency described above is determined according to a predetermined method, and then an integration result value is calculated (step S504).

Result integrated value = ΣWi * frequency, ΣWi = 1, 0 ≦ Wi ≦ 1 (Equation 1),
As a result, the image recognition system 104 sets a weighted total average value of two or more different tracking frequencies as an integrated result value. Therefore, the possibility that the integration result value falls below the lower limit of the predetermined range is low, and the image recognition system 104 is a system that performs image recognition that is resistant to image changes. That is, according to the third embodiment of the present invention, it is possible to provide an image recognition system or the like that reduces the possibility that detection and / or tracking will both fail.

Here, although the flow of the present embodiment has been described with reference to FIG. 5, the execution order of the process of step S201 and the process of step S202 may be interchanged or may be executed simultaneously.

<Fourth Embodiment>
Next, a fourth embodiment based on the first and second embodiments described above will be described.

In the following description, the characteristic part according to the present embodiment will be mainly described, and the same configurations as those in the first and second embodiments described above are denoted by the same reference numerals and overlapped. Description is omitted. Since the second embodiment is based on the first embodiment, the difference between the first embodiment and this embodiment will be described. The same description applies to the fourth embodiment based on the second embodiment, which is not described here.

FIG. 6 is a flowchart showing a procedure of processing executed by the image recognition system according to the fourth embodiment of the present invention. Here, the operation of the image recognition system 104 will be described in detail with reference to FIGS. 1 and 6.

Equation 2 is an equation for calculating the result integrated value. The result integrated value is calculated by using the identification frequency calculated by the identification calculation unit 102 (step S201) or the product of the tracking frequency calculated by the tracking calculation unit 103 (step S202) and the weight W for all the frequencies. It is to take the sum.

The integrated calculation unit 101 calculates a weighted arithmetic average of two values in the object identification frequency calculated by the identification calculation unit 102 and the position frequency calculated by the tracking calculation unit 103 as shown in Expression 2 ( Step S603).

Result integrated value = ΣWi * f (frequency), ΣWi = 1, 0 ≦ Wi ≦ 1, f () is a monotonically increasing function (Equation 2),
Weighting is determined according to a predetermined method, and the integrated result value is calculated corresponding to the weighting (step S504). As a result, the image recognition system 104 sets a weighted arithmetic mean value of two or more different tracking frequencies as an integrated result value. Therefore, the possibility that the integration result value described above falls below the lower limit of the predetermined range is low, and the image recognition system 104 is a system that performs image recognition that is resistant to image changes. That is, according to the present embodiment, it is possible to provide an image recognition system or the like that reduces the possibility that detection and / or tracking will fail.

Here, the flow of the present embodiment has been described with reference to FIG. 6, but the execution order of the process of step S201 and the process of step S202 may be interchanged or may be executed simultaneously.

<Fifth Embodiment>
Next, a fifth embodiment based on the first embodiment described above will be described.

In the following description, the characteristic parts according to the present embodiment will be mainly described, and the same reference numerals will be given to the same configurations as those in the first embodiment described above, and the redundant description will be omitted. To do.

FIG. 7 is a flowchart showing a procedure of processing executed by the image recognition system according to the fifth embodiment of the present invention. Here, the operation of the image recognition system 104 will be described in detail with reference to FIGS. 1 and 7.

The integrated calculation unit 101 uses a predetermined method based on the identification frequency calculated by the identification calculation unit 102 (step S201) and the tracking frequency calculated by the first tracking calculation unit 405 and the second tracking calculation unit 406 (step S202). The weighting is determined according to, and the integrated result value is calculated according to the weighting (step S703). Next, when the integration result value falls below the lower limit of the predetermined range (step S704), the integration calculation unit 101 changes the weighting according to a predetermined method (step S705). The integrated calculation unit 101 calculates the integrated result value and weights it until the integrated result value is equal to or greater than the lower limit of the predetermined range. When the integration result value is equal to or greater than the lower limit of the predetermined range, the integration calculation unit 101 performs an integration calculation corresponding to the integration result value (step S504).

As a predetermined method, the integrated calculation unit 101 generates a random value at least once in a predetermined range and weights the integrated result value so as to maximize the weight, or gradually adds a weight corresponding to a certain frequency. A method of increasing the size can be adopted. Furthermore, the integrated calculation unit 101 changes the weight according to a predetermined method, such as a method of fixing the weight in advance with a value that does not match any one frequency, or a deviation from the average of each frequency. Methods can also be employed. The weighting method in the integrated arithmetic unit 101 is not limited to the above, and methods other than these methods can also be employed.

Since the influence on the integration result value calculated by the integration calculation unit 101 is suppressed, the image recognition system according to the fifth embodiment of the present invention makes it possible to continue tracking. Accordingly, the image recognition system 104 becomes a system that performs image recognition that is resistant to image changes. That is, according to the fifth embodiment of the present invention, it is possible to provide an image recognition system or the like that reduces the possibility of failure in detection and / or tracking.

Here, although the flow of the present embodiment has been described with reference to FIG. 7, the execution order of the process of step S201 and the process of step S202 may be switched, or may be executed simultaneously.

<Sixth Embodiment>
Next, a sixth embodiment based on the above-described second embodiment will be described.

In the following description, the characteristic part according to the present embodiment will be mainly described, and the same components as those in the second embodiment described above will be denoted by the same reference numerals, and redundant description will be omitted. To do.

FIG. 8 is a flowchart showing a procedure of processing executed by the image recognition system according to the sixth embodiment of the present invention. Here, the operation of the image recognition system 104 will be described in detail with reference to FIGS. 1 and 8.

The integrated calculation unit 101 determines the identification frequency calculated by the identification calculation unit 102 (step S201) and the tracking frequency calculated by the first tracking calculation unit 405 and the second tracking calculation unit 406 (step S202). The weighting is determined according to the method. Next, the integration calculation unit 101 calculates an integration result value (step S803), and when the integration result value falls below the lower limit of the predetermined range (step S704), changes the weighting (step S805).

The integration calculation unit 101 repeats the calculation of the integration result value and the determination of the weighting until the integration result value is equal to or greater than the lower limit of the predetermined range. When the integration result value is equal to or greater than the lower limit of the predetermined range, the integration calculation unit 101 performs the integration calculation according to the integration result value (step S504).

As a predetermined method, the integrated calculation unit 101 generates a random value at least once in a predetermined range and weights the integrated result value so as to maximize the weight, or gradually adds a weight corresponding to a certain frequency. A method of increasing the size can be adopted. Furthermore, the integrated calculation unit 101 changes the weight according to a predetermined method, such as a method of fixing the weight in advance with a value that does not match any one frequency, or a deviation from the average of each frequency. A method etc. can also be adopted.

Further, as a predetermined method, the integrated calculation unit 101 uses the above-described weight corresponding to the tracking frequency of the second tracking calculation unit 406 based on the above-described weight corresponding to the frequency of the first tracking calculation unit 405 and the identification calculation unit 102. If the maximum value of the integration result value does not exceed the lower limit of the predetermined range, the weight corresponding to the tracking frequency of the second tracking calculation unit 406 described above is calculated. A method of increasing the size can also be adopted.

The integration calculation unit 101 can also employ a method of terminating the calculation when the maximum value of the integration result values described above exceeds the lower limit of the predetermined range as a predetermined method. The predetermined method in the integrated calculation unit 101 is not limited to the above, and may be realized by a method other than these methods.

The sixth embodiment of the present invention suppresses the influence of the frequency reduction in the identification calculation unit 102, the first tracking calculation unit 405, and the second tracking calculation unit 406 on the integrated result value, and continues tracking. enable. Thus, the image recognition system according to the sixth embodiment of the present invention is a system that performs image recognition that is resistant to image changes. That is, according to the sixth embodiment of the present invention, it is possible to provide an image recognition system or the like that reduces the possibility that detection and / or tracking will both fail.

Here, the flow of the present embodiment has been described with reference to FIG. 8, but the execution order of the process of step S201 and the process of step S202 may be switched, or may be executed simultaneously.

(Hardware configuration example)
Next, a configuration example of hardware resources for realizing the image recognition system in each of the above-described embodiments using one calculation processing device (information processing device, computer) will be described. However, such an image recognition system may be realized physically or functionally using a plurality of calculation processing devices (information processing devices, computers). Such an image recognition system may be realized as a dedicated device.

FIG. 3 is a diagram schematically showing a hardware configuration of a calculation processing device (information processing device, computer) capable of realizing the image recognition system according to the first to sixth embodiments. The calculation processing device 306 includes a CPU (Central Processing Unit) 301, a memory 302, a disk 303, an output device 304, and an input device 305.

That is, the CPU 301 copies a software program (computer program) stored in the disk 303: hereinafter simply referred to as a program to the memory 7 at the time of execution, and executes arithmetic processing. The CPU 301 reads data necessary for program execution from the memory 302. When display is necessary, the CPU 301 displays the output result on the output device 304. When inputting a program from the outside, the CPU 301 reads a program stored in a program storage medium 307 in which a computer-readable non-transitory program is stored via the input device 305. The CPU 301 interprets the image recognition program (FIG. 2A, FIG. 2B, FIG. 5 to FIG. 8) in the memory 302 corresponding to the function (process) represented by each unit shown in FIG. 1 or FIG. Execute. The CPU 301 sequentially performs the processes described in the above embodiments of the present invention.

That is, in such a case, it can be understood that the present invention can also be achieved by such an image recognition program. Furthermore, it can be understood that the present invention can also be realized by a computer-readable recording medium in which such an image recognition program is recorded.

In addition, a part or all of each embodiment mentioned above can be described also as the following additional remarks. However, the present invention described by way of example with the above-described embodiments is not limited to the following. That is,
(Appendix 1)
In the input image, an identification calculation unit that identifies an image area where the tracking target category exists and calculates the identification frequency of the tracking target belonging to the tracking target category;
A tracking calculation unit for calculating a tracking frequency related to the position of the tracking target;
A weighting is determined according to a predetermined method for the results output from the identification calculation unit and the tracking calculation unit, and an integrated calculation unit that integrates the results according to the weighting,
An image recognition system comprising:

(Appendix 2)
The image recognition system according to appendix 1, wherein the identification calculation unit calculates two or more types of identification frequencies.

(Appendix 3)
3. The image recognition system according to appendix 1 or 2, wherein the tracking calculation unit calculates two or more types of tracking frequencies.

(Appendix 4)
The tracking calculation unit includes:
A first tracking calculation unit for calculating a first tracking frequency according to a calculation method using an image feature;
A second tracking calculation unit for calculating a second tracking frequency according to a calculation method using information other than image features;
The image recognition system according to appendix 1, further comprising:

(Appendix 5)
The image recognition system according to appendix 4, wherein the identification calculation unit calculates two or more types of identification frequencies.

(Appendix 6)
The image recognition system according to any one of appendices 4 to 5, wherein the first tracking calculation unit calculates two or more types of discrimination frequencies.

(Appendix 7)
The image recognition system according to any one of appendices 4 to 6, wherein the second tracking calculation unit calculates two or more types of discrimination frequencies.

(Appendix 8)
The integrated calculation unit is configured to calculate the value of the function monotonically increasing to the numerical value calculated by the identification calculation unit and the value of the function monotonically increasing to the numerical value calculated by the tracking calculation unit by using a weighted arithmetic mean. The calculation result is calculated,
The image recognition system according to any one of appendices 1 to 7.

(Appendix 9)
The integrated calculation unit calculates the calculation result by a weighted arithmetic average of a numerical value calculated by the identification calculation unit and a numerical value calculated by the tracking calculation unit,
The image recognition system according to any one of appendices 1 to 7.

(Appendix 10)
The integrated calculation unit is configured to alternately and repeatedly change the weight according to a predetermined method and calculate the calculation result until the calculation result reaches a lower limit of a predetermined range. ,
The image recognition system according to any one of appendices 1 to 7.

(Appendix 11)
The integrated calculation unit increases the weight for the calculation value of the second tracking calculation unit until the calculation result is equal to or greater than the lower limit of the predetermined range, and the calculation value of the first tracking calculation unit and the identification calculation unit. The image recognition system according to any one of appendices 1 to 7, wherein a weight is reduced and the calculation result is calculated alternately.

(Appendix 12)
In the input image, the image area where the tracking target category exists is identified, the identification frequency of the tracking target belonging to the tracking target category, the tracking frequency related to the position of the tracking target are calculated, and the identification frequency and the tracking frequency are On the other hand, an image recognition method in which weighting is determined according to a predetermined method, and the identification power and the tracking power are integrated according to the weighting.

(Appendix 13)
An identification calculation function for identifying an image area where a tracking target category exists in an input image and calculating a discrimination frequency of the tracking target belonging to the tracking target category;
A tracking calculation function for calculating a tracking frequency related to the position of the tracking target;
A weighting is determined according to a predetermined method for the results output by the identification calculation function and the tracking calculation function, and an integrated calculation function for integrating the results according to the weighting,
A computer program to be realized by a computer.

The present invention has been described above using the above embodiment as an exemplary example. However, the present invention is not limited to the above-described embodiment. That is, the present invention can apply various modes that can be understood by those skilled in the art within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2012-046424 filed on March 2, 2012, the entire disclosure of which is incorporated herein.

DESCRIPTION OF SYMBOLS 101 Integrated calculation part 102 Identification calculation part 103 Tracking calculation part 104 Image recognition system 301 CPU
302 memory 303 disk 304 output device 305 input device 306 calculation processing device 307 storage medium 405 first tracking calculation unit 406 second tracking calculation unit 407 tracking calculation unit 408 image recognition system

Claims (10)

In the input image, an identification calculation unit that identifies an image area where the tracking target category exists and calculates the identification frequency of the tracking target belonging to the tracking target category;
A tracking calculation unit for calculating a tracking frequency related to the position of the tracking target;
A weighting is determined according to a predetermined method for the results output from the identification calculation unit and the tracking calculation unit, and an integrated calculation unit that integrates the results according to the weighting,
An image recognition system provided. The image recognition system according to claim 1, wherein the identification calculation unit calculates two or more types of identification frequencies. The image recognition system according to claim 1, wherein the tracking calculation unit calculates two or more types of tracking frequencies. The tracking calculation unit includes:
A first tracking calculation unit for calculating a first tracking frequency according to a calculation method using an image feature;
The image recognition system according to claim 1, further comprising: a second tracking calculation unit that calculates a second tracking frequency according to a calculation method using information other than image features. The integrated calculation unit is configured to calculate the value of the function monotonically increasing to the numerical value calculated by the identification calculation unit and the value of the function monotonically increasing to the numerical value calculated by the tracking calculation unit by using a weighted arithmetic mean. Calculate the calculation result,
The image recognition system according to claim 1. The integrated calculation unit calculates the calculation result by a weighted arithmetic average of the numerical value calculated by the identification calculation unit and the numerical value calculated by the tracking calculation unit,
The image recognition system according to claim 1. The integrated calculation unit alternately and repeatedly changes the weight according to a predetermined method and calculates the calculation result until the calculation result is equal to or higher than a lower limit of a predetermined range.
The image recognition system according to claim 1. The integrated calculation unit increases the weight for the calculation value of the second tracking calculation unit and the calculation values of the first tracking calculation unit and the identification calculation unit until the calculation result is equal to or higher than a lower limit of a predetermined range. The image recognition system according to any one of claims 1 to 4, wherein a weight is reduced and the calculation result is calculated alternately. In the input image, the image area where the tracking target category exists is identified, and the identification frequency of the tracking target belonging to the tracking target category and the tracking frequency related to the position of the tracking target are calculated, and the identification frequency and the tracking frequency are On the other hand, an image recognition method in which weighting is determined according to a predetermined method, and the identification power and the tracking power are integrated according to the weighting. An identification calculation function for identifying an image area where a tracking target category exists in an input image and calculating a discrimination frequency of the tracking target belonging to the tracking target category;
A tracking calculation function for calculating a tracking frequency related to the position of the tracking target;
A weighting is determined according to a predetermined method for the results output by the identification calculation function and the tracking calculation function, and an integrated calculation function for integrating the results according to the weighting,
A computer program to be realized by a computer.

Images