JP5027201B2 - Telop character area detection method, telop character area detection device, and telop character area detection program - Google Patents

Description

  The present invention relates to a technique for determining whether or not there is a telop character in a video (moving image) or a still image, and if there is a telop character, automatically extracting the area.

  For example, the following techniques have been proposed as conventional techniques for extracting a telop character region from a moving image frame or a still image.

  Patent Document 1 discloses a technique for detecting a frame in which a telop character is displayed from a video. In this technique, when a block including a predetermined number or more edge pairs in a video frame continues, A frame is detected as a frame in which a telop character is displayed.

  Patent Document 2 discloses a technique for detecting subtitle characters that are displayed while moving from a video. In this technique, edge pair feature points are detected as feature points that predominantly appear in the subtitle character portion. .

Japanese Patent Laid-Open No. 11-178007 JP-A-11-205679

  However, when telop detection is performed by the above-described conventional technique, there is a problem that a rectangular area determined as a character area is slightly shifted from a correct answer.

  Small misalignment is not a problem if only frames with telop are detected from the video, but slight misalignment significantly lowers the detection rate when judging the headline telop of the program based on the position of the character area. May end up.

  An object of the present invention is to solve the above problems and provide a technique for accurately detecting a character region on an image.

  In the present invention, in order to solve the above-described problem, the character area is detected by paying attention to the color of the character. For characters used in telops, colors such as gradation are rarely used, and there are at most about two to three colors used simultaneously as character colors. Therefore, we focus on the color used for the telop character and propose a technique for accurately detecting the telop character region by estimating the character color.

  The outline of the present invention is as follows. Since characters are composed of strokes, edges (color change points) occur in the character portion of the image. Clustering is performed in which a pair of color change points constituting an edge is taken out as an edge pair for areas where the edge density is equal to or greater than a predetermined threshold, and the edge pairs having similar colors are collected and grouped. Since the number of character colors is usually 2 to 3 at most, the color of the edge pairs that fall within the same clustering color range is considered to be used for telop characters, and the character color is estimated. The telop character area is detected based on the above.

  Specifically, the present invention inputs a moving image frame or still image to be detected in the telop character area, stores it in a storage device, extracts an edge from the image, and changes the color change points constituting the edge. Are detected as edge pairs. Further, a high-density edge region in which edges or edge pairs are present in high density is extracted from the image, and a color on a line connecting the edge pairs in the high-density edge region is extracted and classified by a predetermined clustering method. A color in which the number or ratio of colors belonging to the same classification is equal to or greater than a predetermined threshold is estimated as a telop character color. A telop character area in the image is detected from the estimated telop character color, and the area is displayed or output.

  In the above invention, when detecting the telop character area from the estimated telop character color, the same character color as that estimated in the region within a predetermined range around each edge pair having the estimated telop character color A pixel having a color belonging to the classification is stored as a telop character region candidate, and a telop character region candidate whose area occupied by the telop character region candidate or a ratio of the area within the predetermined range is equal to or greater than a predetermined threshold value, If a method of detecting a telop character area is used, more accurate detection is possible.

  According to the present invention, since the telop character area is determined after estimating the color used for the telop character, the telop character area can be determined accurately. Moreover, since the color cluster classification is performed, even if a plurality of colors are used in the telop, all of them can be detected.

It is a figure which shows the structural example of the telop character area detection apparatus by this Embodiment. It is a telop character area detection process flowchart by this Embodiment. It is a figure explaining the example of the edge detection by this Embodiment. It is a figure explaining the example of the edge pair detection by this Embodiment. It is a figure explaining the example of the high-density edge area | region extraction by this Embodiment. It is a figure which shows the example of the clustering of the color in the Lab space by this Embodiment. It is a figure explaining the recognition method of the telop character area | region by this Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration example of a telop character area detection apparatus according to the present embodiment. The telop character area detection device 10 is configured by, for example, a computer having a CPU and a memory, and is a device that extracts a telop character area from a frame or a still image of a moving image, and is realized by computer hardware and software programs. An image input unit 11, an image storage unit 12, an edge detection unit 13, a high density edge region extraction unit 14, a color classification unit 15, a telop character color estimation unit 16, a telop character region detection unit 17, and a telop character region display unit 18. Prepare.

  The image input unit 11 inputs an image such as a video (moving image) or a still image that is a detection target of the telop character area. The input image is stored in the image storage unit 12.

  The edge detection unit 13 detects an edge that is a color change point from the image stored in the image storage unit 12. The edge pair detection unit 131 in the edge detection unit 13 detects an edge pair that is a pair of edges paired within the range of the assumed character line width from the detected edges. An edge is detected as a pixel whose pixel value changes locally and discontinuously among the pixels that make up the image, but within the detected edge, the distance between two edges is within a certain range. When the color change such as luminance change or saturation change between the edges is small, the edge pair is called an edge pair. Edge pairs appear frequently in the telop part on the image. Details of the edge pair are also described in Patent Document 1 and Patent Document 2.

  The high-density edge area extraction unit 14 extracts a high-density edge area, which is an area where edges or edge pairs are concentrated, from an image that is a detection target of a telop character area. The color classification unit 15 extracts colors between edge pairs existing in the extracted high-density edge region, and classifies (clusters) the colors. The telop character color estimation unit 16 estimates the color used in the telop character based on the color classification extracted from the high-density edge region. The telop character area detection unit 17 detects a telop character area from the estimated telop character color. The telop character area display unit 18 displays or outputs the detected telop character area. In the output of the telop character area, binary data such as “1” may be output if included in the telop character area, and “0” if not included.

  FIG. 2 is a flowchart of telop character area detection processing according to this embodiment. Image files such as moving images and still images are input from the image input unit 11 and stored in the image storage unit 12 (step S10). In the following, an example in which the input image is a still image will be described. However, a telop character area can be detected for a moving image by the same process for each frame.

  The edge detection unit 13 detects a point estimated as an edge from the image stored in the image storage unit 12, and the edge pair detection unit 131 sets an edge pair having a small color change between edges as an edge pair. It detects (step S11).

  Next, the edge density at each point on the image is calculated (step S12). Here, the high-density edge region detection unit 14 counts, for example, the number of points regarded as edges within a certain size rectangle centered on each point, and divides by the area of the rectangle (number of pixels). The value is calculated as the edge density.

  Next, it is confirmed whether or not there is a point where the calculated edge density is equal to or greater than a predetermined threshold (step S13). If there is no point whose edge density is equal to or higher than a predetermined threshold, it is determined that there is no telop character (step S20).

  If there is a point whose edge density is equal to or higher than a predetermined threshold value, a coordinate point having an edge density equal to or higher than the threshold value is set as a high-density edge region, and color information between edge pairs in the high-density edge region is extracted (step S14). The color information may be extracted in the RGB color system, or may be extracted after being converted into a color system such as CIE-Lab.

  Next, the color classification unit 15 classifies the extracted color by clustering using a predetermined clustering method (step S15). The telop character color estimation unit 16 estimates, as the telop character color, the number of colors belonging to the same classification or a color whose ratio to the total number of colors is equal to or greater than a predetermined threshold (step S16).

  Subsequently, the telop character area detection unit 17 has colors belonging to the same classification as the estimated color in an area within a predetermined range (for example, a rectangular area) around each edge pair having the estimated telop character color. The pixel is stored as a telop character area candidate, and the area occupied by the telop character area candidate is measured (step S17). That is, the number of pixels having a color in the same cluster as the estimated telop character color in an area within a predetermined range is counted. Then, it is determined whether the measured area or the ratio of the area in the region within the predetermined range is equal to or greater than a predetermined threshold (step S18). If the measured area is equal to or greater than the predetermined threshold, the telop character region candidate is determined as a telop character. The detected telop character area is displayed by the telop character area display unit 18 (step S19). If there is no telop character area candidate that is equal to or greater than the predetermined threshold, it is determined that there is no telop character (step S20).

  The processing contents of the above units will be described in more detail.

<Edge detection>
FIG. 3 is a diagram for explaining an example of edge detection according to this embodiment. FIG. 3A shows an example in which a vertical edge is detected from an image by checking a color change in the vertical direction, and FIG. 3B detects a horizontal edge from the image by checking a color change in the horizontal direction. An example is shown. Here, the point where the color changes in the vertical direction on the image is called a vertical edge, and the point where the color changes on the image in the horizontal direction is called a horizontal edge.

  Here, it is assumed that the color information of the pixel at the (i, j) coordinate on the image that is the detection target of the telop character area is represented by the vector ↑ c (i, j). i is the coordinate in the horizontal direction of the image, and j is the coordinate in the vertical direction of the image. ↑ c (i, j) is a three-dimensional vector and may be an RGB color system or a color system such as CIE-Lab.

  An example will be described in which it is determined whether or not the pixel at the coordinates (i, j) on the image is a vertical edge. As shown in FIG. 3A, from the relationship of the color information in the vertical direction of the surrounding pixels at coordinates (i, j), the vertical edge determination evaluation value V ( i, j).

V (i, j) = V x (i, j) 2 + V y (i, j) 2 + V z (i, j) 2 ... (1)
In the vertical edge detection function shown in Expression (1), V _x (i, j), V _y (i, j), and V _z (i, j) are obtained from the following functions, respectively.

V _x (i, j) = c _x (i−1, j−1) −c _x (i−1, j + 1)
+2 (c _x (i, j−1) −c _x (i, j + 1))
+ C _x (i + 1, j−1) −c _x (i + 1, j + 1),
_{V y (i, j) =} c y (i-1, j-1) -c y (i-1, j + 1)
_{+2 (c y (i, j} -1) -c y (i, j + 1))
_{+ C y (i + 1,} j-1) -c y (i + 1, j + 1),
V _z (i, j) = c _z (i−1, j−1) −c _z (i−1, j + 1)
+2 (c _z (i, j−1) −c _z (i, j + 1))
+ C _z (i + 1, j−1) −c _z (i + 1, j + 1)
Note that x, y, and z indicate elements of the color information vector ↑ c. For example, in the RGB color system, c _x , c _y , and c _z represent R, G, and B pixel values, respectively.

  Points whose evaluation value V (i, j) obtained from the equation (1) is equal to or greater than a predetermined threshold threshold_edge are regarded as vertical edges.

  The horizontal edge can be detected in the same manner as the vertical edge. An example will be described in which it is determined whether or not the pixel at the coordinates (i, j) on the image is a horizontal edge. As shown in FIG. 3B, from the relationship of the color information in the horizontal direction of the surrounding pixels at the coordinates (i, j), the horizontal edge determination evaluation value H ( i, j).

H (i, j) = H _x (i, j) ² + H _y (i, j) ² + H _z (i, j) ² (2)
In the horizontal edge detection function shown in Expression (2), H _x (i, j), H _y (i, j), and H _z (i, j) are obtained from the following functions, respectively.

H _x (i, j) = c _x (i−1, j−1) −c _x (i + 1, j−1)
+2 (c _x (i−1, j) −c _x (i + 1, j))
+ C _x (i−1, j + 1) −c _x (i + 1, j + 1),
_{H y (i, j) =} c y (i-1, j-1) -c y (i + 1, j-1)
_{+2 (c y (i-1} , j) -c y (i + 1, j))
_{+ C y (i-1,} j + 1) -c y (i + 1, j + 1),
H _z (i, j) = c _z (i−1, j−1) −c _z (i + 1, j−1)
+2 (c _z (i−1, j) −c _z (i + 1, j))
+ C _z (i−1, j + 1) −c _z (i + 1, j + 1)
Points whose evaluation value H (i, j) obtained from Equation (2) is equal to or greater than a predetermined threshold threshold_edge are regarded as horizontal edges.

<Edge pair detection>
FIG. 4 is a diagram for explaining an example of edge pair detection according to the present embodiment. FIG. 4 shows a part of an image which is a detection target of the telop character area. In the example of FIG. 4, the hatched portion indicates a part of the telop character on the image. In FIG. 4, thin line circles indicate vertical edge points detected by edge detection, and thick line circles indicate horizontal edge points detected by edge detection.

  This edge pair is detected as follows. It is assumed that the line width of the telop character is equal to or less than a predetermined threshold value threshold_height set in advance. With respect to h below this threshold threshold_height, vertical edge points (i, j), (V, i, j), V (i, j + h) are both equal to or higher than threshold threshold_edge, i, j + h).

  At this time, the point (i, j) and (i, j + h) of the vertical edge when the value obtained by the following equation (3) falls below a predetermined threshold threshold_var is defined as a vertical edge pair.

  In equation (3), Var () represents the variance.

  The horizontal edge pair can be detected in the same manner as the vertical edge pair. As shown in FIG. 4, the evaluation values H (i ′, j ′) and H (i ′ + h, j ′) of the horizontal edge determination are both equal to or greater than the threshold threshold_edge with respect to h equal to or less than the predetermined threshold threshold_height. It is assumed that there are such horizontal edge points (i ′, j ′), (i ′ + h, j ′).

  At this time, the point (i ′, j ′) and (i ′ + h, j ′) of the horizontal edge when the value obtained by the following equation (4) is below the predetermined threshold threshold_var is To do.

  In this way, a pair of edges with little variation in the color between two edges, that is, the color on the line connecting the two edges, is determined as an edge pair.

<Extraction of high-density edge region>
FIG. 5 is a diagram for explaining an example of high-density edge region extraction according to the present embodiment. Here, an example will be described in which it is determined whether a point of coordinates (i, j) on the image is a point of a high-density edge region.

  As shown in FIG. 5, a rectangular area having a height of height_dense-box × width of width_dense-box with the point (i, j) as the center is set as an edge density calculation area of the point (i, j). The vertical width height_dense-box and the horizontal width width_dense-box are preset constant values. The value obtained by dividing the number of edges detected from the edge density calculation area shown in FIG. 5 by the area of the rectangular area that is the edge density calculation area is the edge density of the point (i, j). When the edge density value of the obtained point (i, j) exceeds a predetermined threshold threshold_edge-dense, the point (i, j) is set as a high-density edge region.

  Instead of obtaining the edge density from the number of all edges detected from the edge density calculation region, the edge density may be obtained from the number of edge pairs detected from the edge density calculation region.

<Color extraction / clustering>
For all the detected edge pairs, when the two edges forming the pair are both in the high-density edge region, the color on the line segment connecting the edge pairs is extracted. The extracted colors are clustered to estimate the colors used in the characters.

  In the extraction of colors between edge pairs, for example, one color may be extracted between edge pairs, such as the average value or intermediate value of the colors of all pixels between edge pairs, or the color of all pixels between edge pairs may be 1 You may make it extract all one by one. Further, the color of several pixels may be sampled from all the pixels between the edge pairs.

In the clustering of the extracted colors, for example, when the color information is obtained in the RGB color system, clustering may be performed using the k-means method (see Reference 1) while maintaining the RGB color information. .
[Reference 1]: Toshihiro Kamisu (National Institute of Advanced Industrial Science and Technology), clustering technique (1) in the field of data mining, Journal of Artificial Intelligence Society Vol.

In addition, color information obtained by RGB may be converted into Lab space (see Reference 2).
[Reference 2]: Noboru Ota, Color Engineering, Tokyo Denki University Press.

  FIG. 6 is a diagram illustrating an example of color clustering in the Lab space according to the present embodiment. Clustering may be performed after the color extracted from the high-density edge region is converted into Lab space to form cylindrical coordinates as shown in FIG. In FIG. 6, a dotted line frame indicates a cluster of classified colors, and a circle in the dotted line frame indicates a color of an edge pair from the high-density edge region classified into the cluster. Even if the colors of different edge pairs are the same color, they are not classified as one color, but are classified separately.

  Since edge pairs appear characteristically in the telop area on the image, if the colors extracted from the edge pairs are clustered, the colors used for the telop characters in the color space shown in FIG. Appear in the same area. When two colors are used for a telop character, two regions in which colors extracted from between edge pairs are concentrated appear in a color space as shown in FIG.

<Estimation of character color>
In the estimation of the character color used for the telop, the number of colors classified into the same cluster is counted, and the cluster whose number exceeds the predetermined threshold threshold_color_count is used as the color used for the character. It is determined that this is a cluster. Note that the color count is performed separately even when a plurality of the same colors are extracted in the color extraction from the edge pair. When only one representative color is extracted from one edge pair at the time of the color clustering described above, the color to be counted and the edge pair have a one-to-one relationship. Note that there may be a plurality of clusters determined as the color clusters used for the characters.

  Rather than determining whether the number of colors exceeds the threshold threshold_color_count, the ratio of the number of colors classified into the same cluster to the number of colors of all clusters is determined according to whether the number is equal to or greater than a predetermined threshold. , It is possible to determine whether the color is a telop character color.

<Detection of telop character area>
In the detection of the character area of the telop, when the character area is in the high-density edge area and the color at that point is in the color area recognized as the character color, that point is set as the character area. Further, the following method can be used to increase the accuracy of detecting the telop character area.

  FIG. 7 is a diagram for explaining a telop character area recognition method according to this embodiment. First, the character color search range is determined on the original image. Here, a range of a predetermined size around an edge pair having a color estimated as a telop character color by clustering is determined as a search range. For example, a rectangular range having a predetermined vertical and horizontal size around each edge pair in the high-density edge region is determined as a search range. In the example of FIG. 7, a rectangular search range A1 centered on the edge pair E1 and a rectangular search range A2 centered on the edge pair E2 are the telop character area candidates. The search range is.

  If there is a pixel that falls within the color range of the cluster to which the color estimated as the telop character color belongs in each search range, that pixel is stored as a telop character region candidate. After searching for the color within the search range, the area of the telop character area candidate is measured by counting the number of pixels of the telop character area candidate. The area candidate is recognized as a telop character area. If the area is smaller than a predetermined threshold, the telop character area candidate is not recognized as a telop character area. The designation of the area threshold may be a predetermined constant value, or may be a value indicating a ratio such as 15% of the search range.

<Output / display of telop character area>
By detecting the telop character area as described above, the telop character area on the image can be distinguished from other areas. For example, the telop character area is binarized to “1” and the other areas are “0”. Output information. Alternatively, the telop character area on the image is displayed separately from the other areas. A minimum rectangular area surrounding the telop character area may be output or displayed.

  The above-described processing for detecting a telop character area on an image can be realized by a computer and a software program, and the program can be recorded on a computer-readable recording medium or provided through a network. .

DESCRIPTION OF SYMBOLS 10 Telop character area detection apparatus 11 Image input part 12 Image memory | storage part 13 Edge detection part 131 Edge pair detection part 14 High-density edge area extraction part 15 Color classification part 16 Telop character color estimation part 17 Telop character area detection part 18 Telop character area display Part

Claims (5)

In a telop character area detection method for detecting a telop character area from an image,
Inputting an image to be detected in the telop character area and storing it in a storage device;
Extracting an edge from the image and detecting a pair of color change points constituting the edge as an edge pair;
Extracting a high-density edge region in which the density of the edge or the edge pair is equal to or higher than a predetermined threshold from the image;
Extracting and classifying a color on a line connecting the edge pairs in the high-density edge region; and
A process of estimating the number of colors belonging to the same classification or the ratio of the number of colors to the total number of colors as a telop character color that exceeds a predetermined threshold;
And a step of detecting a telop character region in the image from the estimated color of the telop character. The telop character area detection method according to claim 1,
In the process of detecting the telop character area,
A pixel having a color belonging to the same classification as the estimated color in an area within a predetermined range around each edge pair having the estimated telop character color is stored as a telop character area candidate, and the telop character area A telop character region detection method, wherein a telop character region candidate whose area occupied by a candidate or a ratio of the area in the region within the predetermined range is equal to or greater than a predetermined threshold is detected as a telop character region. In a telop character area detection device for detecting a telop character area from an image,
Means for inputting an image to be detected in the telop character area and storing it in a storage device;
Means for extracting an edge from the image and detecting a pair of color change points constituting the edge as an edge pair;
Means for extracting from the image a high-density edge region in which the density of the edge or the edge pair is equal to or higher than a predetermined threshold;
Means for extracting and classifying a color on a line connecting the edge pairs in the high-density edge region;
Means for estimating the number of colors belonging to the same classification or a color whose ratio to the total number of colors is equal to or greater than a predetermined threshold as a telop character color;
And a means for detecting a telop character area in the image from the estimated color of the telop character. In the telop character area detection device according to claim 3,
The means for detecting the telop character area is:
A pixel having a color belonging to the same classification as the estimated color in an area within a predetermined range around each edge pair having the estimated telop character color is stored as a telop character area candidate, and the telop character area A telop character area detection device, wherein a telop character area candidate whose area occupied by a candidate or a ratio of the area in the area within the predetermined range is equal to or greater than a predetermined threshold is detected as a telop character area.   A telop character area detection program for causing a computer to execute the telop character area detection method according to claim 1.

Images