CN111563505A - Character detection method and device based on pixel segmentation and merging - Google Patents

Abstract

The present invention provides a method and device for character detection based on pixel segmentation and merging. The method includes: extracting feature information of a picture to be detected, generating a feature map corresponding to the picture to be detected according to the extracted feature information; performing pixel segmentation on the feature map to obtain Multiple pixel points, analyze the confidence score of each pixel belonging to the text pixel point; extract the position information of the pixel point within the preset reliability score range in the text box to which it belongs, and form it according to the position information of the extracted pixel point. Multiple connected domains of pixel points; combine the pixel points in the same connected domain, and use the combined pixel points to determine the text area on the image to be detected. The invention firstly divides the pixel of the feature map of the picture to be detected, then forms a plurality of connected domains according to the position information of the text pixel points, and combines the pixel points in each connected domain, so that the combined pixel points can effectively, Accurately determine the text area on the image to be detected.

Description

A method and device for character detection based on pixel segmentation and merging

technical field

The present invention relates to the technical field of text detection, and in particular, to a text detection method and device based on pixel segmentation and merging.

Background technique

In the prior art, general text detection technologies mainly include text detection in natural scenes and text detection in text scanned images. There are currently two common methods. The first is text detection based on target detection methods. Convolutional neural network Extract the confidence and text box coordinates corresponding to each point of the feature maps of different scales, and then deduplicate all candidate boxes through Non-Maximum Suppression (NMS); the other is to use convolutional neural networks. Segmentation, return the confidence of whether each pixel belongs to the text and the coordinates of the text box.

However, the above two methods have different problems, so they cannot be used as a general text detection technology. The first method cannot accurately fit the coordinate position of the pixel when the text has a large rotation angle. The second method The method cannot completely detect long texts due to the limitation of the size of the convolution kernel.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention is proposed in order to provide a method and device for character detection based on pixel segmentation and merging that overcome the above problems or at least partially solve the above problems.

According to one aspect of the present invention, there is provided a text detection method based on pixel segmentation and merging, comprising:

Extracting feature information of the picture to be detected, and generating a feature map corresponding to the picture to be detected according to the extracted feature information;

Perform pixel segmentation on the feature map to obtain a plurality of pixel points, and analyze the confidence score of each pixel point belonging to a text pixel point;

Extracting the position information of the pixel points within the preset reliability score range in the text box to which they belong, and forming a plurality of connected domains of the pixel points according to the position information of the extracted pixel points;

Pixel points in the same connected domain are combined, and the combined pixel points are used to determine the text area on the picture to be detected.

Optionally, the feature information of the picture to be detected is extracted, and the feature map corresponding to the picture to be detected is generated according to the extracted feature information, including:

Extract the feature information of the picture to be detected based on the deep learning model of the UNet network structure;

Perform up-sampling, down-sampling and corresponding convolution operations on the picture to be detected according to the extracted feature information to obtain a feature map corresponding to the picture to be detected.

Optionally, the greater the confidence score of the pixel belonging to the text pixel, the greater the probability of it belonging to the text pixel and the smaller the probability of belonging to the picture background;

The smaller the confidence score of the pixel belonging to the text pixel, the lower the probability of the pixel belonging to the text pixel and the higher the probability of belonging to the picture background.

Optionally, extract the position information of the pixels within the preset reliability score range in the text box to which they belong, including:

Obtain the pixels whose confidence score is greater than the preset score from the plurality of pixels obtained by segmentation;

Extract the coordinate value of the pixel point greater than the preset score in the text box to which it belongs.

Optionally, forming a plurality of connected domains of pixel points according to the extracted position information of the pixel points, including:

According to the coordinate value of the extracted arbitrary pixel point, determine whether the arbitrary pixel point is cascaded with the pixel point in the specified direction in the text box to which it belongs;

If so, the arbitrary pixel is cascaded with the pixels in the specified direction in the text box to which it belongs;

A plurality of connected domains are formed according to the concatenated pixel points, wherein the pixel points belonging to the same connected domain are concatenated with each other.

Optionally, forming a plurality of connected domains of pixel points according to the extracted position information of the pixel points, including:

Binarization processing is performed on the feature map, and a plurality of connected domains of pixel points are formed in the image after the binarization processing according to the position information of the extracted pixel points.

Optionally, the pixels in the same connected domain are combined, and the combined pixels are used to determine the text area on the picture to be detected, including:

Merge the pixel points in the same connected domain, and locate the position information of the corresponding text box according to the position information of the merged pixel points;

The text area on the picture to be detected is determined according to the position information of the text box.

According to another aspect of the present invention, a text detection device based on pixel segmentation and merging is provided, comprising:

a generating module, adapted to extract feature information of the picture to be detected, and generate a feature map corresponding to the picture to be detected according to the extracted feature information;

an analysis module, adapted to perform pixel segmentation on the feature map to obtain a plurality of pixel points, and analyze the confidence score of each pixel point belonging to a text pixel point;

A forming module is suitable for extracting the position information of the pixel points within the preset reliability score range in the text frame to which it belongs, and forms a plurality of connected domains of the pixel points according to the position information of the extracted pixel points;

The merging module is adapted to merge the pixel points in the same connected domain, and use the merged pixel points to determine the text area on the picture to be detected.

Optionally, the generating module is further adapted to:

Extract the feature information of the picture to be detected based on the deep learning model of the UNet network structure;

Perform up-sampling, down-sampling and corresponding convolution operations on the picture to be detected according to the extracted feature information to obtain a feature map corresponding to the picture to be detected.

Optionally, the greater the confidence score of the pixel belonging to the text pixel, the greater the probability of it belonging to the text pixel and the smaller the probability of belonging to the picture background;

The smaller the confidence score of the pixel belonging to the text pixel, the lower the probability of the pixel belonging to the text pixel and the higher the probability of belonging to the picture background.

Optionally, the forming module includes an acquisition unit and an extraction unit,

The obtaining unit is adapted to obtain pixels whose confidence score is greater than a preset score from a plurality of pixels obtained by segmentation;

The extraction unit is adapted to extract the coordinate value of the pixel point greater than the preset score in the text box to which it belongs.

Optionally, the forming module also includes:

The judgment unit is adapted to judge whether the arbitrary pixel is cascaded with the pixel in the specified direction in the text box to which it belongs, according to the coordinate value of the extracted arbitrary pixel;

Forming unit, suitable for if the judging unit determines that the arbitrary pixel is cascaded with the pixel in the specified direction in the text box to which it belongs, then the arbitrary pixel is cascaded with the pixel in the specified direction in the text box to which it belongs;

A plurality of connected domains are formed according to the concatenated pixel points, wherein the pixel points belonging to the same connected domain are concatenated with each other.

Optionally, the forming module is further adapted to:

Binarization processing is performed on the feature map, and a plurality of connected domains of pixel points are formed in the image after the binarization processing according to the position information of the extracted pixel points.

Optionally, the merging module is further adapted to:

Merge the pixel points in the same connected domain, and locate the position information of the corresponding text box according to the position information of the merged pixel points;

The text area on the picture to be detected is determined according to the position information of the text box.

According to yet another aspect of the present invention, a computer storage medium is also provided, the computer storage medium stores computer program code, when the computer program code is executed on a computing device, causes the computing device to perform any of the above implementations. The method of text detection based on pixel segmentation and merging described in the example.

According to yet another aspect of the present invention, there is also provided a computing device, comprising: a processor; a memory storing computer program code; when the computer program code is executed by the processor, it causes the computing device to execute the above The method for character detection based on pixel segmentation and merging described in any embodiment.

In the embodiment of the present invention, the feature information of the picture to be detected is extracted, a feature map corresponding to the picture to be detected is generated according to the extracted feature information, and then a plurality of pixels are obtained by pixel segmentation of the feature map, and it is analyzed that each pixel belongs to a text pixel The confidence score of the point, extract the position information of the pixel point within the preset reliability score range in the text box to which it belongs, and form multiple connected domains of the pixel point according to the position information of the extracted pixel point. The pixel points in the combination are combined, and the combined pixel points are used to determine the text area on the image to be detected. Therefore, in the embodiment of the present invention, pixel segmentation is performed on the feature map of the image to be detected, and then a plurality of connected domains are formed according to the position information of the text pixels, and the pixels in each connected domain are merged. Corresponding to a text, so that the text area on the image to be detected can be effectively and accurately determined according to the merged pixel points. Further, since the present invention uses the connected domain formed between the pixels to determine the text area, it can effectively detect the rotated text and long text on the picture to be detected.

The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.

The above and other objects, advantages and features of the present invention will be more apparent to those skilled in the art from the following detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered limiting of the invention. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

1 shows a schematic flowchart of a text detection method based on pixel segmentation and merging according to an embodiment of the present invention;

2 shows a schematic diagram of a text detection process based on pixel segmentation and merging according to an embodiment of the present invention;

FIG. 3 shows a schematic structural diagram of a text detection device based on pixel segmentation and merging according to an embodiment of the present invention; and

FIG. 4 shows a schematic structural diagram of a character detection apparatus based on pixel division and merging according to another embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.

In order to solve the above technical problem, an embodiment of the present invention provides a text detection method based on pixel segmentation and merging. FIG. 1 shows a schematic flowchart of a text detection method based on pixel segmentation and merging according to an embodiment of the present invention. Referring to FIG. 1, the method includes at least steps S102 to S108.

Step S102, extracting feature information of the picture to be detected, and generating a feature map corresponding to the picture to be detected according to the extracted feature information.

In step S104, pixel segmentation is performed on the feature map to obtain a plurality of pixel points, and the confidence score of each pixel point belonging to a text pixel point is analyzed.

Step S106 , extracting the position information of the pixel points within the preset reliability score range in the text box to which they belong, and forming a plurality of connected domains of the pixel points according to the position information of the extracted pixel points.

In step S108, the pixel points in the same connected domain are combined, and the combined pixel points are used to determine the text area on the picture to be detected.

In the embodiment of the present invention, the feature information of the picture to be detected can be extracted first, the feature map corresponding to the picture to be detected can be generated according to the extracted feature information, and then the feature map is divided into pixels to obtain a plurality of pixels, and it is analyzed that each pixel belongs to the text The confidence score of the pixel point, extract the position information of the pixel point within the preset reliability score range in the text box to which it belongs, and then form a plurality of connected domains of the pixel point according to the position information of the extracted pixel point. Pixel points in the connected domain are combined, and the combined pixel points are used to determine the text area on the image to be detected. Thus, the embodiment of the present invention forms a plurality of connected domains according to the position information of a plurality of pixel points obtained by dividing the image to be detected, and then combines the pixel points in each connected domain, so that each connected domain corresponds to a character, Therefore, the text area on the picture to be detected can be determined effectively and accurately according to the merged pixel points. Further, since the present invention uses the connected domain formed between the pixels to determine the text area, it can effectively detect the rotated text and long text on the picture to be detected.

Referring to step S102 above, in an embodiment of the present invention, when extracting the feature information of the picture to be detected, and generating the feature map corresponding to the picture to be detected according to the extracted feature information, the picture to be detected can be extracted based on the deep learning model of the UNet network structure Then perform up-sampling, down-sampling and corresponding convolution operations on the image to be detected according to the extracted feature information to obtain the feature map corresponding to the image to be detected, that is, the corresponding feature is obtained after the image to be detected is processed by convolution kernel. picture.

The feature map here can refer to the high-level semantic feature map, which is essentially a matrix. For example, the high-level semantic feature map is essentially a three-dimensional matrix of 100 by 100 by 10, then each 100 by 100 matrix in this three-dimensional matrix can be viewed is a feature map.

In an embodiment of the present invention, in a score area, the greater the confidence score of a pixel belonging to a text pixel, the greater the probability of belonging to a text pixel and the lower the probability of belonging to the picture background. On the contrary, the pixel The smaller the confidence score of the point belonging to the text pixel, the smaller the probability of it belonging to the text pixel and the higher the probability of belonging to the picture background.

For example, in the pixel's score area 0-1, the confidence score of the pixel is 0, which means that the pixel belongs to the background completely, not the text. The confidence score of the pixel is 1, which means that the pixel is It belongs entirely to the text, not to the background. The closer the confidence score is to 1, the greater the probability that the pixel belongs to the text pixel, the lower the probability of belonging to the background, the closer the confidence score is to 0, the higher the probability that the pixel belongs to the background, and the higher the probability of belonging to the text pixel. Small. Assuming that the confidence score of a pixel is 0.1, it can be determined that the pixel has a high probability of belonging to the background, and a small probability of belonging to the text.

In this embodiment, the text pixel is a pixel representing a point on the text. Generally, if the image to be detected contains text, the text is formed by combining a plurality of text pixels. The probability and confidence score that a pixel is a text pixel can represent the possibility that the pixel is a text pixel.

In an embodiment of the present invention, multiple convolution layers in the deep learning model of the UNet network structure can be used to perform a convolution operation on the extracted feature information of the image to be detected, so as to obtain the confidence that each pixel is a text pixel. Degree score. Certainly, other network models may also be used in this embodiment of the present invention to extract feature maps corresponding to the pictures to be detected, and the types of network models are not specifically limited here.

Referring to step S104 above, in an embodiment of the present invention, a convolutional neural network may be used to segment the feature map to obtain multiple pixels, and then analyze the confidence score that each pixel belongs to a text pixel. value, the convolutional neural network here may be a UNet network, and of course other convolutional neural networks may also be used, which is not specifically limited in this embodiment of the present invention.

Referring to the above step S106, in an embodiment of the present invention, when extracting the position information of the pixel points within the preset reliability score range in the text box to which they belong, the confidence score is first obtained from a plurality of pixel points obtained by segmentation. For pixels whose value is greater than the preset score, that is, select the pixel with a high probability of belonging to the text pixel, and then extract the coordinate value of the pixel whose value is greater than the preset score in the text box to which it belongs. For example, the preset reliability score range is greater than 0.7 and less than 1, then the preset score is set to 0.7. If the confidence score of a pixel is greater than 0.7, then the pixel has a high probability of belonging to a text pixel. At this time It can be considered that the pixel is a text pixel.

Continue to refer to step S106 above, in an embodiment of the present invention, when multiple connected domains of pixels are formed according to the extracted position information of the pixel, it can be determined whether the arbitrary pixel is based on the coordinate value of the extracted arbitrary pixel. It is cascaded with the pixels in the specified direction in the text box to which it belongs. If so, it can be determined that the arbitrary pixel is cascaded with the pixels in the specified direction in the text box to which it belongs, and further, according to the mutually cascaded pixels, multiple pixels can be formed. A connected domain, in which the pixels belonging to a connected domain are cascaded with each other.

In this embodiment, the pixel points in the specified direction in the text box to which any pixel point belongs may be four pixels in the upper, lower, left, and right directions in the text box where the arbitrary pixel point is located, or may also be any pixel. Eight pixel points in the upper, lower, left, right, diagonally up (left diagonally up and right diagonally up), and diagonally down (left diagonally down, right diagonally down) directions in the text box where the point is located. The direction is not specifically limited.

In an embodiment of the present invention, when multiple connected domains of pixels are formed according to the position information of the extracted pixels, the feature map may also be binarized first, so that the binarized image can be extracted according to the extracted position information. The position information of the pixel points forms multiple connected domains of the pixel points.

Referring to the above step S108, in an embodiment of the present invention, after merging the pixels in the same connected domain, the position information of the corresponding text frame can be located according to the position information of the merged pixel points, and then according to the position information of the text frame The location information determines the text area on the image to be detected. For example, the position of the text box is located according to the coordinate value of the combined pixel point relative to the text box to which it belongs. One text corresponds to one text box, and the determined text area is a combination of multiple text boxes.

In this embodiment, the positional information of the text can be defined by the positional information of the text box around the text, and the text box is usually in a rectangular shape, such as the smallest circumscribed rectangle around the text. When locating the position information of the corresponding text box, the coordinates of the four vertexes of the text frame can be used to represent the position of the text frame, that is, the position information of the text frame can be located by outputting the coordinate values of the four vertexes of the text frame. Or it can be represented by the coordinates of one vertex of the text box and the width and height of the text box.

After the text area on the picture is detected, in an embodiment of the present invention, the text in the text area may be further recognized, for example, OCR (Optical Character Recognition, Optical Character Recognition) recognition is used to recognize the Chinese in the picture The text area specifically contains those text information. For some units related to network security, this method can be used to determine whether there is illegal text in the image. For example, for a picture containing text information on pornography, gambling and drug use, the solution of the present invention can be used to quickly and accurately detect the text area in the picture that involves textual information on pornography, gambling and drug use, and then use the OCR technology to identify the specific text content. When it comes to illegal words and pictures, find out the source of the website or picture released, and take corresponding measures to deal with it.

Referring to FIG. 2 , the text detection method according to the embodiment of the present invention is introduced by taking the UNet network structure deep learning model as an example.

After the picture to be detected (ie, the original picture) is input into the deep learning model of the UNet network structure, the feature information of the picture to be detected is extracted by the deep learning model, and then the extracted feature information is input into the convolution unit, so that the The product unit performs up-sampling, down-sampling and corresponding convolution operations on the image to be detected according to the extracted feature information, so as to obtain a feature map corresponding to the image to be detected and generate a corresponding feature map. The conv block pool shown in Figure 2 represents the convolution operation corresponding to downsampling, and the conv block uppool represents the convolution operation corresponding to upsampling. After pixel segmentation of the feature map to obtain multiple pixels, the confidence score linkscore of each pixel belonging to a text pixel can be analyzed. Here, score represents the confidence score of a text pixel, and the 4 after the link score Represents the confidence score of the four pixels in the four directions of up, down, left, and right in the text box that are cascaded with the text pixel. Further, the position information of the pixels whose confidence score is greater than the preset score relative to the text box to which they belong is extracted. Here, the 5 behind the text box represents the boundary distance of a pixel from the upper, lower, left, right, and other sides of the text box. The 5 values of the angle of rotation.

Based on the same inventive concept, an embodiment of the present invention further provides an apparatus for character detection based on pixel division and merging. FIG. 3 shows a schematic structural diagram of an apparatus for character detection based on pixel division and merging according to an embodiment of the present invention. Referring to FIG. 3 , the apparatus 300 for character detection based on pixel segmentation and merging includes a generating module 310 , an analyzing module 320 , a forming module 330 and a merging module 340 .

The function of each component or device and the connection relationship between each part of the text detection apparatus 300 based on pixel division and merging according to the embodiment of the present invention are now introduced:

The generating module 310 is adapted to extract feature information of the picture to be detected, and generate a feature map corresponding to the picture to be detected according to the extracted feature information;

The analysis module 320, coupled with the generation module 310, is adapted to perform pixel segmentation on the feature map to obtain a plurality of pixel points, and analyze the confidence score of each pixel point belonging to the text pixel point;

The forming module 330, coupled with the analysis module 320, is adapted to extract the position information of the pixel points within the preset reliability score range in the text box to which they belong, and form a plurality of connected domains of the pixel points according to the position information of the extracted pixel points ;

The combining module 340, coupled with the forming module 330, is adapted to combine the pixel points in the same connected domain, and use the combined pixel points to determine the text area on the picture to be detected.

In an embodiment of the present invention, the generation module 310 is further adapted to extract feature information of the picture to be detected based on the UNet network structure deep learning model, and perform upsampling, downsampling and corresponding convolution operations on the picture to be detected according to the extracted feature information , to obtain the feature map corresponding to the image to be detected.

In an embodiment of the present invention, the greater the confidence score of the pixel belonging to the text pixel, the greater the probability of belonging to the text pixel and the lower the probability of belonging to the picture background;

The smaller the confidence score of the pixel belonging to the text pixel, the lower the probability of it belonging to the text pixel and the higher the probability of belonging to the picture background.

An embodiment of the present invention further provides another apparatus for character detection based on pixel division and merging. FIG. 4 shows a schematic structural diagram of an apparatus for character detection based on pixel division and merging according to another embodiment of the present invention. Referring to FIG. 4 , the forming module 330 in the apparatus 300 for character detection based on pixel segmentation and merging further includes an acquiring unit 331 , an extracting unit 332 , a judging unit 333 , and a forming unit 334 .

The obtaining unit 331 is adapted to obtain a pixel point whose confidence score is greater than a preset score from a plurality of pixel points obtained by segmentation.

The extracting unit 332, coupled with the acquiring unit 331, is adapted to extract the coordinate value of the pixel point greater than the preset score in the text box to which it belongs.

The judging unit 333, coupled with the extracting unit 332, is adapted to judge whether the arbitrary pixel point is cascaded with the pixel point in the specified direction in the text box to which it belongs according to the coordinate value of the extracted arbitrary pixel point.

The forming unit 334, coupled with the judging unit 333, is adapted to if the judging unit 333 determines that the arbitrary pixel is cascaded with the pixel in the specified direction in the text box to which it belongs, then the arbitrary pixel and the pixel in the specified direction in the text box to which it belongs are cascaded. The pixel points are cascaded, and then a plurality of connected domains are formed according to the mutually cascaded pixel points, wherein the pixel points belonging to the same connected domain are cascaded with each other.

In an embodiment of the present invention, the forming module 330 is further adapted to perform binarization processing on the feature map, and form a plurality of connected domains of pixel points in the binarized image according to the position information of the extracted pixel points.

In an embodiment of the present invention, the merging module 340 is further adapted to merge the pixel points in the same connected domain, locate the position information of the corresponding text frame according to the position information of the merged pixel points, and determine the position information of the text frame according to the position information of the text frame. The text area on the image to be detected.

Embodiments of the present invention further provide a computer storage medium, where the computer storage medium stores computer program codes, when the computer program codes are executed on a computing device, the computing device causes the computing device to execute the text based on pixel division and combination in any of the above embodiments method of detection.

An embodiment of the present invention also provides a computing device, comprising: a processor; a memory storing computer program code; when the computer program code is executed by the processor, it causes the computing device to perform the pixel division-based combination in any of the above embodiments method of text detection.

According to any one of the above-mentioned preferred embodiments or a combination of multiple preferred embodiments, the embodiments of the present invention can achieve the following beneficial effects:

In the embodiment of the present invention, the feature information of the picture to be detected is extracted, a feature map corresponding to the picture to be detected is generated according to the extracted feature information, and then a plurality of pixels are obtained by pixel segmentation of the feature map, and it is analyzed that each pixel belongs to a text pixel The confidence score of the point, extract the position information of the pixel point within the preset reliability score range in the text box to which it belongs, and form multiple connected domains of the pixel point according to the position information of the extracted pixel point. The pixel points in the combination are combined, and the combined pixel points are used to determine the text area on the image to be detected. Therefore, in the embodiment of the present invention, pixel segmentation is performed on the feature map of the image to be detected, and then a plurality of connected domains are formed according to the position information of the text pixels, and the pixels in each connected domain are merged. Corresponding to a text, so that the text area on the image to be detected can be effectively and accurately determined according to the merged pixel points. Further, since the present invention uses the connected domain formed between the pixels to determine the text area, it can effectively detect the rotated text and long text on the picture to be detected.

Those skilled in the art can clearly understand that, for the specific working processes of the systems, devices and units described above, reference may be made to the corresponding processes in the foregoing method embodiments, and for the sake of brevity, details are not described herein.

In addition, each functional unit in each embodiment of the present invention may be physically independent of each other, or two or more functional units may be integrated together, or all functional units may be integrated into one processing unit. The above-mentioned integrated functional units may be implemented in the form of hardware, and may also be implemented in the form of software or firmware.

Those skilled in the art can understand that: if the integrated functional unit is implemented in the form of software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, which includes several instructions to make a computer A computing device (such as a personal computer, a server, or a network device, etc.) executes all or part of the steps of the methods described in the embodiments of the present invention when running the instructions. The aforementioned storage medium includes: a U disk, a removable hard disk, a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk and other media that can store program codes.

Alternatively, all or part of the steps of implementing the foregoing method embodiments may be accomplished by program instructions related to hardware (such as a personal computer, a server, or a computing device such as a network device), and the program instructions may be stored in a computer-readable storage In the medium, when the program instructions are executed by the processor of the computing device, the computing device executes all or part of the steps of the methods described in the embodiments of the present invention.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: Within the spirit and principle of the present invention, it is still possible to modify the technical solutions recorded in the foregoing embodiments, or to perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the corresponding technical solutions deviate protection scope of the present invention.

The embodiment of the present invention provides A1, a text detection method based on pixel segmentation and merging, including:

Extracting feature information of the picture to be detected, and generating a feature map corresponding to the picture to be detected according to the extracted feature information;

Perform pixel segmentation on the feature map to obtain a plurality of pixel points, and analyze the confidence score of each pixel point belonging to a text pixel point;

Extracting the position information of the pixel points within the preset reliability score range in the text box to which they belong, and forming a plurality of connected domains of the pixel points according to the position information of the extracted pixel points;

Pixel points in the same connected domain are combined, and the combined pixel points are used to determine the text area on the picture to be detected.

A2. The method according to A1, wherein the extracting feature information of the picture to be detected, and generating a feature map corresponding to the picture to be detected according to the extracted feature information, includes:

Extract the feature information of the picture to be detected based on the deep learning model of the UNet network structure;

Perform up-sampling, down-sampling and corresponding convolution operations on the picture to be detected according to the extracted feature information to obtain a feature map corresponding to the picture to be detected.

A3. The method according to A1 or A2, wherein,

The greater the confidence score of the pixel point belonging to the text pixel point, the greater the probability of it belonging to the text pixel point and the smaller the probability of belonging to the picture background;

The smaller the confidence score of the pixel belonging to the text pixel, the lower the probability of the pixel belonging to the text pixel and the higher the probability of belonging to the picture background.

A4. The method according to A1 or A2, wherein extracting the position information of the pixels within the preset reliability score range in the text box to which they belong includes:

Obtain the pixels whose confidence score is greater than the preset score from the plurality of pixels obtained by segmentation;

Extract the coordinate value of the pixel point greater than the preset score in the text box to which it belongs.

A5. The method according to A4, wherein forming a plurality of connected domains of pixels according to the extracted position information of the pixels includes:

According to the coordinate value of the extracted arbitrary pixel point, determine whether the arbitrary pixel point is cascaded with the pixel point in the specified direction in the text box to which it belongs;

If so, the arbitrary pixel is cascaded with the pixels in the specified direction in the text box to which it belongs;

A plurality of connected domains are formed according to the concatenated pixel points, wherein the pixel points belonging to the same connected domain are concatenated with each other.

A6. The method according to A1 or A2, wherein the forming a plurality of connected domains of the pixel points according to the extracted position information of the pixel points includes:

Binarization processing is performed on the feature map, and a plurality of connected domains of pixel points are formed in the image after the binarization processing according to the position information of the extracted pixel points.

A7. The method according to A1 or A2, wherein the pixel points in the same connected domain are combined, and the combined pixel points are used to determine the text area on the picture to be detected, including:

Merge the pixel points in the same connected domain, and locate the position information of the corresponding text box according to the position information of the merged pixel points;

The text area on the picture to be detected is determined according to the position information of the text box.

B8. A text detection device based on pixel segmentation and merging, comprising:

a generating module, adapted to extract feature information of the picture to be detected, and generate a feature map corresponding to the picture to be detected according to the extracted feature information;

an analysis module, adapted to perform pixel segmentation on the feature map to obtain a plurality of pixel points, and analyze the confidence score of each pixel point belonging to a text pixel point;

A forming module is suitable for extracting the position information of the pixel points within the preset reliability score range in the text frame to which it belongs, and forms a plurality of connected domains of the pixel points according to the position information of the extracted pixel points;

The merging module is adapted to merge the pixel points in the same connected domain, and use the merged pixel points to determine the text area on the picture to be detected.

B9. The apparatus according to B8, wherein the generating module is further adapted to:

Extract the feature information of the picture to be detected based on the deep learning model of the UNet network structure;

Perform up-sampling, down-sampling and corresponding convolution operations on the picture to be detected according to the extracted feature information to obtain a feature map corresponding to the picture to be detected.

B10. The device according to B8 or B9, wherein,

The greater the confidence score of the pixel point belonging to the text pixel point, the greater the probability of it belonging to the text pixel point and the smaller the probability of belonging to the picture background;

The smaller the confidence score of the pixel belonging to the text pixel, the lower the probability of the pixel belonging to the text pixel and the higher the probability of belonging to the picture background.

B11. The apparatus according to B8 or B9, wherein the forming module includes an acquisition unit and an extraction unit,

The obtaining unit is adapted to obtain pixels whose confidence score is greater than a preset score from a plurality of pixels obtained by segmentation;

The extraction unit is adapted to extract the coordinate value of the pixel point greater than the preset score in the text box to which it belongs.

B12. The device according to B11, wherein the forming module further comprises:

The judgment unit is adapted to judge whether the arbitrary pixel is cascaded with the pixel in the specified direction in the text box to which it belongs, according to the coordinate value of the extracted arbitrary pixel;

Forming unit, suitable for if the judging unit determines that the arbitrary pixel is cascaded with the pixel in the specified direction in the text box to which it belongs, then the arbitrary pixel is cascaded with the pixel in the specified direction in the text box to which it belongs;

A plurality of connected domains are formed according to the concatenated pixel points, wherein the pixel points belonging to the same connected domain are concatenated with each other.

B13. The device according to B8 or B9, wherein the forming module is further adapted to:

Binarization processing is performed on the feature map, and a plurality of connected domains of pixel points are formed in the image after the binarization processing according to the position information of the extracted pixel points.

B14. The apparatus according to B8 or B9, wherein the merging module is further adapted to:

Merge the pixel points in the same connected domain, and locate the position information of the corresponding text box according to the position information of the merged pixel points;

The text area on the picture to be detected is determined according to the position information of the text box.

C15. A computer storage medium, which stores computer program code, which, when the computer program code is executed on a computing device, causes the computing device to perform the pixel-based segmentation described in any one of A1-A7 Merged text detection method.

C16. A computing device, comprising: a processor; a memory storing computer program code; when the computer program code is executed by the processor, it causes the computing device to execute the method based on any one of A1-A7. A method for pixel segmentation and merged text detection.

Claims (10)

1. A character detection method based on pixel segmentation and merging comprises the following steps:

extracting the characteristic information of a picture to be detected, and generating a characteristic diagram corresponding to the picture to be detected according to the extracted characteristic information;

carrying out pixel segmentation on the characteristic image to obtain a plurality of pixel points, and analyzing confidence scores of the pixel points belonging to the character pixel points;

extracting the position information of the pixel points within the preset confidence score range in the text box to which the pixel points belong, and forming a plurality of connected domains of the pixel points according to the extracted position information of the pixel points;

and merging the pixel points in the same connected domain, and determining the character region on the picture to be detected by using the merged pixel points.

2. The method according to claim 1, wherein the extracting feature information of the picture to be detected and generating the feature map corresponding to the picture to be detected according to the extracted feature information includes:

extracting the characteristic information of the picture to be detected based on a UNet network structure deep learning model;

and performing up-sampling, down-sampling and corresponding convolution operation on the picture to be detected according to the extracted feature information to obtain a feature map corresponding to the picture to be detected.

3. The method of claim 1 or 2,

the higher the confidence score of the pixel point belonging to the character pixel point is, the higher the probability of the pixel point belonging to the character pixel point is and the lower the probability of the pixel point belonging to the picture background is;

the smaller the confidence score of the pixel point belonging to the character pixel point is, the smaller the probability of the pixel point belonging to the character pixel point is and the larger the probability of the pixel point belonging to the picture background is.

4. The method according to claim 1 or 2, wherein extracting the position information of the pixel points in the preset confidence score range in the text box to which the pixel points belong comprises:

obtaining pixel points with confidence scores larger than preset scores from a plurality of pixel points obtained by segmentation;

and extracting the coordinate value of the pixel point with the value larger than the preset value in the character frame to which the pixel point belongs.

5. The method of claim 4, wherein the forming of the plurality of connected domains of the pixel points according to the extracted position information of the pixel points comprises:

judging whether any pixel point is cascaded with the pixel point in the appointed direction in the character frame to which the pixel point belongs according to the extracted coordinate value of the any pixel point;

if yes, the arbitrary pixel point and the pixel point in the appointed direction in the text frame to which the arbitrary pixel point belongs are cascaded;

and forming a plurality of connected domains according to the mutually cascaded pixel points, wherein the pixel points belonging to the same connected domain are mutually cascaded.

6. The method according to claim 1 or 2, wherein the forming of the plurality of connected domains of the pixel point according to the extracted position information of the pixel point comprises:

and carrying out binarization processing on the feature map, and forming a plurality of connected domains of the pixel points in the image after binarization processing according to the extracted position information of the pixel points.

7. The method according to claim 1 or 2, wherein merging the pixels in the same connected domain, and determining the text region on the picture to be detected by using the merged pixels comprises:

merging the pixel points in the same connected domain, and positioning the position information of the corresponding text frame according to the position information of the merged pixel points;

and determining the character area on the picture to be detected according to the position information of the character frame.

8. A text detection device based on pixel segmentation and merging, comprising:

the generating module is suitable for extracting the characteristic information of the picture to be detected and generating a characteristic graph corresponding to the picture to be detected according to the extracted characteristic information;

the analysis module is suitable for carrying out pixel segmentation on the characteristic image to obtain a plurality of pixel points and analyzing the confidence score of each pixel point belonging to a character pixel point;

the forming module is suitable for extracting the position information of the pixel points within the preset confidence score value range in the text box to which the pixel points belong, and forming a plurality of connected domains of the pixel points according to the extracted position information of the pixel points;

and the merging module is suitable for merging the pixel points in the same connected domain, and determining the character region on the picture to be detected by using the merged pixel points.

9. A computer storage medium having computer program code stored thereon which, when run on a computing device, causes the computing device to perform the method of pixel segmentation merging based text detection of any one of claims 1-7.

10. A computing device, comprising: a processor; a memory storing computer program code; the computer program code, when executed by the processor, causes the computing device to perform the method for pixel segmentation merging based text detection of any one of claims 1-7.

Images