RU2828605C1 - Method of binding fields of conditionally rigid business documents during their recognition - Google Patents

Abstract

FIELD: methods of recognizing documents.

SUBSTANCE: in the method of associating fields of conditionally rigid business documents during their recognition on a digitized paper version of a document, graphic primitives are determined taking into account information on the mutual arrangement of graphic primitives in each reference document, namely minimum distances from the middle of each of the four edges of the reference document to the nearest point of each graphic primitive, minimum distances from each of the four corners of the reference document to the nearest point of each graphic primitive, based on the mutual arrangement of graphic primitives, the digital image of the current document is identified with the digital image of one of the reference documents stored in a predetermined database of reference documents. Further, the information is recognized and the formal correctness of filling the document is checked.

EFFECT: high quality of document recognition.

1 cl, 1 dwg

Description

The invention relates to computer science, namely to methods of document recognition.

The closest analogue of the claimed invention is a method for neural network control of text data on document images (patent for invention of the Russian Federation No. 2806012), according to which an input image of a text field is fed to the input, wherein it is known for the image that in the original document the text information on it has property A, the width of the text field image is also known, and the input image field in the RGB color space, containing the text field of the document, is processed by a neural network detector for control of text data according to the following algorithm: the image of the field is converted into a single-channel one, after which it is fed to the input of a trained fully convolutional neural network; at the output, the neural network associates for each vertical line corresponding to the middle of the receptive field, values and confidence estimates for two possible classes: a class for which the property in the text field is absent; class A, for which the property in the text field is present; calculate the sums and values of confidence estimates for two possible classes along all vertical lines of the text field image: check for the presence of an anomaly in the image, and if an anomaly is not found in the text field in question, then the image of the text field in question has property A, if an anomaly is found in the text field in question, then the image of the text field is considered to have property A in the presence of an anomaly, if the condition is met in other cases the image of the text field does not have property A. The disadvantage of this technical solution is its focus on taking into account only the text fields of the image, although many documents (primarily business documents intended for data exchange with organizations and individuals) also contain other primitives - QR codes, bar codes, static text words, segments (underlining lines), bar codes, marks (check boxes).

The technical objective of the claimed invention is to develop methods for automatic recognition of digital documents.

The solution to the technical problem is achieved by forming a database of reference documents containing information on the relative position of graphic primitives in the reference document and indications of the type of information contained in each graphic primitive: QR code, bar code, segment, bar code, type - text, digital or combined - and the language of the information in each check box; the paper version of the document is digitized, thereby forming a digital image of the document, graphic primitives are defined on the digital image of the document, which include QR codes, bar codes, static text words, segments, bar codes, check boxes; based on the relative position of graphic primitives, the digital image of the document is identified with the digital image of one of the reference documents stored in a pre-formed database of reference documents containing information on the relative position of graphic primitives in the reference document and indications of the type of information contained in each graphic primitive: QR codes, bar codes, segments, bar codes, type - text, digital or combined - and the language of the information in each check box; based on the results of the identification, the information in each check box is recognized; the formal correctness of filling out the document is carried out as the correspondence of the recognized information to the expected information; if the formal correctness is confirmed, then the document is considered to be successfully recognized and the information from the document is entered into the table “document number - graphic primitive - information contained in the graphic primitive”, otherwise - the document is sent for manual processing, and the information about the mutual arrangement of graphic primitives in each reference document stored in the database of reference documents contains: the minimum distances from the middle of each of the four edges of the reference document to the closest point of each graphic primitive, the minimum distances from each of the four corners of the reference document to the closest point of each graphic primitive.

The technical result achieved by the specified set of features consists in improving the quality of document recognition.

The developed method is aimed at improving the quality of recognition of conditionally rigid business documents by linking fields during their recognition. A conditionally rigid business document is a document that has a rigid structure, characterized by the presence of primitives with a clear location in the document, and the task of document recognition can be reduced only to the recognition of information contained in QR codes, bar codes, bar codes and check boxes, as well as in words of static text.

The implementation of the claimed method is as follows (see figure).

A digital document is defined as a set of fields and static information. Many documents are characterized by a relatively simple structure and a limited vocabulary of static texts. Document fields (recognition areas) are defined as an object that is limited by several static elements, such as words of static text, segments (underlining lines); bar codes, marks (check boxes).

Extraction of information from recognized business documents has a number of features: small volume of the dictionary of words of static text; possible significant number of recognition errors; possible errors in detection of graphic elements.

The formulation of the document recognition problem is as follows. Based on the recognition of text objects and found graphic primitives, find the boundaries of the fields (fill areas) and extract information from the field areas.

Document image recognition is implemented in the following stages:

normalization of the page image, including searching for a document area and converting it to a rectangular form;

word recognition;

extraction of graphic objects;

document type classification;

search for local features;

search for the boundaries of fields of a document of a known type using the boundaries of local features;

extracting or recognizing the contents of fields within the found boundaries using field attributes;

post-processing of recognized fields using dictionary models.

The criterion for the quality of the recognition task solution is the extraction of information from the boundaries of the maximum number of fields with the least number of errors for each field. The extracted information may be in the form of not only a set of characters, but also the boundaries of the found field.

In documents created using a rigid form, irreparable distortions may be introduced into the image of these documents during the printing and digitization process:

page jams, resulting in severe distortion of the geometric shape of the page and image areas;

the use of fonts and other small static elements, which leads to significant losses of points that could be taken as key (reference) points, etc.

A method for linking fields of conditionally rigid business documents during their recognition, characterized by the fact that:

on the digitalized paper version (digital image) of the document (information on the relative position of graphic primitives in each reference document stored in the database of reference documents contains: minimum distances from the middle of each of the four edges of the reference document to the closest point of each graphic primitive, minimum distances from each of the four corners of the reference document to the closest point of each graphic primitive) graphic primitives are determined, which include QR codes, bar codes, static text words, segments, bar codes, check boxes;

Based on the mutual arrangement of graphic primitives (angles and normalized distances between them), the digital image of the current document is identified with the digital image of one of the reference documents stored in a previously formed database of reference documents. The database of reference documents contains information on the mutual arrangement of graphic primitives in each reference document, as well as indications of the type of information contained in each graphic primitive: QR codes, bar codes, segments, bar codes, type - text, digital or combined - and the language of the information in each check box.

The current digital image of the document is identified with one of the images stored in the database, and based on the results of the identification, the information in each checkbox is recognized - a priori information about the information improves the quality of recognition.

Then the formal correctness of the document filling is checked as the correspondence of the recognized information to the expected information.

If the formal correctness is confirmed, then the information from the document is entered into the table “document number - graphic primitive - information contained in the graphic primitive”, otherwise the document is sent for manual processing.

The quality of the claimed method was tested on our own test dataset containing 418 images of a conventionally rigid document. The documents were printed on A4 sheets and digitized by mobile device cameras under various lighting and shooting conditions. During the digitization process, the document images were subjected to projective distortions and nonlinear deformations of the sheets.

As an alternative recognition method, the Smart Document Engine SDK (https://smartengines.com/) was used.

As a result, it was shown that the average accuracy of recognizing one mark increased from 87.85% to 88.94%) - that is, on the test dataset, the proportion of mark recognition errors decreased by more than 2 times.

This proves the achievement of the stated technical result.

Claims (1)

A method for linking fields of conditionally rigid business documents during their recognition, characterized by the fact that: a database of reference documents is formed, containing information on the mutual arrangement of graphic primitives in the reference document and with indications of the type of information contained in each graphic primitive: QR code, barcode, segment, barcode, type: text, digital or combined, and the language of the information in each checkbox; the paper version of the document is digitized, thereby forming a digital image of the document, graphic primitives are determined on the digital image of the document, which include QR codes, barcodes, words of static text, segments, barcodes, checkboxes; based on the mutual arrangement of graphic primitives, the digital image of the document is identified with the digital image of one of the reference documents stored in a pre-formed database of reference documents containing information on the mutual arrangement of graphic primitives in the reference document and with indications of the type of information contained in each graphic primitive: QR codes, barcodes, segments, barcodes, type: text, digital or combined, and the language of the information in each checkbox; based on the results of the identification, the information in each checkbox is recognized; the formal correctness of filling out the document is carried out as the correspondence of the recognized information to the expected information; if the formal correctness is confirmed, then the document is considered to be successfully recognized and the information from the document is entered into the table “document number - graphic primitive - information contained in the graphic primitive”, otherwise the document is sent for manual processing, and the information about the mutual arrangement of graphic primitives in each reference document stored in the database of reference documents contains: the minimum distances from the middle of each of the four edges of the reference document to the closest point of each graphic primitive, the minimum distances from each of the four corners of the reference document to the closest point of each graphic primitive.

Images